Database events - Part 2 - polling an events table with apache camel

The problem with approache discussed in part one is that it's good for one table. Assuming we have an ERPsystem with more than 1000+ tables, and we need t omonitor 300 of these tables. Would polling in this case be acceptable ?

The second approache still uses polling a single table. Let's call it "the events table". We will use triggers to log transactions to the events table. Each inserted record will have a boolean field "processed" indicating that it was read by apache camel . Our example is done with postgresql. However the idea is applicable to other relational database engines. The next SQL script is a simpliefied version of Audit Trigger. Save it in a file "postgresql.events.sql"

Now in order to use this script, we need to:

Load it

Create the events table

Monitor a table

Modify the java code from part 1 to read this table

The follwing script should handle this part.

Now all we need to do is to update our camel route to read from this table and possibly to update the field "processed" to true if we don't want to use the stateful scheduler in quarz2.

Adding an external JDK to gentoo

Oracle keeps on making it hard to install java directly. On gentoo system if you do:

neptune ~ # emerge oracle-jdk-bin

 * IMPORTANT: 6 news items need reading for repository 'gentoo'.
 * Use eselect news to read news items.

Calculating dependencies... done!

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) dev-java/oracle-jdk-bin-1.7.0.65
 * Fetch failed for 'dev-java/oracle-jdk-bin-1.7.0.65', Log file:
 *  '/var/tmp/portage/dev-java/oracle-jdk-bin-1.7.0.65/temp/build.log'
 *
 * Oracle requires you to download the needed files manually after
 * accepting their license through a javascript capable web browser.
 *
 * Download the following files:
 *   jdk-7u65-linux-i586.tar.gz
 * at 'http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html'
 * and move them to '/usr/portage/distfiles'
 *

>>> Failed to emerge dev-java/oracle-jdk-bin-1.7.0.65, Log file:

>>>  '/var/tmp/portage/dev-java/oracle-jdk-bin-1.7.0.65/temp/build.log'

 * Messages for package dev-java/oracle-jdk-bin-1.7.0.65:

 * Fetch failed for 'dev-java/oracle-jdk-bin-1.7.0.65', Log file:
 *  '/var/tmp/portage/dev-java/oracle-jdk-bin-1.7.0.65/temp/build.log'

Which means you need to hit their webpage directly and download the "exact" version of the jdk, then copy it to the specified directory, to proceed with the installation. This is really pain. Alternativley you can download the JDK directly, and place it any where you want. Now to make it avaialble through "java-config", there's additional work needs to be done.

• cd to /usr/share/java-config-2/vm
• Create a file indicating the name and version of the jdk you want to manage through java-config. For example oracle-jdk-1.7
• Copy the following into the file:

  VERSION="Oracle JDK 1.7.0_67" 
  JAVA_HOME="/opt/jdk1.7.0_67"
  JDK_HOME="/opt/jdk1.7.0_67"
  JAVAC="${JAVA_HOME}/bin/javac"
  PATH="${JAVA_HOME}/bin:${JAVA_HOME}/jre/bin"
  ROOTPATH="${JAVA_HOME}/bin:${JAVA_HOME}/jre/bin"
  LDPATH="${JAVA_HOME}/jre/lib/i386/:${JAVA_HOME}/jre/lib/i386/native_threads/:${JAVA_HOME}/jre/lib/i386/xawt/:${JAVA_HOME}/jre/lib/i386/server/"
  MANPATH="${JAVA_HOME}/man"
  PROVIDES_TYPE="JDK JRE"
  PROVIDES_VERSION="1.8"
  # Taken from sun.boot.class.path property
  BOOTCLASSPATH="${JAVA_HOME}/jre/lib/resources.jar:${JAVA_HOME}/jre/lib/rt.jar:${JAVA_HOME}/jre/lib/jsse.jar:${JAVA_HOME}/jre/lib/jce.jar:${JAVA_HOME}/jre/lib/charsets.jar"
  GENERATION="2"
  ENV_VARS="JAVA_HOME JDK_HOME JAVAC PATH ROOTPATH LDPATH MANPATH"
  VMHANDLE="oracle-jdk-1.7"
  BUILD_ONLY="FALSE"
  

  
  you will need to modify the path to point to your JDK.
• cd /usr/lib/jvm/
• Create a symbolic link to the home of you JDK. The link name should be exactely the same as in the field "VMHANDLE" in the file. For example:
  neptune jvm # ln -s /opt/jdk1.7.0_67/ oracle-jdk-1.7
• Test it with java-config -L. You should see your new JDK installed and you can select it with "java-config -S 1", then java -version.

eclipse seizure - 1

In some cases when you are trying to install a  eclipse plugin through their update manager, and you are on Linux, you may get something like:


Cannot complete the install because one or more required items could not be found. Software currently installed: Shared profile ....... etc.

It happened to me many times. What do to ?

Login as root, and install the updates you need. Big chance it may work. You may try to change the permissions for the base eclipse installation, but I didn't try this yet.

Back to Ant

Now with apache ivy, there's no need for maven, and I can easily rewrite Ant targets in xml without diving into extending maven. However, I still miss maven's "Convention over Configuration", and I hate reconfiguring every single war file, when using ANT+IVY combo. Gradle looked promising, but hey, when will I stop learning new build systems ? How many build system available and all of them address one single problem. If ant+ivy can do it, then they are good enough for me.

Plus, I was hit with gradle limitation, the third day I started using it and tried to extend it.

runnig task programmatically

To me, nothing is more transparent than ant and easier to customize, except those repetitive configurations.

May be it's the time for a simple antlib that saves me for writing those configuration again and again.