Coherence can really accelerate and improve your application because it's fast, high available, easy to setup and it's scalable. But when you even use it together with the JCache framework of Java 8 or the new Coherence Adapter in Oracle SOA Suite and OSB 12c it will even be more easier to use Coherence as your main HA Cache.
Before Coherence 12.1.2 when you want to use Coherence together with JPA for the database connectivity, you must make sure that there is no batch job or application doing modifications directly in the database. This will lead to an out of sync Coherence Cache. But with Coherence 12.1.2 together with GoldenGate you can capture these database changes and send updates to the Coherence Cache. This is called Coherence HotCache.
Here you can see how it basically works.
And how it works in GoldenGate. First GoldenGate will capture all the database changes and a datapump process will send the trails to a remote GoldenGate for Java client which will update the Coherence Cache.
In this blogpost you can follow all the steps to setup your own Coherence HotCache Cluster
I also made Vagrant / VirtualBox environment which uses Puppet to create a WebLogic 12.1.2 cluster together with GoldenGate for Java 11.2.1 and also an Oracle 11.2.0.4 Database with GoldenGate 12.1.2. You only need to download your licensed software and start it up.
In this example the Coherence HotCache uses the Oracle Database HR demo schema and I also made with OEPE 12.1.2 a Coherence JPA application which we will deploy to the Dynamic WebLogic 12.1.2 Coherence Cluster.
First we need to have a working database with a WebLogic Cluster. After this you can configure GoldenGate 12.1.2 on the database server and GoldenGate Java Adapters version 11.2.1 ( V38714-01.zip downloaded from EDelivery ) on the WebLogic Admin Server machine.
Database Configuration
Next step is to configure GoldenGate on the database server.
We will create a new goldengate admin user, enable database archiving and unlock HR schema user
Make a new archive folder for the TEST database
su - oracle mkdir -p /oracle/archives/test
Allow the oracle user to generate spool files in the GoldenGate home
su - ggate chmod 775 /oracle/product/12.1.2/ggate
Log in as oracle and go the goldengate home
su - oracle export ORAENV_ASK=NO; export ORACLE_SID=test; . oraenv cd /oracle/product/12.1.2/ggate/ sqlplus /nolog connect / as sysdba
SHUTDOWN IMMEDIATE; STARTUP MOUNT; ALTER DATABASE ARCHIVELOG; ALTER DATABASE OPEN; alter system set log_archive_dest_1='LOCATION=/oracle/archives/test' scope=both; alter system set ENABLE_GOLDENGATE_REPLICATION=true scope=both; alter system set undo_retention=86400 scope=both;
the ENABLE_GOLDENGATE_REPLICATION parameter is only for Oracle Database 11.2.0.4 or higher, this allows me to use ADD SCHEMATRANDATA in GoldenGate.
Unlock the HR demo schema
alter user hr account unlock; alter user hr identified by hr;
Add supplemental log data
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA; ALTER DATABASE FORCE LOGGING; ALTER SYSTEM SWITCH LOGFILE;
Check if everything is Ok for GoldenGate
SELECT supplemental_log_data_min, force_logging FROM v$database;
Create the goldengate admin user and grant him the necessary rights
create tablespace ggate
logging
datafile '/oracle/oradata/test/ggate01.dbf'
size 32m
autoextend on
next 32m maxsize 2048m
extent management local;
CREATE USER GGATE_ADMIN identified by GGATE_ADMIN
DEFAULT TABLESPACE ggate
TEMPORARY TABLESPACE temp
QUOTA UNLIMITED ON GGATE;
GRANT CREATE SESSION, ALTER SESSION to GGATE_ADMIN;
GRANT ALTER SYSTEM TO GGATE_ADMIN;
GRANT CONNECT, RESOURCE to GGATE_ADMIN;
GRANT SELECT ANY DICTIONARY to GGATE_ADMIN;
GRANT FLASHBACK ANY TABLE to GGATE_ADMIN;
GRANT SELECT ON DBA_CLUSTERS TO GGATE_ADMIN;
GRANT EXECUTE ON DBMS_FLASHBACK TO GGATE_ADMIN;
GRANT SELECT ANY TRANSACTION To GGATE_ADMIN;
GRANT SELECT ON SYS.V_$DATABASE TO GGATE_ADMIN;
GRANT FLASHBACK ANY TABLE TO GGATE_ADMIN;
EXEC DBMS_GOLDENGATE_AUTH.GRANT_ADMIN_PRIVILEGE('GGATE_ADMIN');
/
Add GoldenGate repository tables and enable the capture of the DDL changes.
Give GGATE_ADMIN as input to the following GoldenGate scripts ( marker_setup, ddl_setup & role_setup )
@marker_setup.sql @ddl_setup.sql @role_setup.sql GRANT GGS_GGSUSER_ROLE TO GGATE_ADMIN; @ddl_enable
Grant select on all the HR tables to the GoldenGate schema user
GRANT SELECT ON HR.REGIONS to GGATE_ADMIN; GRANT SELECT ON HR.DEPARTMENTS to GGATE_ADMIN; GRANT SELECT ON HR.JOBS to GGATE_ADMIN; GRANT SELECT ON HR.EMPLOYEES to GGATE_ADMIN; GRANT SELECT ON HR.JOB_HISTORY to GGATE_ADMIN; GRANT SELECT ON HR.COUNTRIES to GGATE_ADMIN; GRANT SELECT ON HR.LOCATIONS to GGATE_ADMIN;
This finishes our database part
Coherence Application
When the WebLogic Dynamic Cluster is active we can deploy the HR Coherence demo application to the WebLogic Cluster and test if it works.
In my case the cluster nodes are running on the following IP addresses 10.10.10.100 and 10.10.10.200 and I have a Employee and Department Cache which are connected with JPA to the employees and departments tables of the HR demo schema.
You can find the Eclipse workspace project here. This workspace contains the following projects
- HrClient, The client test project which uses wls12_remote_cache.xml to connect to the coherence cluster nodes.
- HrModel, JPA project with the entities and the HrModel persistence unit.
- HrHotCache, which is the Coherence project and will be added as a Grid Archive (GAR)
- HrHotCacheWeb, a dummy Web Application project.
- HrHotCacheEAR, Generates an Ear which the GAR, WAR and JPA jar
The persistence.xml in the HrModel project has a resource entry called HrModel which uses test.oracle.com as service name and connects to 10.10.10.5 ( location of the Database server )
To test Coherence we need to have a Coherence client file which defines the cache entries and all the addresses of the Coherence nodes. We will also use this Coherence client file in GoldenGate
a Java class to retrieve department with 10 as id.Employee CustomRemoteCacheScheme Department CustomRemoteCacheScheme CustomRemoteCacheScheme CustomExtendTcpCacheService 10s 10.10.10.200 9099 10.10.10.100 9099 5s 500ms 5s
package test;
import com.tangosol.net.CacheFactory;
import com.tangosol.net.NamedCache;
public class Department {
public static void main(String[] args) {
NamedCache department = CacheFactory.getCache("Department");
model.Department dept = (model.Department) department.get(10L);
System.out.println("Department: "+dept.getDepartmentName());
}
}
Run the Department class with an Oracle Coherence run profile ( use this coherence client file as input) and test if it works.
GoldenGate for Java configuration
Next step is to configure GoldenGate 11.2.1 for Java on the WebLogic 12.1.2 AdminServer.
Log in as the goldengate user, change directory to the goldengate home
su - ggate
cd /opt/oracle/ggate_java
export JAVA_HOME=/usr/java/jdk1.7.0_45
export PATH=${JAVA_HOME}/bin:${PATH}
export LD_LIBRARY_PATH=${JAVA_HOME}/jre/lib/amd64/server:${LD_LIBRARY_PATH}
For the GoldenGate java delivery we need to have a property file called hr-cgga.properties and add this to the dirprm folder of your GoldenGate home ( /opt/oracle/ggate_java/dirprm/hr-cgga.properties )
In the jvm.bootoptions I made the following changes:
- My WebLogic 12.1.2 Middleware home is /opt/oracle/middleware12c, you probably need to change this to your own middleware home
- The jar which contains the entities and the persistence.xml is and this is an export of the Eclipse HrModel project.
- Log4j property file, Here is a link to my Log4j properties file
- HrModel as the toplink.goldengate.persistence-unit value
- The Coherence client file, which is the same as we used to test the Coherence Cache, Here is a link to my coherence client file
# ==================================================================== # List of active event handlers. Handlers not in the list are ignored. # ==================================================================== gg.handlerlist=cgga # ==================================================================== # Coherence cache updater # ==================================================================== gg.handler.cgga.type=oracle.toplink.goldengate.CoherenceAdapter # ==================================================================== # Native JNI library properties # ==================================================================== goldengate.userexit.nochkpt=true goldengate.userexit.writers=jvm # ====================================== # Java boot options # ====================================== jvm.bootoptions=-Djava.class.path=dirprm:ggjava/ggjava.jar:/vagrant/hr/bin/hrmodel.jar:/opt/oracle/middleware12c/oracle_common/modules/oracle.jdbc_11.2.0/ojdbc6.jar:/opt/oracle/middleware12c/coherence/lib/coherence.jar:/opt/oracle/middleware12c/oracle_common/modules/javax.persistence_2.0.0.0_2-0.jar:/opt/oracle/middleware12c/oracle_common/modules/oracle.toplink_12.1.2/eclipselink.jar:/opt/oracle/middleware12c/oracle_common/modules/oracle.toplink_12.1.2/toplink-grid.jar -Xmx32M -Xms32M -Dtoplink.goldengate.persistence-unit=HrModel -Dlog4j.configuration=/vagrant/hr/log4j-default.properties -Dtangosol.coherence.distributed.localstorage=false -Dtangosol.coherence.cacheconfig=/vagrant/hr/client-cache-config.xml -Dtangosol.coherence.ttl=0
Next we can configure GoldenGate on the WebLogic AdminServer, this GoldenGate will receive the trail from the GoldenGate client located at the Database Server.
./ggsci CREATE SUBDIRS
Configure the manager.
status mgr stop mgr EDIT PARAMS MGR
Add the following content
PORT 16100 DYNAMICPORTLIST 16110-16120, 16130 AUTOSTART ER * AUTORESTART ER *, RETRIES 4, WAITMINUTES 4
Start the manager
start mgr status mgr
Configure the Java delivery process which will connect to the Coherence Cache
EDIT PARAMS HR-CGGA
Add the following content
EXTRACT HR-CGGA SETENV ( GGS_USEREXIT_CONF = "dirprm/hr-cgga.properties" ) SETENV ( GGS_JAVAUSEREXIT_CONF = "dirprm/hr-cgga.properties") SOURCEDEFS dirdef/hr.def CUserExit libggjava_ue.so CUSEREXIT PassThru IncludeUpdateBefores GETUPDATEBEFORES NoTcpSourceTimer Table hr.*;
We will generate and copy the hr.def definition at a later time This process will listen to the jj trail which will be deliverd by the GoldenGate client on the database server
DELETE EXTRACT HR-CGGA ADD EXTRACT HR-CGGA, EXTTRAILSOURCE dirdat/jj exit
We will start the Java delivery process after the database GoldenGate configuration.
GoldenGate 12.1.2 configuration on the Database server
log in as ggate and change directory to the GoldenGate home
su - ggate cd /oracle/product/12.1.2/ggate/ export ORAENV_ASK=NO; export ORACLE_SID=test; . oraenv
Initial GoldenGate configuration
./ggsci ADD CREDENTIALSTORE ALTER CREDENTIALSTORE ADD USER GGATE_ADMIN, PASSWORD GGATE_ADMIN, ALIAS gg1 ADD MASTERKEY gg1 CREATE WALLET OPEN WALLET ADD MASTERKEY INFO MASTERKEY ALL exit
I did a silent install of GoldenGate 12.1.2 and this install will also start the manager plus create all the required GoldenGate directories
./ggsci # mgr status mgr stop mgr EDIT PARAMS MGR
Add the following content
PORT 16000 DYNAMICPORTLIST 16010-16020, 16030 AUTOSTART ER * AUTORESTART ER *, RETRIES 4, WAITMINUTES 4 STARTUPVALIDATIONDELAY 5 USERIDALIAS gg1 PURGEOLDEXTRACTS dirdat/*, USECHECKPOINTS, MINKEEPHOURS 2
Start the manager
start mgr status mgr
Next step is to configure a Classic Capture Extract
EDIT PARAMS HRTEST
Here I also do something extra like adding DDL capture and because the other GoldenGate client is not a 12.1.2 client I need to set the format release to 11.2
Add the following content
EXTRACT HRTEST USERIDALIAS gg1 LOGALLSUPCOLS DDL INCLUDE MAPPED EXTTRAIL dirdat/st, FORMAT RELEASE 11.2 SEQUENCE hr.*; TABLE hr.*; BR BROFF getUpdateBefores TranLogOptions excludeUser hr
Add the extract and remove some old configuration or trails because I use an Oracle 11.2.0.4 Database and enabled the ENABLE_GOLDENGATE_REPLICATION init parameter so I can use ADD SCHEMATRANDATA hr.
START MGR DBLOGIN USERIDALIAS gg1 STOP EXTRACT HRTEST DELETE EXTRACT HRTEST ADD SCHEMATRANDATA hr ADD EXTRACT HRTEST, TRANLOG, BEGIN NOW SHELL rm -f dirdat/st* ADD EXTTRAIL dirdat/st, EXTRACT HRTEST start HRTESTYou can check the output in dirrpt/HRTEST.rpt or do
./ggsci info all
Now we can add a datapump which will send the trail to the GoldenGate Java adpater
EDIT PARAMS PJAVA
Here I will connect to adminwls.example.com and also need to set the expected format to 11.2
Add the following content
EXTRACT PJAVA USERIDALIAS gg1 RMTHOST adminwls.example.com, MGRPORT 16100 RMTTRAIL dirdat/jj, FORMAT RELEASE 11.2 PASSTHRU GETUPDATEBEFORES TABLE hr.*;
Configure the datapump
DBLOGIN USERIDALIAS gg1 STOP EXTRACT PJAVA DELETE EXTRACT PJAVA ADD EXTRACT PJAVA, EXTTRAILSOURCE dirdat/st ADD RMTTRAIL dirdat/jj, EXTRACT PJAVA, megabytes 100 START EXTRACT PJAVA
You can check the output in dirrpt/PJAVA.rpt or do
./ggsci info all
Generate the HR defintions and copy this definition to the GoldenGate Java Adapter configuration
Make a new hrdefgen.prm file and add the following content
DEFSFILE ./dirdef/hr.def PURGE FORMAT RELEASE 11.2 USERIDALIAS gg1 TABLE hr.*;
Generate the HR table definitions
./defgen paramfile /vagrant/hrdefgen.prm
Copy /oracle/product/12.1.2/ggate/dirdef/hr.def to /opt/oracle/ggate_java/dirdef/hr.def ( located on the WebLogic AdminServer)
Start the Java delivery process
Go the GoldenGate Java Adapter home
Look at the output, it should load the dirdat/jj trail and connect to coherence cluster
su - ggate
cd /opt/oracle/ggate_java
export JAVA_HOME=/usr/java/jdk1.7.0_45
export PATH=${JAVA_HOME}/bin:${PATH}
export LD_LIBRARY_PATH=${JAVA_HOME}/jre/lib/amd64/server:${LD_LIBRARY_PATH}
./extract pf dirprm/hr-cgga.prm
Next time you can do it from ggsci and use start HR-CGGA
Test the HotCache configuration
Use the Coherence Test client to retrieve department 10 again.
Connect to the Oracle database and don't use the HR schema user ( very important, changes made by HR will be ignored cause coherence will also use this user to connect to the database). Update the department name of department 10.
Wait a few seconds and use the Coherence Test client again to retrieve department 10 and look if the department name has changed.
Publish the HR Database changes to a JMS Queue
As an extra we can also publish the HR database changes to a JMS Queue by creating a new property file called hr-oggq.properties and add this to /opt/oracle/ggate_java/dirprm
### oggq.properties ### gg.handlerlist=oggjms ### Path to WebLogic jars ### gg.classpath=/usr/java/jdk1.7.0_45/lib/tools.jar:/opt/oracle/middleware12c/wlserver/server/lib/weblogic_sp.jar:/opt/oracle/middleware12c/wlserver/server/lib/weblogic.jar:/opt/oracle/middleware12c/wlserver/server/lib/webservices.jar: ### JNDI properties java.naming.provider.url=t3://adminwls.example.com:7001 java.naming.factory.initial=weblogic.jndi.WLInitialContextFactory java.naming.security.principal=weblogic java.naming.security.credentials=weblogic1 ### JMS Handler gg.handler=oggjms gg.handler.oggjms.type=jms gg.handler.oggjms.format=xml2 gg.handler.oggjms.format.mode=op gg.handler.oggjms.destinationType=queue gg.handler.oggjms.destination=HR-Queue gg.handler.oggjms.connectionFactoryJndiName=HR-CF ### native library config ### goldengate.userexit.nochkpt=TRUE goldengate.userexit.timestamp=utc goldengate.log.logname=cuserexit goldengate.log.level=DEBUG goldengate.log.tofile=TRUE goldengate.userexit.writers=javawriter javawriter.stats.display=TRUE javawriter.stats.full=TRUE
Next we go back to ./ggsci and add this JMS extract and this will watch the same audit trail.
EDIT PARAMS HR-OGGQ
add the following content
EXTRACT HR-OGGQ SOURCEDEFS dirdef/hr.def CUserExit libggjava_ue.so CUSEREXIT PassThru IncludeUpdateBefores GETUPDATEBEFORES Table hr.*;
Add the extract
DELETE EXTRACT HR-OGGQ ADD EXTRACT HR-OGGQ, EXTTRAILSOURCE dirdat/jj
Exit ggsci and start this JMS extract outside ggsci, so we can see all the log output
./extract pf dirprm/hr-oggq.prm
