I need quick help with solution to fix my sag WM code logic built in WM 9.9 where i need to built code logic to sync data from DB1 to DB2 where as
data which is getting sync from DB1 to DB2 is very huge it will start from 35,000 records/per hours to million records
Here is logic written to sync data from DB1 to DB2 where we have WM 9.9 running with 2 Stand alone node and connected to 3 UMs which are in cluster nodes
Flow :-
DB1 → Publish service (only used custom built basic notification) → Trigger → UM → Subscribe Service → select/insert/update/delete → DB2
We have used basic notification to pick up data from buffer table from DB1 which is listen by listener and publish document is subscribe by trigger > where
in subscribe side service will do processing in UM and used select/insert/update/delete records into DB2
In Basic notification we have checked such as Extact Once Notification 2. Record ID 3. Mark ID (which will update record as Y to prevent duplication) 4. Max Row 10,000 5. Query Time out -1.
So Issue start here when record get pick up from basic notication as a sag built in feature it will update record Y which is taking so much time
to process 35,000 records for 1hr where when we change in basic notication without markup id it will process with in 2 min time frame.
So i need solution to built cutome code logic or what are the other ways or other options i have in publish service side which prevent to do the duplication of records as well as process of 35,000 records with in bench mark of 10-15 mins
Kindly reply back with publish service logic which help to resolve my ongoing issue
is it really neccessary to process the data via wM?
It might be better to have the 2 DB do the sync by their own.
Please provide some more details about your environmnent, i.e. database type and version as well as JDBC Adapter version with Fix-Level and Fix-Level for IS and ART.
My customer has recently purchase sag webMethods product so they want to get read of point to point architure so there is no second thought to remove WM product suite or by pass this middle-ware.
Database Oracle 11g and JDBC adapter 9.0 with Fix 8 and IS 9.9 Red hat linux server platform
IS_9.9_Core_TestPatch_PIE-44390_v2, IS_9.9_Core_TestPatch_PIE-44089_v1 and IS_9.9_Core_Fix6
2 stand alone Integration Servers with 3 UM are in clustered
By applying this IS and JDBC Fix will it resolve my publish service which used (built in basic notification) issue ?
because data which get pick up by basic notification with exact once notification with mark up id to prevent duplication records (this publishing service creating problem and taking so much time to process records)
It will be very difficult to analyse further without these fixes as you will have to try to find the root cause (i.e. network or database issue) why it is taking so much time.
Additionally when opening an incident at SAG Support the first thing they will ask you is to apply these fixes and test again.
Why not consider cleaning up the data in DB1 (get rid of duplicate records)first using SQL before using webMethods to move records from DB1 to DB2 ?
webMethods may not be a best in class (suitable) tool of choice to perform all sorts of ETL related functionalities…