i need information about capability of webemthods

[anilellendula]

Hi All,

Am using webMethods 7.1.2 in unix oprating system, and oracle is my database.

i want to insert the 3million records at time using JDBC Adters and i also nee pick the 17 mb and 32 mb file using FTP, i need information that webMethods can able to insert/select the 3 million records from database.

if yes to reduce the time what are steps need to take and also update the steps handle the 17mb and 32mb flatfile by FTP.

i have 32GB of RAM and CPU is 1500MHz

and in JVM i have set the heap memory as

JAVA_MIN_MEM=4096M
JAVA_MAX_MEM=6144M
JAVA_MEMSET=“-ms${JAVA_MIN_MEM} -mx${JAVA_MAX_MEM}”

with this settings i can’t able to pick the 17mb flatfile by FTP.

and getting OutOfMemmory error for 3 million records for insertion/selection from oracle table.

and also getting same OutOfMemmory for 1 million records for insertion/selection from oracle table.

just am using insert and select adapter service in flow service.

In databse connection i have given Minimum POOl size is 0
and MAX POOL size is 3000 and also tried with 5000 and 1000 size, no sucess on this, please any one suggest me capability of webMethods on this, we have the requirement to handle the 17 mb and 32 mb flatfile BY FTP it has 1million/3 million records in that and need to insert in the Oracle Database.

and also tried with select adapter service for selecting 1million recoreds/3millions records from ORacle Dtabase and insert adapter service to insert same records in another Oracle Database.

Regards,
Anil Kumar ellendula

[shahid2 (shahid *)]

how abt using using ETL tools for this task?

[reamon]

Yes, Integration Server can be used to perform this task. As the other poster mentioned, with that data volume it may be more appropriate to use an ETL tool.

The key in doing this with Integration Server is to avoid loading the file completely into memory.

Use node iteration.
Use batch inserts.
The number of DB connections you mentioned is far too high. The processing won’t use that many connections.

You’ll want to consider transactional support and restart capability when the process stops in the middle for any reason.

[anilellendula]

thanks for reply reamon,it is very difficult to go for ETL tool at this moment, we have changed the design, am getting flat file…

deigned with below steps.

1. taking the flatfile from share folder and ftp’ing the file to remote server box by FTP, remote server box has oracle scripts, reads the data insert in the export tables, webMethods need to have schedule service get the data from export table and insrt in to actual tables by custom sqls.

this is the solution.

regards,
anil kumar ellendula

[adityarams]

Hi All,

Instead of export tables, directly we can use the actual target tables ? Any Suggestions

Thanks & Regards,
Kumar