I would evaluate several things before making this decision and it sounds like you are already on that track with your options.
-Migrating from Windows to Unix is a big step depending on your organization and its support capabilities. I am a big fan of Unix but if you are primarily a Windows shop, I would weigh that. Are you using any of the Windows specific features that the Integration Server offers?
-webMethods product line is fully supported on Windows meaning everything they make runs on Windows and it is usually their development platform. Solaris is their second platform that almost all of their product line runs on, they don’t port the developer any more in 6.5 to Solaris. We run Solaris at my shop, very robust and very stable. The other platforms don’t have all of the options, I would check their platform guide to see what components run on what platforms. For example, the Mainframe adapter doesn’t run on Linux.
-64bit JVM is good is you have memory intense applications like maybe greater than 2GB. There are some more performance gains with threads in some applications. I would check and see if webMethods has done any performance testing with the 64bit JVM’s. The latest performance papers I saw from webMethods using the 6.5 product line had Windows 32 bit doing very well performance wise.
-If you are looking to the future and the Operating System war winners, most analyst are predicting Windows, Linux and Solaris. If it were me, I would look at my shop and see where those fit. HPUX is a very good OS but HP doesn’t seem to know what to do with it.
-If you are having performance issues I would look at the root cause of the issues to determine if it is a logic or flow service issue, memory issue, cpu issue, disk issue or OS. More than likely its going to be logic/flow service(Integration Server), memory, cpu or disk. The OS causing the performance is less likely although possible. Adding memory or faster disk is pretty easy, CPU’s cost a lot because of the webMethods licensing structure.