Personally, I wouldn’t expect to see a big performance boost by installing multiple IS’s on the same box. They’re sharing the same system resources, so I don’t see much value-add there. However, I do think you’re on the right track looking at system configuration changes.
You should be aware of things like kernel parameters, OS patches, and (as you noted) JVM versions and settings. While the choice of OS itself may have some performance implications, you can most likely tune your selected OS appropriately to get it in line with your requirements. As a general recommendation, I’d suggest you check out some of the performance and scalability whitepapers on Advantage. They should give you a good idea of what each OS vendor has done to optimize IS performance on their platform.
I have heard that the 1.4.1 JVM is supposed to have some nice performance tweaks, but can’t attest to that personally yet. Certainly, you’ll want to be sure that you’re starting whatever JVM you use with the right parameters, especially making sure you’ve allocated enough RAM to the JVM.
Lastly, there are configuration settings within IS (not involving code) that can dramatically impact your performance. For example, if you’re using TN you’ll most likely want to ensure you’re not saving the entire contents of inbound documents. You should also turn logging down to a minimum. Validating XML inputs unnecessarily can slow you down. Etc. HTH.