Having tried both of them a bit, LWJGL seems to be a lot faster, but in Java3D I think there were classes for collison testing and so on. Which do you prefer?

it's not really apples to apples comparison.
lwjgl enables raw opengl access, while java3d is a scenegraph build on top of opengl/directx.
You should compare jMe and Xith3d to java3d instead, which should be faster in most cases.

There was a benchmark on JGO about the speed in a Quake BSP viewer. The fastest was a custom lwjgl renderer followed by java3d (quite surprising). However some optimizations were made to jMe which made it quite a bit faster.

http://www.javagaming.org/forums/index.php?topic=11748.0