When the mouse X and Y values are clipped to the window area, getDX/DY return abnormally large numbers.
As an example of this, extract this ZIP file (http://www.ra4king.com/public/Graph3D-Bug.zip), double click the Graph3D.jar file, and hold and drag the mouse to outside the window boundaries. You should see the graph rotate wildly.
The fix has been identified and a patch is provided (http://www.ra4king.com/public/Mouse.java.patch). :)
Patch committed, should be part of next nightly build.
Thanks.