RFE - Add concurrent HTTP requests for resource information like modification date, content length, etc (used by the progress bar when downloading the files, etc).
When you have only only one jar as a dependency, it doesn't matters, but when you have multiple jars, it becomes really faster to request each jar information using multiple threads.
One problem is that the progress bar can't be easily updated when using multiple threads.
I have attached a zip with a project with a test for using multiple threads for requesting the jars info, as an example of how it works.
looks good. I'll look into adding this, and some configuration.
needs a bit of work - looks like some weekend work :)
I could provide a clean patch for that if you want :D
I would appreciate that. Basically, I was looking to add a concurrentLookupThreads (param: al_lookup_threads), and then change the getJarInfo's for loop to use a thread pool of max(1, concurrentLookupThreads) to look it up.
Well, it is done, but on the way of making it, I discovered (you may already know it) that getJarInfo is not failing if the file doesn't exists on the server, urlconnection.getContentLength() is returning -1 and getLastModified() is returning 0 in that case. Dunno if that is the expected behavior, only to take note.
I have used as you propose a variable concurrentLookupThreads with default value 1, and added a new parameter al_lookup_threads, I have also tested it using our game server to see speed improvements, but please verify the code and test it if you can before applying.
The same thing for downloading jar information using multiple threads to improve speed a bit, could be applied for downloading all resources as well.
P.S.: if you apply the patch, you could say it was made by ruben01 :P
Quote from: arielsan on February 15, 2011, 13:26:50
I discovered (you may already know it) that getJarInfo is not failing if the file doesn't exists on the server, urlconnection.getContentLength() is returning -1 and getLastModified() is returning 0 in that case. Dunno if that is the expected behavior, only to take note.
Its expected behaviour, not all servers support content length or last modified (its rare but they do exist). Anyway failure will occur once the file is attempted to be downloaded and that is where it will be caught.
I know that probably nobody is using the concurrent lookup stuff but I am, I have updated the patch to work with the latest LWJGL AppletLoader version, if someone is interested, the patch is attached.
Pretty cool stuff, patch is now applied, also tweaked progress bar stuff so that it updates correctly.
Do test and confirm all is well.
Lastly, the thread pool currently defaults to 1, any harm in bumping it to 3 as default?
Thx
fwiw, keeping it at 1 means default behavior is similar to pre-concurrent. Which means people would need to activate it themselves.
I agree with Matzon about keeping the default value at 1, however, that doesn't mean it will do the same logic as it did before.
Testing it right now.
Thanks for applying the patch.