As some of you know, we have talked about replacing FMOD and DevIL, primarly because:
1) FMOD is 3.x, upgrading to 4.x is a lot of work
2) FMOD isn't open source, nor is it free for commercial usage
3) no 64 bit support (not that big a deal)
4) DevIL is somewhat buggy in its image support - depending on the libs installed
5) DevIL is a nuisance when it comes to building the native parts
6) DevIL has had no release since 2006, and seems somewhat unmaintained.
How this should be done has been somewhat of a discussion, but we talked about it in IRC today (please join us! #lwjgl on freenode).
http://echelog.matzon.dk/logs/browse/lwjgl/1193349600 - from [10:36:39]
FMOD
- New audio player thingy, that support loading and playing of WAV (I assume its PCM, since WAV is just a container), OGG (vorbis), MOD and XM support. Preferably more modules (s3c, it etc).
- streaming support
- Straightforward API
DEVIL
- Basic Image loader (using custom loaders, and fallback to ImageIO)
- Basic image manipulation (flip comes to mind).
I encourage everybody to join in and help on this task!
We will be distributing the replacement libs and devil/fmod for some time, before removing fmod and devil entirely. Please let us know of which functions you cannot live without in either of the two APIs.
All I want is to easily play and loop OGGs and stream them reliably. Jorbis does a good job of decoding OGGs but streaming them through OpenAL is woefully unreliable and I can't figure out why.
The API I put on top of OpenAL went some way to doing this stuff but because of the streaming problems I couldn't really recommend it to anyone :(
As for DevIL the only useful thing it brought to the table was JPEG loading without AWT. The other two useful formats are PNG and TGA but I never used either of them as I had a very simple proprietry format and compiled my images ahead of time. A bit of a pain in the arse so it'd be nice not to have to but as I pack all my PNGs into single textures anyway in a precompilation step it never really bothered me enough to moan.
Cas :)
For DevIL i can help with custom loader for TGA (RLE) and DXTn.
I use .it files for music, since they're the format my sound guy prefers. I will try and find out why and what advantages it has (although I have no idea how technically hard it would be to write something to play it back).
Oh, and another thing - fmod allows you to get callbacks when playing tracker formats when notes/events get triggered. I was planning a music reactive game which used this functionality so that'd be good to have. IIRC it even delayed the callbacks according on the state of the sound buffer so you'd get the callbacks exactly the same frame as when the sound became audible, rather than when they were buffered up.
Quote from: Orangy Tang on October 26, 2007, 14:02:42
Oh, and another thing - fmod allows you to get callbacks when playing tracker formats when notes/events get triggered. I was planning a music reactive game which used this functionality so that'd be good to have. IIRC it even delayed the callbacks according on the state of the sound buffer so you'd get the callbacks exactly the same frame as when the sound became audible, rather than when they were buffered up.
obviously you missed the part in the chat that said:
Quote[10:46:49] <Mazon> but we'd need a first stab at an replacement for 95% of the people and then take it from there
:P
as for module support, we would be limited to whatever 3rd party libraries are doing.
As a first slab i think the slick stuff is pretty good, does all the major image formats, it does use imageIO, but theres already a custom tga loader in there(pretty damn fast), also theres a nice png loader (which is faster than ImageIO) somewhere on JGO as well as a DXTn library floating around also on JGO, the rest of the formats can be replaced over time to become ImageIO free.
As for sound lwjgl_util package already has a wav file loader(could do with some streaming support), add with jorbis for ogg, and ibxm lbrary for .mod and .it support.
just a matter of collecting the stuff and wrapping it.
I just have a couple of thoughts on all of this:
1) Feel free to ignore anything I have to say. My C and C++ skills are all but dead, so I feel kinda bad that I can't really offer much in the way of hands-on help.
2) As I understand it, LWJGL is meant to be a wrapper to put the basic necessities of advanced game development into Java. With this in mind, you might want to be careful of what I call "scope creep" - IOW, don't try to make it a full blown game engine like jME or Xith3D. You're allowed to stick to the basics. ;D
3) Devil support can probably just be taken out. Not right away, of course, since there are people who currently rely on it. But really, using the built in Java image libraries is much cleaner, and removes a dependancy. There are a couple of things you might lose, but most of those can either be worked around, or even done with the Java2D library (e.g., image manipulation).
4) OGG, WAV, etc. - Once again, I wouldn't recommend trying to be a full blown engine, as it's kind of beyond the scope of LWJGL (as I understand it). HOWEVER, that said, I sure wouldn't mind a solid sound implementation (maybe in the Utils collection). I agree with Cas, in that OGG streaming is rather flakey. I use jorbis and still have had to put too many try-catches (to catch erroneous errors and fix them) into my openAL code to feel comfortable with it. I'd rather not have to deal with FMOD, as openAL is viable enough and free; I just want a nice, error free approach to using it.
5) refer to #1. ;)
oh, and... 6) I wouldn't bother with MIDI support. If someone wants it, they can implement it themselves in Java - I personally don't care for it though, as it doesn't use hardware, and things like volume control, speed, etc., all have to be funneled through Java's funny MIDI system.
Quote from: Matzon on October 26, 2007, 14:16:08
Quote from: Orangy Tang on October 26, 2007, 14:02:42
Oh, and another thing - fmod allows you to get callbacks when playing tracker formats when notes/events get triggered. I was planning a music reactive game which used this functionality so that'd be good to have. IIRC it even delayed the callbacks according on the state of the sound buffer so you'd get the callbacks exactly the same frame as when the sound became audible, rather than when they were buffered up.
obviously you missed the part in the chat that said:
Quote[10:46:49] <Mazon> but we'd need a first stab at an replacement for 95% of the people and then take it from there
:P
as for module support, we would be limited to whatever 3rd party libraries are doing.
I saw it :P I agree it's a somewhat obscure feature but it's something which isn't immediately obvious and would be a shame to loose.
I couldn't really care less about module support... truthfully it probably belongs in its own library anyway.
Cas :)
@elias4444: I agree on the feature creep danger, but:
1. We are specifically trying to _replace_ functionality already present, not augment it. Devil does all sorts of image formats and manipulation, fmod does streaming of various formats and more.
2. We're putting all this in optional libraries, so it shouldn't affect users who just want access to the basics (OpenGL, OpenAL, and input)
- elias
And how to call that "new" Package? LWJGL_IO? o_O
Sry, but i think some basic texture loading support like ImageIO could be done in the lib itself. It just have to be decided which format counts as basic texture format.
Uh oh, the Eliases are making lists! Watch out! :P
That's all fine. You won't hear me complain if someone else wants to write code that saves me from having to write it later. So, I guess thanks are in order. ;)
Do you think you'll write these functions on the C side? Or on the Java side? I'm doing a little research to see if the Java2D image filters are even close to being fast enough for texture manipulation. I'll get back with some numbers if I can can come up with something concrete.
Quote from: elias4444 on October 26, 2007, 19:18:48
Do you think you'll write these functions on the C side?
I hope not! :) My primary motivation for getting rid of devil and fmod is the fact that they're native code.
- elias
Quote from: elias4444 on October 26, 2007, 19:18:48
Or on the Java side?
Why not? Most code is allready done by some of us. And the only formats we need were TGA, BMP, PCX, DXTn. Its just how we add and arrange them for fast and easy usage.
Quote from: javalwjgl on October 26, 2007, 15:16:41
As a first slab i think the slick stuff is pretty good, does all the major image formats, it does use imageIO, but theres already a custom tga loader in there(pretty damn fast), also theres a nice png loader (which is faster than ImageIO) somewhere on JGO as well as a DXTn library floating around also on JGO, the rest of the formats can be replaced over time to become ImageIO free.
As for sound lwjgl_util package already has a wav file loader(could do with some streaming support), add with jorbis for ogg, and ibxm lbrary for .mod and .it support.
just a matter of collecting the stuff and wrapping it.
We've been looking at doing that, and if you're interested in helping out with that effort, please post about it on the Slick forums. I was responsible for the initially working on this (the TGA, DXTn stuff), but I didn't get around to all the file formats. I also don't think my DXTn stuff ever went in (http://slick.javaunlimited.net/viewtopic.php?t=216) (don't know why), but I'll ask Kev again about it.
http://slick.javaunlimited.net/viewtopic.php?t=438
I also remember the PNG code being thrown around, but I don't have it anymore, nor would I want to make that from scratch.
- Jon
Edit: Now I remember why. The JSquish library dependency. Not big, but still yet another dependency. It'll work for LWJGL though.
Quote from: Orangy Tang on October 26, 2007, 13:03:11
I use .it files for music, since they're the format my sound guy prefers. I will try and find out why and what advantages it has (although I have no idea how technically hard it would be to write something to play it back).
They are sequenced music files like MIDI, except that the samples are stored in the file rather than looked up in a wavetable or soundbank. This means that they sound the same (perfect) on every computer. They were used all over in video game consoles up until this current generation.
You can play back such files using this library.
http://www.geocities.com/sunet2000/
- Jon
>And the only formats we need were TGA, BMP, PCX, DXTn.
TGA, BMP and PCX are all raw formats. TGA is the most feature complete one... so, only that one is needed. RLE shouldn't be supported, because it's far worse than deflate (jar) or any other compression scheme you might be using (eg lzma).
To cover the usual game cases 3 formats are required:
1. TGA - simple hassle free raw format
2. DDS - covers different raw formats, DXTn flavors and other game specific stuff
3. JPG - lossy format which is easy to create (no CMYK, but gray scale and progressive should be eventually supported)
TGA is a pretty simple format. Implementing a loader for that shouldn't be much of an issue. For the most part it will be a matter of C&P, I guess.
DDS is quite complex, but fortunately it's sort of modular. "Sub formats" can be implemented as needed. Eg it would be a good idea to start with DXTn, then the vector stuff, then all raw modes, then cube maps etc.
As annoying as JPG is, it's widely used for games - especially in the downloadable game sector. Supporting it is crucial for adoption.
Surely png should be supported too? A bunch of pngs should be smaller than the zipped raw equivilents (although I have no data to back this up, so I really should try a quick test to see).
I know I use PNGs. They're easy to implement via ImageIO too.
So basically, the four essential formats are:
1. PNG, because it's ubiquitous
2. TGA, because it's simple
3. JPEG, because it's lossy and therefore very compressible
4. DXTn, because it's graphics card native
and what's more we don't really need to be able to write these formats, just read them.
As for music... well, I don't use FMOD, but I would really like something that reliably streams OGGs. Jorbis seems reliable enough as a decoder, the problem is really OpenAL being unreliable. Perhaps we could ask the AL devs why it's such a problem. Otherwise all that's needed are some really simple functions to wrap all the complexity and then add more later to mimic FMOD functionality.
Cas :)
Quote from: princec on October 30, 2007, 17:22:12
and what's more we don't really need to be able to write these formats, just read them.
Although if anyone does want to write to png files, they can pinch this handy bit of code. (http://www.javagaming.org/forums/index.php?topic=16290.0)
Hello, here's my take:
- Need to read PNG and write PNG or JPG.
Preferrably without using ImageIO to allow trimming down the Windows JRE. - Currently using DevIL to load PNG & JPG files, convert between pixel formats, upload to OpenGL, write PNG or JPG files (screenshots).
- Would probably do fine with TGA only - as long as it's fast enough.
- Not using FMod - shareware license is too restrictive.
- Don't care about modules.
- Need to load OGG and WAV and stream OGG music from disk
(with varying degrees of success [ hey Cas ;) ] ). I'm also using Jorbis for that now.
It would be very, very nice to see a small Java-only library that loads a few audio formats, can stream stuff from disk and takes care of basic channel management. Easy to write custom audio effect and loader plugins for it. Main requirement - 100% stable on all platforms. To achieve that it would probably need to bypass OpenAL channel ("Source") management and do it's own mixing. Can't imagine it taking up too much CPU on today's machines.
yoyo,
Cant really help with the texture stuff. I would really love to have a way of loading jpgs, for online downloading of a package.
I could spend some time on the audio side of things. Currently i load my music using jogg/jorbis (streaming). only my sound fx end up working with openAL. I can spend some time to tweek the music code and make streaming audio package(using alSourceQueueBuffers). Or did you want the sound to be independant of outside packages? waiting for a thumbs up.
I didn't mention PNG in my list, because it uses Deflate - the same algorithm ZIP/JAR is using. The only difference is that it may use filters (http://www.libpng.org/pub/png/pngintro.html#filters), which may improve the compression ratio. However, lzma usually performs better... yes, even with images which benefit a lot from filtering. (One of the few exceptions is the "extreme and unrealistic case" from the libpng page.)
Those filters don't help with pixel art. They help with photo-based textures, but PNG isn't a good choice for that kind of image data anyways. JPG and DXTn will usually yield way smaller files at acceptable quality.
A useful feature of PNG, which isn't covered by TGA are the indexed modes with one or several tRNS entries (that's bitmasked and RGBA palette respectively).
However, those raw modes are covered by DDS as well. That's why I think that PNG isn't worth the trouble.
If someone is willing to write it - great, but it shouldn't be a top priority. The other formats are far more important, because they have distinctive characteristics.
The reason for using png is that it is ubiquitous. The loader would probably allow any "loaders" to try and read it and fall back to ImageIO (pending any awt dependency issues). So in that case PNG would be supported too.
However, this replacement should be *fast*, and we therefor shouldn't even be sniffing filetypes and doing a pluing based architecture. A simple ImageData img = ImageLoader.load("/path/to/file", ImageLoader.FILETYPE_TGA); should suffice.
Of course, there might be a need for an ImageLoader.load(path, type, options). We just need to take a stab at it and see how it works out.
Any volunteers?
Quote from: Matzon on October 31, 2007, 08:06:19
The reason for using png is that it is ubiquitous. The loader would probably allow any "loaders" to try and read it and fall back to ImageIO (pending any awt dependency issues). So in that case PNG would be supported too.
However, this replacement should be *fast*, and we therefor shouldn't even be sniffing filetypes and doing a pluing based architecture. A simple ImageData img = ImageLoader.load("/path/to/file", ImageLoader.FILETYPE_TGA); should suffice.
Of course, there might be a need for an ImageLoader.load(path, type, options). We just need to take a stab at it and see how it works out.
Any volunteers?
I'd avoid FILETYPE_* and instead use the filename extension (".tga") to guess the filetype. Other than that, we'll need to support InputStreams too in case the data is loaded through ClassLoader.getResource().
- elias
yes to inputstream, no to guessing file type. ;D
I don't see the point in guessing the file type since the developer ought to know what file type is being used.
Alternatively we need to parse the path and then compare with a list of known lowercase or uppercase file types. By forcing a file type you are effectively removing some processor overhead and you also get an idea of what file types are supported.
Quote from: Matzon on October 31, 2007, 09:56:03
yes to inputstream, no to guessing file type. ;D
I don't see the point in guessing the file type since the developer ought to know what file type is being used.
Alternatively we need to parse the path and then compare with a list of known lowercase or uppercase file types. By forcing a file type you are effectively removing some processor overhead and you also get an idea of what file types are supported.
Still yes to guessing file type :). The processor overhead of looking up <lower case extension>->Plugin is negliable, and as for the developer knowing the file type, consider the case where the file name is configurable from e.g. an xml file. Then the developer need to do his own mapping from extension to type or the type needs to be in the configuration file. Besides, ImageIO doesn't use separate file type arguments in ImageIO.read().
- elias
Quote from: elias on October 31, 2007, 10:14:22
Still yes to guessing file type :). The processor overhead of looking up <lower case extension>->Plugin is negliable, and as for the developer knowing the file type, consider the case where the file name is configurable from e.g. an xml file. Then the developer need to do his own mapping from extension to type or the type needs to be in the configuration file. Besides, ImageIO doesn't use separate file type arguments in ImageIO.read().
How do you guess the type from an input stream? - without sniffing (which causes performance overhead)
Sniffing doesn't cause any measurable performance overhead. Open the file, set a mark, read in enough signature until something's recognised, reset the mark, hand over the stream to the appropriate sub-loader.
Cas :)
Quote from: Matzon on October 31, 2007, 10:30:37
Quote from: elias on October 31, 2007, 10:14:22
Still yes to guessing file type :). The processor overhead of looking up <lower case extension>->Plugin is negliable, and as for the developer knowing the file type, consider the case where the file name is configurable from e.g. an xml file. Then the developer need to do his own mapping from extension to type or the type needs to be in the configuration file. Besides, ImageIO doesn't use separate file type arguments in ImageIO.read().
How do you guess the type from an input stream? - without sniffing (which causes performance overhead)
Read a few bytes, and figure out the file format, like ImageIO does.
- elias
I am not sure I like the fact that N loaders have to read K bytes, when the user basically knows anyway.
of course you could do a load(input) and a load(input, type)
If you do a quick guess based on the file extension, then it's very likely that the first loader will be the correct one anyway. However I suspect we're well into overengineering territory now. ;D
i like the idea of load(input, type), since many times you want to rename your image files so average joe doesn't see the extension and realise that he can easily open it in any paint program and steal your game art.
Quote from: Matzon on October 31, 2007, 13:43:03
I am not sure I like the fact that N loaders have to read K bytes, when the user basically knows anyway.
of course you could do a load(input) and a load(input, type)
Yea, lets do both ways. Some more work but in the end we all benefit from ;)
Quote from: princec on October 26, 2007, 12:43:56
All I want is to easily play and loop OGGs and stream them reliably. Jorbis does a good job of decoding OGGs but streaming them through OpenAL is woefully unreliable and I can't figure out why.
The API I put on top of OpenAL went some way to doing this stuff but because of the streaming problems I couldn't really recommend it to anyone :(
This (http://kcat.strangesoft.net/openal.html) seems to be a software implementation of OpenAL, (found via this post (http://www.gamedev.net/community/forums/topic.asp?topic_id=472878)). If reliability of OpenAL is the main problem (it's certainly one of the main reasons I don't like it) perhaps LWJGL could use this instead? It does sound promising as it lists "Many other small and large bugs squashed" from the original OpenAL source.
That's an interesting project. I'm also concerned about the stability of the official OpenAL library (and why on earth do they still have a completely different implementation for each platform?). I'd like to have it support mac os x too (and written in java!), but you can't have it all.
- elias
i have an image loader, that uses the java api. or did you want it api indepedant. it has tga support (not compressed). as well as a flip command. it can also convert bgr to rgb + flip, in 1 pass. The code isnt clean because i understand it, but if your interested i could clean it up for the sake of other people who may use it.
The reason we're using Devil instead of loading our texture on java side is performance. It's quite a bit slower loading large png files in java, then passing them through jni to the native functions. So if at all possible, I'd like the Devil replacer to be at least as fast as Devil ever was.
Quote from: ville on December 13, 2007, 12:02:50
The reason we're using Devil instead of loading our texture on java side is performance. It's quite a bit slower loading large png files in java, [...]
The only reason to use PNGs is convenience. Compared to the other options it actually looks pretty bad. It's rather slow to load for what it has to offer, compression isn't great, the compression scheme is Deflate (the same Jar/Zip uses... and usually you use that already or something better), and it also uses the full amount of VRAM.
Using PNGs and being overly worried about performance is a bit... how should I put it... weird? :)
Well it's the same for any image type really, java actually reads the files pretty fast. The slow part is I think getting the image data from java to the c code, so that's why I'm thinking the loading should be done in native code.
I find it dead fast, but that's because I use a custom format very similar to TGA which reads raw image data directly into native ByteBuffers and then passes it straight to OpenGL as a pointer. About as fast as you can get in Java in fact.
Cas :)