Replacing FMOD and DevIL

[Matzon]

As some of you know, we have talked about replacing FMOD and DevIL, primarly because:
1) FMOD is 3.x, upgrading to 4.x is a lot of work
2) FMOD isn't open source, nor is it free for commercial usage
3) no 64 bit support (not that big a deal)
4) DevIL is somewhat buggy in its image support - depending on the libs installed
5) DevIL is a nuisance when it comes to building the native parts
6) DevIL has had no release since 2006, and seems somewhat unmaintained.

How this should be done has been somewhat of a discussion, but we talked about it in IRC today (please join us! #lwjgl on freenode).
http://echelog.matzon.dk/logs/browse/lwjgl/1193349600 - from [10:36:39]

FMOD
- New audio player thingy, that support loading and playing of WAV (I assume its PCM, since WAV is just a container), OGG (vorbis), MOD and XM support. Preferably more modules (s3c, it etc).
- streaming support
- Straightforward API

DEVIL
- Basic Image loader (using custom loaders, and fallback to ImageIO)
- Basic image manipulation (flip comes to mind).

I encourage everybody to join in and help on this task!

We will be distributing the replacement libs and devil/fmod for some time, before removing fmod and devil entirely. Please let us know of which functions you cannot live without in either of the two APIs.

[princec]

All I want is to easily play and loop OGGs and stream them reliably. Jorbis does a good job of decoding OGGs but streaming them through OpenAL is woefully unreliable and I can't figure out why.

The API I put on top of OpenAL went some way to doing this stuff but because of the streaming problems I couldn't really recommend it to anyone

As for DevIL the only useful thing it brought to the table was JPEG loading without AWT. The other two useful formats are PNG and TGA but I never used either of them as I had a very simple proprietry format and compiled my images ahead of time. A bit of a pain in the arse so it'd be nice not to have to but as I pack all my PNGs into single textures anyway in a precompilation step it never really bothered me enough to moan.

Cas

[Evil-Devil]

For DevIL i can help with custom loader for TGA (RLE) and DXTn.

[Orangy Tang]

I use .it files for music, since they're the format my sound guy prefers. I will try and find out why and what advantages it has (although I have no idea how technically hard it would be to write something to play it back).

[Orangy Tang]

Oh, and another thing - fmod allows you to get callbacks when playing tracker formats when notes/events get triggered. I was planning a music reactive game which used this functionality so that'd be good to have. IIRC it even delayed the callbacks according on the state of the sound buffer so you'd get the callbacks exactly the same frame as when the sound became audible, rather than when they were buffered up.

[Matzon]

Oh, and another thing - fmod allows you to get callbacks when playing tracker formats when notes/events get triggered. I was planning a music reactive game which used this functionality so that'd be good to have. IIRC it even delayed the callbacks according on the state of the sound buffer so you'd get the callbacks exactly the same frame as when the sound became audible, rather than when they were buffered up.

obviously you missed the part in the chat that said:

[10:46:49] <Mazon> but we'd need a first stab at an replacement for 95% of the people and then take it from there

as for module support, we would be limited to whatever 3rd party libraries are doing.

[kappa]

As a first slab i think the slick stuff is pretty good, does all the major image formats, it does use imageIO, but theres already a custom tga loader in there(pretty damn fast), also theres a nice png loader (which is faster than ImageIO) somewhere on JGO as well as a DXTn library floating around also on JGO, the rest of the formats can be replaced over time to become ImageIO free.

As for sound lwjgl_util package already has a wav file loader(could do with some streaming support), add with jorbis for ogg, and ibxm lbrary for .mod and .it support.

just a matter of collecting the stuff and wrapping it.

[elias4444]

I just have a couple of thoughts on all of this:

1) Feel free to ignore anything I have to say. My C and C++ skills are all but dead, so I feel kinda bad that I can't really offer much in the way of hands-on help.

2) As I understand it, LWJGL is meant to be a wrapper to put the basic necessities of advanced game development into Java. With this in mind, you might want to be careful of what I call "scope creep" - IOW, don't try to make it a full blown game engine like jME or Xith3D. You're allowed to stick to the basics.  

3) Devil support can probably just be taken out. Not right away, of course, since there are people who currently rely on it. But really, using the built in Java image libraries is much cleaner, and removes a dependancy. There are a couple of things you might lose, but most of those can either be worked around, or even done with the Java2D library (e.g., image manipulation).

4) OGG, WAV, etc. - Once again, I wouldn't recommend trying to be a full blown engine, as it's kind of beyond the scope of LWJGL (as I understand it). HOWEVER, that said, I sure wouldn't mind a solid sound implementation (maybe in the Utils collection). I agree with Cas, in that OGG streaming is rather flakey. I use jorbis and still have had to put too many try-catches (to catch erroneous errors and fix them) into my openAL code to feel comfortable with it. I'd rather not have to deal with FMOD, as openAL is viable enough and free; I just want a nice, error free approach to using it.

5) refer to #1.  

oh, and... 6) I wouldn't bother with MIDI support. If someone wants it, they can implement it themselves in Java - I personally don't care for it though, as it doesn't use hardware, and things like volume control, speed, etc., all have to be funneled through Java's funny MIDI system.

[Orangy Tang]

Oh, and another thing - fmod allows you to get callbacks when playing tracker formats when notes/events get triggered. I was planning a music reactive game which used this functionality so that'd be good to have. IIRC it even delayed the callbacks according on the state of the sound buffer so you'd get the callbacks exactly the same frame as when the sound became audible, rather than when they were buffered up.

obviously you missed the part in the chat that said:

[10:46:49] <Mazon> but we'd need a first stab at an replacement for 95% of the people and then take it from there

as for module support, we would be limited to whatever 3rd party libraries are doing.

I saw it I agree it's a somewhat obscure feature but it's something which isn't immediately obvious and would be a shame to  loose.

[princec]

I couldn't really care less about module support... truthfully it probably belongs in its own library anyway.

Cas

[elias]

@elias4444: I agree on the feature creep danger, but:

1. We are specifically trying to _replace_ functionality already present, not augment it. Devil does all sorts of image formats and manipulation, fmod does streaming of various formats and more.
2. We're putting all this in optional libraries, so it shouldn't affect users who just want access to the basics (OpenGL, OpenAL, and input)

- elias

[Evil-Devil]

And how to call that "new" Package? LWJGL_IO? o_O

Sry, but i think some basic texture loading support like ImageIO could be done in the lib itself. It just have to be decided which format counts as basic texture format.

[elias4444]

Uh oh, the Eliases are making lists! Watch out! 

That's all fine. You won't hear me complain if someone else wants to write code that saves me from having to write it later. So, I guess thanks are in order.

Do you think you'll write these functions on the C side? Or on the Java side? I'm doing a little research to see if the Java2D image filters are even close to being fast enough for texture manipulation. I'll get back with some numbers if I can can come up with something concrete.

[elias]

Do you think you'll write these functions on the C side?

I hope not! My primary motivation for getting rid of devil and fmod is the fact that they're native code.

- elias

[Evil-Devil]

Or on the Java side?

Why not? Most code is allready done by some of us. And the only formats we need were TGA, BMP, PCX, DXTn. Its just how we add and arrange them for fast and easy usage.