[howto] real time mixing

[codifies]

I found the openal api a little difficult to grok. So I thought others might find this useful.

import java.nio.ByteBuffer;
import org.lwjgl.BufferUtils;

import org.lwjgl.openal.*;
import static org.lwjgl.openal.AL10.*;

import java.io.IOException;

public final class Synth {

	public static void main(String[] args) throws Exception {

		ALContext alContext = ALContext.create();

		// a sound maker
    int source = alGenSources();

    // hold new data for transfer to AL
    ByteBuffer pcm = BufferUtils.createByteBuffer(11025); // about 1/4 second at 44.1khz
    
    float ang=0;
    
    // throw 3 chunks of sound into a queue
    for (int i=0;i<3;i++) {
      ang = fillBuffer(ang, pcm);
      int b = alGenBuffers();
      alBufferData(b, AL_FORMAT_MONO8, pcm, 44100);
      AL10.alSourceQueueBuffers(source, b);
    }

		System.out.println("Playing sound ...");
		alSourcePlay(source);

    boolean done = false;
    int elapsed=0;
    while (!done) {

        // processed since last asked
        int p = alGetSourcei(source, AL_BUFFERS_PROCESSED);
        while(p!=0) {
          elapsed++;
          if (elapsed>15) break; // stop adding to queue
          
          // at least 1 buffer had played so remove it from the queue
          int b = alSourceUnqueueBuffers(source);
          
          // refill the next byte buffer with the next chunk of sound
          ang = fillBuffer(ang, pcm);
          
          // send the buffer to AL
          alBufferData(b, AL_FORMAT_MONO8, pcm, 44100);
          
          // requeue the buffer
          alSourceQueueBuffers(source, b);
          
          // check if any more buffers have played
          p = alGetSourcei(source, AL_BUFFERS_PROCESSED);
        }
        
        Thread.sleep(20); // lets not hog the cpu... (we're not some crappy AAA game!)
        
        // just to show if we stop filling buffers it stops!
        if (alGetSourcei(source, AL_SOURCE_STATE)==AL_STOPPED) { 
          done = true;
          System.out.println("source stopped ");
        }
    }
    
		AL.destroy(alContext);
	}

  // make a new chunk of sound picking off from where we where
  // when we made the last chunk of sound
  static int trig=-1;
  public static float fillBuffer(float ang, ByteBuffer buff) {
    int size = buff.capacity();
    trig++;
    for (int i=0;i<size;i++) {
       
      int source1 = (int)(Math.sin(ang)*127+128);
      int source2 = 0;
      
      if (trig>1) source2 = (int)(Math.sin(ang*4)*127+128);
      if (trig>2) trig=0;
      
      // yes this really is how you do it.... *sigh*
      buff.put(i,(byte)((source1+source2)/2));
      ang+=0.1f;
    }
    return ang;   
  }

}

[codifies]

I should just point out this technique is for use when you are algorithmically creating or steaming the data...

If you have known sound data and you want to play stuff at the same time, just use multiple sources!

[edit] written it up here http://bedroomcoders.co.uk/lwjgl-streaming-sound-with-openal/

[philfrei]

I attempted to enter this program in Eclipse Neon, using LWJGL 3.0.0 build 90. I am quite new to LWJGL. But I was able to get the "Hello World" red screen example program to run (from lwjgl.org, "getting started" page).

The class ALContext is not being found by Eclipse.
I don't see it when inspecting the listed import files in the attached jar.
Is this program now obsolete?
Any suggestions on how to rewrite the program?

[spasi]

This works with the latest nightly build:

import org.lwjgl.BufferUtils;
import org.lwjgl.openal.*;

import java.nio.ByteBuffer;
import java.nio.IntBuffer;

import static org.lwjgl.openal.AL10.*;
import static org.lwjgl.openal.ALC10.*;
import static org.lwjgl.system.MemoryUtil.*;

public final class Synth {

	public static void main(String[] args) throws Exception {
		long device = alcOpenDevice((ByteBuffer)null);
		if ( device == NULL )
			throw new IllegalStateException("Failed to open the default device.");

		ALCCapabilities deviceCaps = ALC.createCapabilities(device);

		long context = alcCreateContext(device, (IntBuffer)null);
		if ( context == NULL )
			throw new IllegalStateException("Failed to create an OpenAL context.");

		alcMakeContextCurrent(context);
		AL.createCapabilities(deviceCaps);

		// a sound maker
		int source = alGenSources();

		// hold new data for transfer to AL
		ByteBuffer pcm = BufferUtils.createByteBuffer(11025); // about 1/4 second at 44.1khz

		float ang = 0;

		// throw 3 chunks of sound into a queue
		for ( int i = 0; i < 3; i++ ) {
			ang = fillBuffer(ang, pcm);
			int b = alGenBuffers();
			alBufferData(b, AL_FORMAT_MONO8, pcm, 44100);
			AL10.alSourceQueueBuffers(source, b);
		}

		System.out.println("Playing sound ...");
		alSourcePlay(source);

		boolean done = false;
		int elapsed = 0;
		while ( !done ) {

			// processed since last asked
			int p = alGetSourcei(source, AL_BUFFERS_PROCESSED);
			while ( p != 0 ) {
				elapsed++;
				if ( elapsed > 15 ) break; // stop adding to queue

				// at least 1 buffer had played so remove it from the queue
				int b = alSourceUnqueueBuffers(source);

				// refill the next byte buffer with the next chunk of sound
				ang = fillBuffer(ang, pcm);

				// send the buffer to AL
				alBufferData(b, AL_FORMAT_MONO8, pcm, 44100);

				// requeue the buffer
				alSourceQueueBuffers(source, b);

				// check if any more buffers have played
				p = alGetSourcei(source, AL_BUFFERS_PROCESSED);
			}

			Thread.sleep(20); // lets not hog the cpu... (we're not some crappy AAA game!)

			// just to show if we stop filling buffers it stops!
			if ( alGetSourcei(source, AL_SOURCE_STATE) == AL_STOPPED ) {
				done = true;
				System.out.println("source stopped ");
			}
		}

		alcDestroyContext(context);
		alcCloseDevice(device);
	}

	// make a new chunk of sound picking off from where we where
	// when we made the last chunk of sound
	static int trig = -1;
	public static float fillBuffer(float ang, ByteBuffer buff) {
		int size = buff.capacity();
		trig++;
		for ( int i = 0; i < size; i++ ) {

			int source1 = (int)(Math.sin(ang) * 127 + 128);
			int source2 = 0;

			if ( trig > 1 ) source2 = (int)(Math.sin(ang * 4) * 127 + 128);
			if ( trig > 2 ) trig = 0;

			// yes this really is how you do it.... *sigh*
			buff.put(i, (byte)((source1 + source2) / 2));
			ang += 0.1f;
		}
		return ang;
	}

}

The ALContext class was a simple wrapper over a context handle and has been removed.

[spasi]

Btw, there are a lot of LWJGL demos here and here that can help you get started. They are always synced with the latest nightly build.

[philfrei]

Thank you spasi!

I do have the openal demos from your first link up and running and am looking forward very much for stepping through that code in more detail, as well as trying out your code below.

[philfrei]

Again thanks! I've gone through this example, and the ALCDemo code with a pretty close reading (looking up every command in the api, basically) and have learned a lot.

I have some questions, pertaining to the back end code that reads/empties the Source buffers. Is there any access to this, either hooks via settings or properties, or via sub-classing? Or is it all native code that is out of reach?

I'm wondering, for example, about lowering latency by putting in a more dynamic blocking arrangement rather than relying on a fixed Thread.sleep() amount. This could perhaps be facilitated if the code which empties the buffers issued a Notify, for example, that could be used to open a latch for the "fillBuffers" functionality to automatically proceed on an as-needed basis.

In comparison, javax.sound.sampled.SourceDataLine's write() method has blocking built in, so there is no need to come up with a "safe" sleep amount and one can be confident that the thread is not consuming any more cpu than is needed.

[spasi]

I have some questions, pertaining to the back end code that reads/empties the Source buffers. Is there any access to this, either hooks via settings or properties, or via sub-classing? Or is it all native code that is out of reach?

The LWJGL bindings simply call the OpenAL functions provided by the implementation. The default implementation that comes bundled with LWJGL is OpenAL Soft. You'll find answers to any questions you might have in the OpenAL documentation. You can also read OpenAL Soft specific documentation and supported extensions.