Audio capture example not working.

[moci]

Hi,

I need to capture the microphone audio and process it in real time. I'm currently looking at your capture example: http://java-game-lib.svn.sourceforge.net/viewvc/java-game-lib/trunk/LWJGL/src/java/org/lwjgl/test/openal/ALCCaptureTest.java?revision=2983&view=markup which looks simple enough. But I've got a problem.

Here's the code I'm using:

package capturetest;

import java.nio.ByteBuffer;
import java.nio.IntBuffer;

import org.lwjgl.BufferUtils;
import org.lwjgl.LWJGLException;
import org.lwjgl.openal.AL;
import org.lwjgl.openal.AL10;
import org.lwjgl.openal.ALC10;
import org.lwjgl.openal.ALC11;
import org.lwjgl.openal.ALCdevice;
import org.lwjgl.openal.OpenALException;

public class CaptureDeviceTest {
	public static void main(String[] args) {
		CaptureDeviceTest test = new CaptureDeviceTest();
		test.start();
	}
	
	public void start() {
		// Initialize OpenAL and clear the error bit.
		try{
			AL.create();
		} catch (LWJGLException le) {
			le.printStackTrace();
		  return;
		}
		AL10.alGetError();
		
		this.captureTest();
		
		AL.destroy();
	}
	
	private void captureTest() {
		int lastError = ALC10.ALC_NO_ERROR;
		
		int state = AL10.AL_PLAYING;
		
		int format = AL10.AL_FORMAT_STEREO16;
		int formatSize = 16 / 8;
		int freq = 44100;
		int time = 5;
		int bufferSize = (freq * time);
		
		IntBuffer sampleCount = BufferUtils.createIntBuffer(1);
		ByteBuffer buf = BufferUtils.createByteBuffer(bufferSize * formatSize);
		
		if (!ALC10.alcIsExtensionPresent(AL.getDevice(), "ALC_EXT_CAPTURE")) {
			throw new OpenALException("ALC_EXT_CAPTURE extension not available");
		}
		
		String[] capDevices = ALC10.alcGetString(null, ALC11.ALC_CAPTURE_DEVICE_SPECIFIER).split("\0");
		System.out.println("Available Capture Devices: ");
		for (int i = 0; i < capDevices.length; i++) {
			System.out.println(i + ": " + capDevices[i]);
		}
		
		String defCap = ALC10.alcGetString(AL.getDevice(), ALC11.ALC_CAPTURE_DEFAULT_DEVICE_SPECIFIER);
		System.out.println("Default capture Device: " + defCap);
		
		ALCdevice device = ALC11.alcCaptureOpenDevice(null, freq, format, bufferSize);
		
		if (device != null) {
			//CAPTURE
			System.out.println("Recordin using " + ALC10.alcGetString(device, ALC11.ALC_CAPTURE_DEVICE_SPECIFIER) + " ..." );
			ALC11.alcCaptureStart(device);
			
			while (sampleCount.get(0) < bufferSize) {
				ALC10.alcGetInteger(device, ALC11.ALC_CAPTURE_SAMPLES, sampleCount);
			}
			
			System.out.println("Done recording.");
			ALC11.alcCaptureStop(device);
			
			ALC11.alcCaptureSamples(device, buf, bufferSize);
			ALC11.alcCaptureCloseDevice(device);
			
			//PLAYBACK
			IntBuffer buffers = BufferUtils.createIntBuffer(1);
			IntBuffer sources = BufferUtils.createIntBuffer(1);
			
			buffers.position(0).limit(1);
			AL10.alGenBuffers(buffers);
			
			sources.position(0).limit(1);
			AL10.alGenBuffers(sources);
			
			System.out.println("Playing ...");
			
			AL10.alBufferData(buffers.get(0), format, buf, freq);
			AL10.alSourcei(sources.get(0), AL10.AL_BUFFER, buffers.get(0));
			AL10.alSourcei(sources.get(0), AL10.AL_LOOPING, AL10.AL_FALSE);
			AL10.alSourcePlay(sources.get(0));
			
			while (state == AL10.AL_PLAYING) {
				state = AL10.alGetSourcei(sources.get(0), AL10.AL_SOURCE_STATE);
			}
			
			System.out.println("Done playing.");
			
			AL10.alDeleteBuffers(buffers);
			AL10.alDeleteSources(sources);
		}
	}
}

I have not changed much besides some variable names, I also didn't include the "pause" function as I don't know where that came from... If Java has it by default, I didn't find it. Is this the problem?

The error I get is the following:

Available Capture Devices:
0: Microphone (SoundMAX Integrated
Default capture Device: Microphone (SoundMAX Integrated
Recordin using Microphone (SoundMAX Integrated ...
Done recording.
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x184fbfaa, pid=3744, tid=4540
#
# JRE version: 6.0_29-b11
# Java VM: Java HotSpot(TM) Client VM (20.4-b02 mixed mode windows-x86 )
# Problematic frame:
# C  [OpenAL32.dll+0x1bfaa]
#
# An error report file with more information is saved as:
# D:\School\MAEI\2011-2012\thesis\code\java\eclipse\LWJGL_OpenAL\hs_err_pid3744.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
AL lib: ReleaseALC: 2 devices not closed

Thank you!

PS: if I have multiple sources with different effects. Is the source-buffer copied or referenced? (If one source applies the effect will it actually change the buffer as well? If so, I'll have to make a copy of the buffer for each independent source?)

[Matzon]

since it's crashing in OpenAL32, try and set the logging level to 3:
set ALSOFT_LOGLEVEL=3

and run again, see if something interesting pops up.

Please make sure to use 2.8.2, since 2.8.1 has a known problem when closing OpenAL32

[moci]

Sorry, how do I set this? I'm using 2.8.2 and it's crashing at this line:

ALC11.alcCaptureSamples(device, buf, bufferSize);

Btw, I'm planning on buying Rapture3D. I'm assuming I can change LWJGL to use Rapture3D instead of OpenAL Soft? I need HRTF filtering on the 3D sound, and as I undestand OpenAL Soft doesn't have HRTF at the moment (or at least inferior to Rapture3D's implementation). - this is key for my project.

[Matzon]

just write it on the command line - it's an environment variable that OpenAL-Soft will pick up.
I suggest you go to #openal on freenode to discuss HRTF.

[moci]

I chanced it, the error message doesn't look different to me.

Available Capture Devices: 
0: Microphone (SoundMAX Integrated
Default capture Device: Microphone (SoundMAX Integrated
Recordin using Microphone (SoundMAX Integrated ...
Done recording.
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x182dbfaa, pid=5332, tid=5900
#
# JRE version: 6.0_29-b11
# Java VM: Java HotSpot(TM) Client VM (20.4-b02 mixed mode windows-x86 )
# Problematic frame:
# C  [OpenAL32.dll+0x1bfaa]
#
# An error report file with more information is saved as:
# D:\School\MAEI\2011-2012\thesis\code\java\eclipse\LWJGL_OpenAL\hs_err_pid5332.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
[color=red]AL lib: ReleaseALC: 2 devices not closed[/color]

I've attached the log file.

I have already talked about HRTF on the IRC channel, and the nightly build has some HRTF functionality. Eventually I want to use Rapture3D, it is possible to switch the driver right? If not, I'll have to look at a different engine (or just use JOGL).

[Matzon]

doesn't look like the variable is seen ... check that its 3 by typing 'set'

I dont understand the stacktrace! - the error frames for java and native, are in completely different places!

I dont know about the rapture drivers, but you should be able to just remove out OpenAL32 dll and install theirs. Add -Dorg.lwjgl.util.Debug=true to the command line to see where it's loading the driver from.

I dont see what JOGL has to do with the OpenAL issue ??

[moci]

Looks like I had to manually add it to my variables before it took an effect (just typing it in the command line would clear it when I closed the command line).

The error log is quite different now:

AL lib: GetConfigValue: Key general:hrtf_tables not found
AL lib: GetConfigValue: Key general:rt-prio not found
AL lib: GetConfigValue: Key general:resampler not found
AL lib: GetConfigValue: Key general:trap-alc-error not found
AL lib: GetConfigValue: Key general:trap-al-error not found
AL lib: GetConfigValue: Key reverb:boost not found
AL lib: GetConfigValue: Key reverb:emulate-eax not found
AL lib: GetConfigValue: Key general:drivers not found
AL lib: MMDevApiMsgProc: Starting message thread
AL lib: MMDevApiMsgProc: Starting message loop
AL lib: alc_initconfig: Initialized backend "mmdevapi"
AL lib: alc_initconfig: Added "mmdevapi" for playback
AL lib: alc_initconfig: Initialized backend "dsound"
AL lib: alc_initconfig: Initialized backend "winmm"
AL lib: alc_initconfig: Added "winmm" for capture
AL lib: GetConfigValue: Key general:excludefx not found
AL lib: GetConfigValue: Key general:frequency not found
AL lib: GetConfigValue: Key general:format not found
AL lib: GetConfigValue: Key general:periods not found
AL lib: GetConfigValue: Key general:period_size not found
AL lib: GetConfigValue: Key general:sources not found
AL lib: GetConfigValue: Key general:slots not found
AL lib: GetConfigValue: Key general:sends not found
AL lib: GetConfigValue: Key general:cf_level not found
AL lib: MMDevApiMsgProc: Got message 1024
AL lib: alcOpenDevice: Created device 18780048
AL lib: UpdateDeviceParams: Format pre-setup: Stereo, Signed Short, 44100hz, 1024 update size x4
AL lib: MMDevApiMsgProc: Got message 1025
AL lib: UpdateDeviceParams: Format post-setup: Stereo, Signed Short, 48000hz, 960 update size x4
AL lib: GetConfigValue: Key general:layout_STEREO not found
AL lib: GetConfigValue: Key general:layout not found
AL lib: GetConfigValue: Key general:hrtf not found
AL lib: UpdateDeviceParams: HRTF disabled
AL lib: UpdateDeviceParams: BS2B disabled
AL lib: UpdateDeviceParams: Stereo duplication disabled
AL lib: alcCreateContext: Created context 18690E80
Available Capture Devices:
0: Microphone (SoundMAX Integrated
Default capture Device: Microphone (SoundMAX Integrated
AL lib: alcCaptureOpenDevice: Created device 187A89C0
Recordin using Microphone (SoundMAX Integrated ...
Done recording.
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x1852bfaa, pid=1692, tid=4968
#
# JRE version: 6.0_29-b11
# Java VM: Java HotSpot(TM) Client VM (20.4-b02 mixed mode windows-x86 )
# Problematic frame:
# C  [OpenAL32.dll+0x1bfaa]
#
# An error report file with more information is saved as:
# D:\School\MAEI\2011-2012\thesis\code\java\eclipse\LWJGL_OpenAL\hs_err_pid1692.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
AL lib: ReleaseALC: 2 devices not closed

Sorry to confuse you, I made a typo: JOGL should have been JOAL. But if I can just swap out the dll files it's ok! I chose LWJGL because I had the option of using OpenGL if I ever needed it to make a visual presentation of the sources and listener positions.

When I change this line:

int format = AL10.AL_FORMAT_STEREO16;

to

int format = AL10.AL_FORMAT_MONO16;

I get the following output:

AL lib: GetConfigValue: Key general:hrtf_tables not found
AL lib: GetConfigValue: Key general:rt-prio not found
AL lib: GetConfigValue: Key general:resampler not found
AL lib: GetConfigValue: Key general:trap-alc-error not found
AL lib: GetConfigValue: Key general:trap-al-error not found
AL lib: GetConfigValue: Key reverb:boost not found
AL lib: GetConfigValue: Key reverb:emulate-eax not found
AL lib: GetConfigValue: Key general:drivers not found
AL lib: MMDevApiMsgProc: Starting message thread
AL lib: MMDevApiMsgProc: Starting message loop
AL lib: alc_initconfig: Initialized backend "mmdevapi"
AL lib: alc_initconfig: Added "mmdevapi" for playback
AL lib: alc_initconfig: Initialized backend "dsound"
AL lib: alc_initconfig: Initialized backend "winmm"
AL lib: alc_initconfig: Added "winmm" for capture
AL lib: GetConfigValue: Key general:excludefx not found
AL lib: GetConfigValue: Key general:frequency not found
AL lib: GetConfigValue: Key general:format not found
AL lib: GetConfigValue: Key general:periods not found
AL lib: GetConfigValue: Key general:period_size not found
AL lib: GetConfigValue: Key general:sources not found
AL lib: GetConfigValue: Key general:slots not found
AL lib: GetConfigValue: Key general:sends not found
AL lib: GetConfigValue: Key general:cf_level not found
AL lib: MMDevApiMsgProc: Got message 1024
AL lib: alcOpenDevice: Created device 18790048
AL lib: UpdateDeviceParams: Format pre-setup: Stereo, Signed Short, 44100hz, 1024 update size x4
AL lib: MMDevApiMsgProc: Got message 1025
AL lib: UpdateDeviceParams: Format post-setup: Stereo, Signed Short, 48000hz, 960 update size x4
AL lib: GetConfigValue: Key general:layout_STEREO not found
AL lib: GetConfigValue: Key general:layout not found
AL lib: GetConfigValue: Key general:hrtf not found
AL lib: UpdateDeviceParams: HRTF disabled
AL lib: UpdateDeviceParams: BS2B disabled
AL lib: UpdateDeviceParams: Stereo duplication disabled
AL lib: alcCreateContext: Created context 18780E80
Available Capture Devices:
0: Microphone (SoundMAX Integrated
Default capture Device: Microphone (SoundMAX Integrated
AL lib: alcCaptureOpenDevice: Created device 187B89C0
2:33:51 - Recording using Microphone (SoundMAX Integrated ...
2:33:56 - Done recording.
AL lib: FreeDevice: 187B89C0
2:33:56 - Playing ...
2:33:56 - Done playing.
AL lib: FreeContext: 18780E80
AL lib: MMDevApiMsgProc: Got message 1026
AL lib: MMDevApiMsgProc: Got message 1027
AL lib: FreeDevice: 18790048
AL lib: FreeDevice: (18790048) Deleting 1 Buffer(s)
AL lib: alcMMDevApiDeinit: Sending WM_QUIT to Thread 0db0
AL lib: MMDevApiMsgProc: Message loop finished

I've added the current time HH:MM:SS to the statements, it's supposed to record for 5 seconds (looks like that's ok) but it also has to play for 5 seconds, which it doesn't.

[Matzon]

aha! -  didn't notice the mono/stereo change... if recording in stereo (not sure if that works?) you'd need to double the buffer size - else you'll exceed the buffer size.

Try to clear any errors before player - and/or examining any alErrors.

The pause function is part of the basic test - and basically calls Thread.sleep(100) - if you don't sleep between the state calls, you'll be busy waiting.

I suggest you try the unmodified org.lwjgl.test.openal.ALCCaptureTest just to check that stuff works as expected.

[moci]

I must have missed some initialization code in BasicTest. The example works as expected, however it will randomly crash (not every time) while closing, yes I'm using 2.8.2.

AL lib: alcMMDevApiDeinit: Sending WM_QUIT to Thread 1384
** RANDOM (IN TIME) CRASH HERE **
AL lib: MMDevApiMsgProc: Message loop finished

I've adjusted the example file so it automatically adjusts the buffer depending on the format (stereo or mono), I also dropped the extending so it's all in one file. I find that more convenient when looking at examples.

Thanks!

[Matzon]

fwiw, this crash-at-exit is exactly the reason why a newer openal was included in 2.8.2 ... please double check you are in fact using the latest OpenAL32.dll

[moci]

I only started using LWJGL a couple of days ago. After the 2.8.2 release, it's the only one I've got. If I have to actually change some dll's on the computer somewhere else then I didn't do that.

I did what was mentioned in the installation guide. Just include the jar files and link to the native folders. (Eclipse)