Playing Multiple Wav Out Multiple Channels Avaudioengine

AVAudioEngine playing multi channel audio

So here's what I managed so far. It's far from perfect but it somewhat works.

To get all channels you need to use AVAudioPCMBuffer and store two channels from file in each. Also, for each channel pair you need separate AVAudioPlayerNode, then just connect each player to AVAudioMixerNode and we're done. Some simple code for 6-channel audio:

AVAudioFormat *outputFormat = [[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatFloat32 sampleRate:file.processingFormat.sampleRate channels:2 interleaved:false];
AVAudioFile *file = [[AVAudioFile alloc] initForReading:[[NSBundle mainBundle] URLForResource:@"nums6ch" withExtension:@"wav"] error:nil];
AVAudioPCMBuffer *wholeBuffer = [[AVAudioPCMBuffer alloc] initWithPCMFormat:file.processingFormat frameCapacity:(AVAudioFrameCount)file.length];
AVAudioPCMBuffer *buffer1 = [[AVAudioPCMBuffer alloc] initWithPCMFormat:outputFormat frameCapacity:(AVAudioFrameCount)file.length];
AVAudioPCMBuffer *buffer2 = [[AVAudioPCMBuffer alloc] initWithPCMFormat:outputFormat frameCapacity:(AVAudioFrameCount)file.length];
AVAudioPCMBuffer *buffer3 = [[AVAudioPCMBuffer alloc] initWithPCMFormat:outputFormat frameCapacity:(AVAudioFrameCount)file.length];
memcpy(buffer1.audioBufferList->mBuffers[0].mData, wholeBuffer.audioBufferList->mBuffers[0].mData, wholeBuffer.audioBufferList->mBuffers[0].mDataByteSize);
memcpy(buffer1.audioBufferList->mBuffers[1].mData, wholeBuffer.audioBufferList->mBuffers[1].mData, wholeBuffer.audioBufferList->mBuffers[1].mDataByteSize);
buffer1.frameLength = wholeBuffer.audioBufferList->mBuffers[0].mDataByteSize/sizeof(UInt32);
memcpy(buffer2.audioBufferList->mBuffers[0].mData, wholeBuffer.audioBufferList->mBuffers[2].mData, wholeBuffer.audioBufferList->mBuffers[2].mDataByteSize);
memcpy(buffer2.audioBufferList->mBuffers[1].mData, wholeBuffer.audioBufferList->mBuffers[3].mData, wholeBuffer.audioBufferList->mBuffers[3].mDataByteSize);
buffer2.frameLength = wholeBuffer.audioBufferList->mBuffers[0].mDataByteSize/sizeof(UInt32);
memcpy(buffer3.audioBufferList->mBuffers[0].mData, wholeBuffer.audioBufferList->mBuffers[4].mData, wholeBuffer.audioBufferList->mBuffers[4].mDataByteSize);
memcpy(buffer3.audioBufferList->mBuffers[1].mData, wholeBuffer.audioBufferList->mBuffers[5].mData, wholeBuffer.audioBufferList->mBuffers[5].mDataByteSize);
buffer3.frameLength = wholeBuffer.audioBufferList->mBuffers[0].mDataByteSize/sizeof(UInt32);

AVAudioEngine *engine = [[AVAudioEngine alloc] init];
AVAudioPlayerNode *player1 = [[AVAudioPlayerNode alloc] init];
AVAudioPlayerNode *player2 = [[AVAudioPlayerNode alloc] init];
AVAudioPlayerNode *player3 = [[AVAudioPlayerNode alloc] init];
AVAudioMixerNode *mixer = [[AVAudioMixerNode alloc] init];
[engine attachNode:player1];
[engine attachNode:player2];
[engine attachNode:player3];
[engine attachNode:mixer];
[engine connect:player1 to:mixer format:outputFormat];
[engine connect:player2 to:mixer format:outputFormat];
[engine connect:player3 to:mixer format:outputFormat];
[engine connect:mixer to:engine.outputNode format:outputFormat];
[engine startAndReturnError:nil];

[player1 scheduleBuffer:buffer1 completionHandler:nil];
[player2 scheduleBuffer:buffer2 completionHandler:nil];
[player3 scheduleBuffer:buffer3 completionHandler:nil];
[player1 play];
[player2 play];
[player3 play];

Now this solution is far from perfect since there will be a delay between pairs of channels because of calling play for each player at different time. I also still can't play 8-channel audio from my test files (see link in OP). The AVAudioFile processing format has 0 for channel count and even if I create my own format with correct number of channels and layout, I get error on buffer read. Note that I can play this file perfectly fine using AUGraph.

So I will wait before accepting this answer, if you have better solution please share.

EDIT

So it appears that both my unable to sync nodes problem and not being able to play this particular 8-channel audio are bugs (confirmed by Apple developer support).

So little advice for people meddling with audio on iOS. While AVAudioEngine is fine for simple stuff, you should definitely go for AUGraph with more complicated stuff, even stuff that's suppose to work with AVAudioEngine. And if you don't know how to replicate certain things from AVAudioEngine in AUGraph (like myself), well, tough luck.

How to play multiple sounds from buffer simultaneously using nodes connected to AVAudioEngine's mixer

It turns out that having the .interrupt option wasn't the issue (in fact, this actually turned out to be the best way to restart the sound that was playing in my experience, as there was no noticeable pause during the restart, unlike the stop() function). The actual problem that was preventing multiple sounds from playing simultaneously was this particular line of code.

// One AVAudioPlayerNode per note
    var audioFilePlayer: [AVAudioPlayerNode] = Array(repeating: AVAudioPlayerNode(), count: 7)

What happened here was that each item of the array was being assigned the exact same AVAudioPlayerNode value, so they were all effectively sharing the same AVAudioPlayerNode. As a result, the AVAudioPlayerNode functions were affecting all of the items in the array, instead of just the specified item. To fix this and give each item a different AVAudioPlayerNode value, I ended up changing the above line so that it starts as an empty array of type AVAudioPlayerNode instead.

// One AVAudioPlayerNode per note
    var audioFilePlayer = [AVAudioPlayerNode]()

I then added a new line to append to this array a new AVAudioPlayerNode at the beginning inside of the second for-loop of the viewDidLoad() function.

// For each note, attach the corresponding node to the audioEngine, and connect the node to the audioEngine's mixer.
for i in 0...6
{
    audioFilePlayer.append(AVAudioPlayerNode())
    // audioEngine code
}

This gave each item in the array a different AVAudioPlayerNode value. Playing a sound or restarting a sound no longer interrupts the other sounds that are currently being played. I can now play any of the notes simultaneously and without any noticeable latency between note press and playback.

How do you access multiple channels in AVAudioEngine playback?

Install your tap on the playerNode and don't bother specifying the format:

playerNode.installTap(
    onBus: 0,
    bufferSize: 1024,
    format: nil
) { buffer, time in

    // 6 channel buffer
    for channel in 0..<buffer.format.channelCount {
    }
}

Change audio volume of some channels using AVAudioEngine

Changing audio volume by channel instead of by input, requires MatrixMixer. AVAudioEngine MainMixer is not a matrix mixer (mxmx) but a multi channel mixer (mcmx).

In order to use a matrix mixer, use this code:

AudioComponentDescription   mixerUnitDescription;

mixerUnitDescription.componentType          = kAudioUnitType_Mixer;
mixerUnitDescription.componentSubType       = kAudioUnitSubType_MatrixMixer;
mixerUnitDescription.componentManufacturer  = kAudioUnitManufacturer_Apple;
mixerUnitDescription.componentFlags         = 0;
mixerUnitDescription.componentFlagsMask     = 0;

[AVAudioUnit instantiateWithComponentDescription:mixerUnitDescription options:0 completionHandler:^(__kindof AVAudioUnit * _Nullable mixer, NSError * _Nullable error) {

}];

And change audio levels using

AudioUnitSetParameter([_mixer audioUnit], kMatrixMixerParam_Volume, kAudioUnitScope_Input, i, volume, 0);

---

Related Topics