Recording from Remoteio: Resulting .Caf Is Pitch Shifted Slower + Distorted

Recording from RemoteIO: resulting .caf is pitch shifted slower + distorted

Ok - found some code that solves this - though I don't fully understand why.

I had been setting the mBitsPerChannel to 16 for both the RemoteIO output stream and the ExtFileRef. The result was slowed down & scratchy audio. Setting the ExtFileRef mBitsPerChannel to 32 plus adding the kAudioFormatFlagsNativeEndian flag solves the problem: the .caf audio is perfect (while leaving the RemoteIO output stream settings to what they were).

But then also setting the RemoteIO output stream settings to match my new settings also works. So I'm confused. Shouldn't this work so long as the AudioStreamBasicDescription settings are symmetrical for the RemoteIO instance and the ExtFileRef?

Anyway... the working setting is below.

size_t bytesPerSample = sizeof (AudioUnitSampleType);

AudioStreamBasicDescription audioFormat;
audioFormat.mSampleRate= graphSampleRate;
audioFormat.mFormatID=kAudioFormatLinearPCM;
audioFormat.mFormatFlags=kAudioFormatFlagsNativeEndian|kAudioFormatFlagIsSignedInteger|kAudioFormatFlagIsPacked;
audioFormat.mBytesPerPacket=bytesPerSample;
audioFormat.mBytesPerFrame=bytesPerSample;
audioFormat.mFramesPerPacket=1;
audioFormat.mChannelsPerFrame=1;
audioFormat.mBitsPerChannel= 8 * bytesPerSample;
audioFormat.mReserved=0;

RemoteIO recorded audio file is either silent or 4KB

Aside from the (fairly arduous but better covered by Apple's example code) setup process of RemoteIO itself, the key points of insight were:

1. Using the same AudioStreamBasicDescription (*audioFormat) that I used to set up the stream in the first place. I don't know how long I spent trying to set up a new one with slightly different parameters, based on other questions and posts. Just referencing the stream attributes from my ivar was sufficient.
2. Set an "isRecording" bool so that you can turn on and off write-to-file without having to tear down and re-set-up your RemoteIO session
3. It is ok to write to a file in the recordingCallback, um, callback, but do it asynchronously. Lots of info talks about doing it in the playbackCallback or setting up some third audioFileWriteCallback. This resulted in silent files or 4KB (i.e. empty) files. Don't do it.
4. Also, be sure to use a copy of the ioData that got passed into the callback

in recordingCallback after AudioUnitRender into bufferList:

    AudioDeviceManager* THIS = (__bridge AudioDeviceManager *)inRefCon;
    if (THIS->isRecording) {
      ExtAudioFileWriteAsync(THIS->extAudioFileRef, inNumberFrames, bufferList);
    }

1. Start and stop recording functions, for reference:


-(void)startRecording {

  NSArray  *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
  NSString *documentsDirectory = [paths objectAtIndex:0];
  NSString *destinationFilePath = [documentsDirectory stringByAppendingPathComponent:kAudioFileName];
  CFURLRef destinationURL = CFURLCreateWithFileSystemPath(kCFAllocatorDefault, (CFStringRef)destinationFilePath, kCFURLPOSIXPathStyle, false);

  OSStatus status;

  // create the capture file
  status = ExtAudioFileCreateWithURL(destinationURL, kAudioFileWAVEType, &audioFormat, NULL, kAudioFileFlags_EraseFile, &extAudioFileRef);
  if (status) NSLog(@"Error creating file with URL: %ld", status);

 // use the same "audioFormat" AudioStreamBasicDescription we used to set up RemoteIO in the first place
  status = ExtAudioFileSetProperty(extAudioFileRef, kExtAudioFileProperty_ClientDataFormat, sizeof(AudioStreamBasicDescription), &audioFormat);

  ExtAudioFileSeek(extAudioFileRef, 0);
  ExtAudioFileWrite(extAudioFileRef, 0, NULL);

  isRecording = YES;
}

- (void)stopRecording {

  isRecording = NO;
  OSStatus status = ExtAudioFileDispose(extAudioFileRef);
  if (status) printf("ExtAudioFileDispose %ld \n", status);
}

That's it!

Example of saving audio from RemoteIO?

Due to the tight latency requirements of Audio Unit callbacks, one should not to do any synchronous file access (or any other calls that could potentially block, involve memory management or OS locking actions) inside the RemoteIO callback. Instead, just copy the audio data out to another buffer (a larger circular buffer for example), and set some state indicating how much data has been copied. Then, in another thread, when the amount of data is sufficient, write the contents of that buffer out to a file. This could be a raw PCM file, which can later be converted by AVAssetReader/Writer into another audio file type.

Recording to AAC from RemoteIO: data is getting written but file unplayable

So I finally sorted this out! Ugh, what an information scavenger hunt.

Anyway, here is the bit in the docs for ExtAudioFile that I missed (see bolded text). I wasn't setting this property. Data was being written to my .m4a file but it was unreadable at playback. So to sum up: I have a bunch of AUSamplers -> AUMixer -> RemoteIO. A render callback on the RemoteIO instance writes the data out to disk in a compressed m4a format. So it is possible to generate compressed audio on the fly (iOS 5/iPad 2)

Seems pretty robust - I had some printf statements in the rendercallback and the write worked fine.

Yay!

ExtAudioFileProperty_CodecManufacturer
The manufacturer of the codec to be used by the extended audio file object. Value is a read/write UInt32. You must specify this property before setting the kExtAudioFileProperty_ClientDataFormat (page 20) property, which in turn triggers the creation of the codec. Use this property in iOS to choose between a hardware or software encoder, by specifying kAppleHardwareAudioCodecManufacturer or kAppleSoftwareAudioCodecManufacturer.
Available in Mac OS X v10.7 and later.
Declared in ExtendedAudioFile.h.

// specify codec
UInt32 codec = kAppleHardwareAudioCodecManufacturer;
size = sizeof(codec);
result = ExtAudioFileSetProperty(extAudioFileRef, 
                                 kExtAudioFileProperty_CodecManufacturer, 
                                 size, 
                                 &codec);

if(result) printf("ExtAudioFileSetProperty %ld \n", result);

Can anybody help me in recording iPhone output sound through Audio Unit

In initializaeOutputUnit you only created your audio file:

OSStatus setupErr = ExtAudioFileCreateWithURL(destinationURL, kAudioFileWAVEType, &audioFormat, NULL, kAudioFileFlags_EraseFile, &effectState.audioFileRef);

by passing 0 (frames) and NULL (audiobuffer) is just for init internal buffers:

setupErr =  ExtAudioFileWriteAsync(effectState.audioFileRef, 0, NULL);

That's what's going wrong in recordingCallback:

1) ioActionFlags are always 0 and inBusNumber are always 1, because thats how you setup your callback (kInputBus = 1):

if (*ioActionFlags == kAudioUnitRenderAction_PostRender&&inBusNumber==0)

so just remove the if statement.

2) From AudioUnitRender you will receive -50 error, which is defined in CoreAudioTypes.h as an kAudio_ParamError error. This happens by bufferList is not defined and NULL!

 OSStatus status; 
 status = AudioUnitRender(THIS->mAudioUnit,     
                          ioActionFlags, 
                          inTimeStamp, 
                          kInputBus, 
                          inNumberFrames, 
                          &bufferList);

 if (noErr != status) {
      printf("AudioUnitRender error: %ld", status);   
      return noErr; 
 }

You just need to define an valid AudioBuffer and pass it to AudioUnitRender, this is my working RenderCallback:

  static OSStatus recordingCallback       (void *                      inRefCon,
                                           AudioUnitRenderActionFlags *      ioActionFlags,
                                           const AudioTimeStamp *            inTimeStamp,
                                           UInt32                            inBusNumber,
                                           UInt32                            inNumberFrames,
                                           AudioBufferList *                 ioData)  {
      double timeInSeconds = inTimeStamp->mSampleTime / kSampleRate;
      printf("\n%fs inBusNumber: %lu inNumberFrames: %lu ", timeInSeconds, inBusNumber, inNumberFrames);
      //printAudioUnitRenderActionFlags(ioActionFlags);

      AudioBufferList bufferList;

      SInt16 samples[inNumberFrames]; // A large enough size to not have to worry about buffer overrun
      memset (&samples, 0, sizeof (samples));

      bufferList.mNumberBuffers = 1;
      bufferList.mBuffers[0].mData = samples;
      bufferList.mBuffers[0].mNumberChannels = 1;
      bufferList.mBuffers[0].mDataByteSize = inNumberFrames*sizeof(SInt16);

      ViewController* THIS = THIS = (__bridge ViewController *)inRefCon;

      OSStatus status;
      status = AudioUnitRender(THIS->mAudioUnit,     
                               ioActionFlags, 
                               inTimeStamp, 
                               kInputBus, 
                               inNumberFrames, 
                               &bufferList);

      if (noErr != status) {

          printf("AudioUnitRender error: %ld", status); 
          return noErr;
      }

      // Now, we have the samples we just read sitting in buffers in bufferList
      ExtAudioFileWriteAsync(THIS->mAudioFileRef, inNumberFrames, &bufferList);

      return noErr;      
 }

In stopRecord you should close the audio file with ExtAudioFileDispose:

  - (void)stopRecording:(NSTimer*)theTimer {
      printf("\nstopRecording\n");
      AudioOutputUnitStop(mAudioUnit);
      AudioUnitUninitialize(mAudioUnit);

      OSStatus status = ExtAudioFileDispose(mAudioFileRef);
      printf("OSStatus(ExtAudioFileDispose): %ld\n", status); 
 }

Full source code: http://pastebin.com/92Fyjaye

Write Audio To Disk From IO Unit

After a couple of days of tears & hair pulling I have a solution.

In my code and in other examples I have seen extaudiofilewriteasync was called in the callback for the remoteio unit like so.

** remoteiounit callback **

static OSStatus masterChannelMixerUnitCallback(void *inRefCon, 
                              AudioUnitRenderActionFlags *ioActionFlags, 
                              const AudioTimeStamp *inTimeStamp, 
                              UInt32 inBusNumber, 
                              UInt32 inNumberFrames, 
                              AudioBufferList *ioData)

{

    AudioUnitRender(engineDescribtion.equnit, ioActionFlags, inTimeStamp, 0, inNumberFrames, ioData);

    if(isrecording)
    {
        ExtAudioFileWriteAsync(engine->recordingfileref, inNumberFrames, ioData);

    }

    return 0;

}

In this callback I'm pulling audio data from another audio unit that applies eqs and mixes audio.

I removed the extaudiofilewriteasync call from the remoteio callback to this other callback that the remoteio pulls and the file writes successfully!!

*equnits callback function *

static OSStatus outputCallback(void *inRefCon, 
                               AudioUnitRenderActionFlags *ioActionFlags, 
                               const AudioTimeStamp *inTimeStamp, 
                               UInt32 inBusNumber, 
                               UInt32 inNumberFrames, 
                               AudioBufferList *ioData) {  

    AudioUnitRender(engineDescribtion.masterChannelMixerUnit, ioActionFlags, inTimeStamp, 0, inNumberFrames, ioData);

   //process audio here    

    Engine *engine= (Engine *) inRefCon;

    OSStatus s;

    if(engine->isrecording)
    {
        s=ExtAudioFileWriteAsync(engine->recordingfileref, inNumberFrames, ioData);

    }

    return noErr;

}

In the interest of fully understanding why my solution worked could somebody explain to me why writing data to file from the iodata bufferlist of the remoteio causes distorted audio but writing data one further step down the chain results in perfect audio?

iOS - RemoteIO AudioUnits, possible to have 2?

I have learnt since that 2 remoteIOs are not possible for iOS. (please correct me if I am wrong).
RemoteIO acts like a socket in the wall - one plug says "Input" and the other says "Output".
"Input" is not connected to "Output".
Hence I was able to connect my mixer's output to the remoteIO's output.
At the same time, I captured mic audio from RemoteIO input.

converting from AudioUnit format for ExtAudioFileWrite

You want to use that ASBD as your ExtAudioFile's client format (kExtAudioFileProperty_ClientDataFormat), set via property after file creation. This is the format of the data you pass in ExtAudioFileWrite. The format of the output file is specified in ExtAudioFileOpen.

---

Related Topics