Deep Copy of Cmimagebuffer or Cvimagebuffer

Pulling data from a CMSampleBuffer in order to create a deep copy

Alright, I think I finally got it. I created a helper extension to make a full copy of a CVPixelBuffer:

extension CVPixelBuffer {
    func copy() -> CVPixelBuffer {
        precondition(CFGetTypeID(self) == CVPixelBufferGetTypeID(), "copy() cannot be called on a non-CVPixelBuffer")

        var _copy : CVPixelBuffer?
        CVPixelBufferCreate(
            nil,
            CVPixelBufferGetWidth(self),
            CVPixelBufferGetHeight(self),
            CVPixelBufferGetPixelFormatType(self),
            CVBufferGetAttachments(self, kCVAttachmentMode_ShouldPropagate)?.takeUnretainedValue(),
            &_copy)

        guard let copy = _copy else { fatalError() }

        CVPixelBufferLockBaseAddress(self, kCVPixelBufferLock_ReadOnly)
        CVPixelBufferLockBaseAddress(copy, 0)

        for plane in 0..<CVPixelBufferGetPlaneCount(self) {
            let dest = CVPixelBufferGetBaseAddressOfPlane(copy, plane)
            let source = CVPixelBufferGetBaseAddressOfPlane(self, plane)
            let height = CVPixelBufferGetHeightOfPlane(self, plane)
            let bytesPerRow = CVPixelBufferGetBytesPerRowOfPlane(self, plane)

            memcpy(dest, source, height * bytesPerRow)
        }

        CVPixelBufferUnlockBaseAddress(copy, 0)
        CVPixelBufferUnlockBaseAddress(self, kCVPixelBufferLock_ReadOnly)

        return copy
    }
}

Now you can use this in your didOutputSampleBuffer method:

guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }

let copy = pixelBuffer.copy()

toProcess.append(copy)

But be aware, one such pixelBuffer takes up about 3MB of memory (1080p), which means that in 100 frames you got already about 300MB, which is about the point at which the iPhone says STAHP (and crashes).

Note that you don't actually want to copy the CMSampleBuffer since it only really contains a CVPixelBuffer because it's an image.

Deep Copy of Audio CMSampleBuffer

Here is a working solution I finally implemented. I sent this snippet to Apple Developer Technical support and asked them to check if it is a correct way to copy incoming sample buffer. The basic idea is copy AudioBufferList and then create a CMSampleBuffer and set AudioBufferList to this sample.

AudioBufferList audioBufferList;
CMBlockBufferRef blockBuffer;
//Create an AudioBufferList containing the data from the CMSampleBuffer,
//and a CMBlockBuffer which references the data in that AudioBufferList.
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer, NULL, &audioBufferList, sizeof(audioBufferList), NULL, NULL, 0, &blockBuffer);
NSUInteger size = sizeof(audioBufferList);
char buffer[size];

memcpy(buffer, &audioBufferList, size);
//This is the Audio data.
NSData *bufferData = [NSData dataWithBytes:buffer length:size];

const void *copyBufferData = [bufferData bytes];
copyBufferData = (char *)copyBufferData;

CMSampleBufferRef copyBuffer = NULL;
OSStatus status = -1;

/* Format Description */

AudioStreamBasicDescription audioFormat = *CMAudioFormatDescriptionGetStreamBasicDescription((CMAudioFormatDescriptionRef) CMSampleBufferGetFormatDescription(sampleBuffer));

CMFormatDescriptionRef format = NULL;
status = CMAudioFormatDescriptionCreate(kCFAllocatorDefault, &audioFormat, 0, nil, 0, nil, nil, &format);

CMFormatDescriptionRef formatdes = NULL;
status = CMFormatDescriptionCreate(NULL, kCMMediaType_Audio, 'lpcm', NULL, &formatdes);
if (status != noErr)
{
  NSLog(@"Error in CMAudioFormatDescriptionCreator");
  CFRelease(blockBuffer);
  return;
}

/* Create sample Buffer */
CMItemCount framesCount = CMSampleBufferGetNumSamples(sampleBuffer);
CMSampleTimingInfo timing   = {.duration= CMTimeMake(1, 44100), .presentationTimeStamp= CMSampleBufferGetPresentationTimeStamp(sampleBuffer), .decodeTimeStamp= CMSampleBufferGetDecodeTimeStamp(sampleBuffer)};

status = CMSampleBufferCreate(kCFAllocatorDefault, nil , NO,nil,nil,format, framesCount, 1, &timing, 0, nil, ©Buffer);

if( status != noErr) {
  NSLog(@"Error in CMSampleBufferCreate");
  CFRelease(blockBuffer);
  return;
}

/* Copy BufferList to Sample Buffer */
AudioBufferList receivedAudioBufferList;
memcpy(&receivedAudioBufferList, copyBufferData, sizeof(receivedAudioBufferList));

//Creates a CMBlockBuffer containing a copy of the data from the
//AudioBufferList.
status = CMSampleBufferSetDataBufferFromAudioBufferList(copyBuffer, kCFAllocatorDefault , kCFAllocatorDefault, 0, &receivedAudioBufferList);
if (status != noErr) {
  NSLog(@"Error in CMSampleBufferSetDataBufferFromAudioBufferList");
  CFRelease(blockBuffer);
  return;
}

Code-Level Support answer:

This code looks ok (though you’ll want to add some additional error
checking). I've successfully tested it in an app that implements the
AVCaptureAudioDataOutput delegate
captureOutput:didOutputSampleBuffer:fromConnection: method to
capture and record audio. The captured audio I'm getting when using
this deep copy code appears to be the same as what I get when directly
using the provided sample buffer (without the deep copy).

Apple Developer Technical Support

How do you clone a BufferedImage

Something like this?

static BufferedImage deepCopy(BufferedImage bi) {
 ColorModel cm = bi.getColorModel();
 boolean isAlphaPremultiplied = cm.isAlphaPremultiplied();
 WritableRaster raster = bi.copyData(null);
 return new BufferedImage(cm, raster, isAlphaPremultiplied, null);
}

How to get Image from URL to a File in Flutter?

Managed to get the File object, had to use Uint8List in a different way, so will use it for now, here is the working code:

final http.Response responseData = await http.get(strURL);
uint8list = responseData.bodyBytes;
var buffer = uint8list.buffer;
ByteData byteData = ByteData.view(buffer);
var tempDir = await getTemporaryDirectory();
File file = await File('${tempDir.path}/img').writeAsBytes(
    buffer.asUint8List(byteData.offsetInBytes, byteData.lengthInBytes));

Vim file navigation

I don't find drilling down into subdirectories via plain old :e to be that cumbersome given a decent configuration for tab-completion.

Look into the 'wildmenu' option to have Vim show a list of completions (filenames) in the modeline above the commandline. You can change the 'wildmode' option to further configure the kind of tab-completion Vim will do.

Personally I use :set wildmode=full.

My workflow is like this:

1. :cd into the toplevel directory of my project.

2. To open file foo/bar/baz:

   • Simplest scenario: type :e f<tab>b<tab>b<tab><enter>.

   • If there are more than one file starting with b in one of those directories you might have to do a <left> or <right> or another <tab> on the keyboard to jump between them (or type a few more letters to disambiguate).

   • Worst-case scenario there are files and directories that share a name and you need to drill down into the directory. In this case tab-complete the directory name and then type *<tab> to drill down.

3. Open 2 or 3 windows and open files in all of them as needed.
4. Once a file is open in a buffer, don't kill the buffer. Leave it open in the background when you open new files. Just :e a new file in the same window.
5. Then, use :b <tab> to cycle through buffers that are already open in the background. If you type :b foo<tab> it will match only against currently-open files that match foo.

I also use these mappings to make it easier to open new windows and to jump between them because it's something I do so often.

" Window movements; I do this often enough to warrant using up M-arrows on this"
nnoremap <M-Right> <C-W><Right>
nnoremap <M-Left> <C-W><Left>
nnoremap <M-Up> <C-W><Up>
nnoremap <M-Down> <C-W><Down>

" Open window below instead of above"
nnoremap <C-W>N :let sb=&sb<BAR>set sb<BAR>new<BAR>let &sb=sb<CR>

" Vertical equivalent of C-w-n and C-w-N"
nnoremap <C-w>v :vnew<CR>
nnoremap <C-w>V :let spr=&spr<BAR>set nospr<BAR>vnew<BAR>let &spr=spr<CR>

" I open new windows to warrant using up C-M-arrows on this"
nmap <C-M-Up> <C-w>n
nmap <C-M-Down> <C-w>N
nmap <C-M-Right> <C-w>v
nmap <C-M-Left> <C-w>V

It takes me a matter of seconds to open Vim, set up some windows and open a few files in them. Personally I have never found any of the third-party file-browsing scripts to be very useful.

---

Related Topics