How do I control AVAssetWriter to write at the correct FPS
I'm reaching here, but I think this is where you're going wrong. Think of your video capture as a pipeline.
(1) Capture buffer -> (2) Do Something With buffer -> (3) Write buffer as frames in video.
Sounds like you've successfully completed (1) and (2), you're getting the buffer fast enough and you're processing them so you can vend them as frames.
The problem is almost certainly in (3) writing the video frames.
https://developer.apple.com/reference/avfoundation/avmutablevideocomposition
Check out the frameDuration setting in your AVMutableComposition, you'll need something like CMTime(1, 60) //60FPS or CMTime(1, 240) // 240FPS to get what you're after (telling the video to WRITE this many frames and encode at this rate).
Using AVAssetWriter, it's exactly the same principle but you set the frame rate as a property in the AVAssetWriterInput outputSettings adding in the AVVideoExpectedSourceFrameRateKey.
NSDictionary *videoCompressionSettings = @{AVVideoCodecKey : AVVideoCodecH264,
AVVideoWidthKey : @(videoWidth),
AVVideoHeightKey : @(videoHeight),
AVVideoExpectedSourceFrameRateKey : @(60),
AVVideoCompressionPropertiesKey : @{ AVVideoAverageBitRateKey : @(bitsPerSecond),
AVVideoMaxKeyFrameIntervalKey : @(1)}
};
To expand a little more - you can't strictly control or sync your camera capture exactly to the output / playback rate, the timing just doesn't work that way and isn't that exact, and of course the processing pipeline adds overhead. When you capture frames they are time stamped, which you've seen, but in the writing / compression phase, it's using only the frames it needs to produce the output specified for the composition.
It goes both ways, you could capture only 30 FPS and write out at 240 FPS, the video would display fine, you'd just have a lot of frames "missing" and being filled in by the algorithm. You can even vend only 1 frame per second and play back at 30FPS, the two are separate from each other (how fast I capture Vs how many frames and what I present per second)
As to how to play it back at different speed, you just need to tweak the playback speed - slow it down as needed.
If you've correctly set the time base (frameDuration), it will always play back "normal" - you're telling it "play back is X Frames Per Second", of course, your eye may notice a difference (almost certainly between low FPS and high FPS), and the screen may not refresh that high (above 60FPS), but regardless the video will be at a "normal" 1X speed for it's timebase. By slowing the video, if my timebase is 120, and I slow it to .5x I know effectively see 60FPS and one second of playback takes two seconds.
You control the playback speed by setting the rate property on AVPlayer https://developer.apple.com/reference/avfoundation/avplayer
AVAssetWriter AVVideoExpectedSourceFrameRateKey (frame rate) ignored
You can control the timing of each sample you append to your AVAssetWriterInput
directly with CMSampleBufferCreateCopyWithNewTiming
.
You need to adjust the timing in the CMSampleTimingInfo
you provide.
Retrieve current timing info with CMSampleBufferGetOutputSampleTimingInfoArray
and just go over the duration of each sample and calculate the correct duration to get 12 frames per second and adjust presentation and decode timestamps to match this new duration.
You then make your copy and feed it to your writer's input.
Let's say you have existingSampleBuffer
:
CMSampleBufferRef sampleBufferToWrite = NULL;
CMSampleTimingInfo sampleTimingInfo = {0};
CMSampleBufferGetSampleTimingInfo(existingSampleBuffer, 0, &sampleTimingInfo);
// modify duration & presentationTimeStamp
sampleTimingInfo.duration = CMTimeMake(1, 12) // or whatever frame rate you desire
sampleTimingInfo.presentationTimeStamp = CMTimeAdd(previousPresentationTimeStamp, sampleTimingInfo.duration);
previousPresentationTimeStamp = sampleTimingInfo.presentationTimeStamp; // should be initialised before passing here the first time
OSStatus status = CMSampleBufferCreateCopyWithNewTiming(kCFAllocatorDefault, existingSampleBuffer, 1, &sampleTimingInfo, &sampleBufferToWrite);
if (status == noErr) {
// you can write sampleBufferToWrite
}
I'm making some assumptions in this code:
- SampleBuffer contains only one sample
- SampleBuffer contains uncompressed video (otherwise, you need to handle decodeTimeStamp as well)
Reduce Video Frame Rate While Keeping Same Length/Duration Using AVAssetWriter
For each dropped frame, you need to compensate by doubling the duration of the sample you're going to write.
while(videoInput.isReadyForMoreMediaData){
if let sample = assetReaderVideoOutput.copyNextSampleBuffer() {
if counter % 2 == 0 {
let timingInfo = UnsafeMutablePointer<CMSampleTimingInfo>.allocate(capacity: 1)
let newSample = UnsafeMutablePointer<CMSampleBuffer?>.allocate(capacity: 1)
// Should check call succeeded
CMSampleBufferGetSampleTimingInfo(sample, 0, timingInfo)
timingInfo.pointee.duration = CMTimeMultiply(timingInfo.pointee.duration, 2)
// Again, should check call succeeded
CMSampleBufferCreateCopyWithNewTiming(nil, sample, 1, timingInfo, newSample)
videoInput.append(newSample.pointee!)
}
counter = counter + 1
}
}
Length of last frame with AVAssetWriter
Yeah, that's not what endSessionAtSourceTime is for. You either need to set the duration of that last frame's sample to something longer than every other sample, (which makes the recording variable-frame rate which some software does not like), or simply keep adding that last sample over and over again with the same single-frame duration until you reach the desired total amount of time.
AVAssetWriter failing output settings with 720p @ 60fps
What happens if you remove the AVVideoMaxKeyFrameIntervalKey
key? I think you're saying 1 keyframe every 60 frames with that.
Related Topics
How Long Does a Push Notification Sit in Queue Before Being Removed
Apple Watch - Only Getting Data If App on Phone Is Active
Admob Sdk - iOS -File Not Found
Type 'String.Index' Does Not Conform Protocol 'Integerliteralconvertible'
Gamecenter iOS 9 Gamecenter Gklocalplayerlistener Methods Not Called
Arkit Session Paused and Not Resuming
Check If Username Already Exist'S:Swift, Firebase
iOS - Mkmapview - Draggable Annotations
Sending Current Location to Server in Background as Well as in Running App Using Afnetworking 3
Error Domain=Nsposixerrordomain Code=100 "Protocol Error"
Uploads Using Backgroundsessionconfiguration and Nsurlsessionuploadtask Cause App to Crash
Disable iOS Simulator 'Connect Hardware Keyboard' Programmatically
Apple Push Notification Service Server Certificate Update
Warning in Custom Map Annotations Iphone
Switching from Xcode3 to Xcode4 - Can't Load Programs Onto Older Ipod Touch
How to Properly Structure Firebase Database to Allow Easy Reading and Removal