iOS How to Correctly Handle Orientation When Capturing Video Using Avassetwriter

iOS how to correctly handle orientation when capturing video using AVAssetWriter

Video orientation is handled by the AVAssetWriterInput.transform, looks like the getVideoTransform() implementation is not correct - CGAffineTransform expects the rotation angle to be in radians, so need to change to something like this:

private func getVideoTransform() -> CGAffineTransform {
    switch UIDevice.current.orientation {
        case .portrait:
            return .identity
        case .portraitUpsideDown:
            return CGAffineTransform(rotationAngle: .pi)
        case .landscapeLeft:
            return CGAffineTransform(rotationAngle: .pi/2)
        case .landscapeRight:
            return CGAffineTransform(rotationAngle: -.pi/2)
        default:
            return .identity
        }
    }

---

From Apple Technical Q&A:
https://developer.apple.com/library/archive/qa/qa1744/_index.html

If you are using an AVAssetWriter object to write a movie file, you
can use the transform property of the associated AVAssetWriterInput to
specify the output file orientation. This will write a display
transform property into the output file as the preferred
transformation of the visual media data for display purposes. See the
AVAssetWriterInput.h interface file for the details.

AVFoundation - why can't I get the video orientation right

The problem is that modifying the writerInput.transform property only adds a tag in the video file metadata which instructs the video player to rotate the file during playback. That's why the videos play in the correct orientation on your device (I'm guessing they also play correctly in a Quicktime player as well).

The pixel buffers captured by the camera are still laid out in the orientation in which they were captured. Many video players will not check for the preferred orientation metadata tag and will just play the file in the native pixel orientation.

If you want the user to be able to record video holding the phone in either landscape mode, you need to rectify this at the AVCaptureSession level before compression by performing a transform on the CVPixelBuffer of each video frame. This Apple Q&A covers it (look at the AVCaptureVideoOutput documentation as well):
https://developer.apple.com/library/ios/qa/qa1744/_index.html

Investigating the link above is the correct way to solve your problem. An alternate fast n' dirty way to solve the same problem would be to lock the recording UI of your app into only one landscape orientation and then to rotate all of your videos server-side using ffmpeg.

Need to Mirror Video Orientation and Handle Rotation When Using Front Camera

Based on this answer: Video Saving in the wrong orientation AVCaptureSession

I faced the same issue and was able to fix it following this post.

var videoConnection:AVCaptureConnection?
  for connection in self.fileOutput.connections {
    for port in connection.inputPorts! {
      if port.mediaType == AVMediaTypeVideo {
        videoConnection = connection as? AVCaptureConnection
          if videoConnection!.supportsVideoMirroring {
            videoConnection!.videoMirrored = true
          }
        }
      }
   }
}

Please let me know if it helps you James

How do I set the orientation for a frame-by-frame-generated video using AVFoundation?

Your post opened my eyes about how Apples Videos app plays back video. I recorded several items with my app with the device in the four orientations. They all played back properly oriented. I just noticed that the Videos app doesn't support rotation like the player in the Photos Album app. The Videos app expects you to hold the device (at least my iPod touch) in landscape. I did some portrait recordings, added them to iTunes, and all, including the one created with Apple's camera app, did not rotate when rotating the device to portrait orientation.

Anyway...

My app is a time lapse app that does not do any extra processing on the frames like you are doing, so YMMV on the following. I have my app set up so that it does not rotate the window as the device is rotated. This way I'm always dealing with one orientation of the device. I use AVFoundation to grab every Nth frame from the video stream and write that out.

As I set up for recording, I do the following.

inputWriterBuffer = [AVAssetWriterInput assetWriterInputWithMediaType: AVMediaTypeVideo outputSettings: outputSettings];
    // I call this explicitly before recording starts. Video plays back the right way up.
[self detectOrientation];
inputWriterBuffer.transform = playbackTransform;

That detectOrientation calls the following method. I've reduced the actual code for clarity here. In my app I also rotate some buttons, so notice they do not get the same transformation. The thing to pay attention to is how I'm setting up the playbackTransform ivar.

-(void) detectOrientation {
CGAffineTransform buttonTransform;

switch ([[UIDevice currentDevice] orientation]) {
    case UIDeviceOrientationUnknown:
        NULL;
    case UIDeviceOrientationFaceUp:
        NULL;
    case UIDeviceOrientationFaceDown:
        NULL;
        break;
    case UIDeviceOrientationPortrait:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( 0 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];            

        playbackTransform = CGAffineTransformMakeRotation( ( 90 * M_PI ) / 180 );
        break;
    case UIDeviceOrientationLandscapeLeft:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( 90 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];            

        // Transform depends on which camera is supplying video
        if (theProject.backCamera == YES) playbackTransform = CGAffineTransformMakeRotation( 0 / 180 );
        else playbackTransform = CGAffineTransformMakeRotation( ( -180 * M_PI ) / 180 );

        break;
    case UIDeviceOrientationLandscapeRight:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( -90 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];

        // Transform depends on which camera is supplying video
        if (theProject.backCamera == YES) playbackTransform = CGAffineTransformMakeRotation( ( -180 * M_PI ) / 180 );
        else playbackTransform = CGAffineTransformMakeRotation( 0 / 180 );

        break;
    case UIDeviceOrientationPortraitUpsideDown:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( 180 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];

        playbackTransform = CGAffineTransformMakeRotation( ( -90 * M_PI ) / 180 );
        break;
    default:
        playbackTransform = CGAffineTransformMakeRotation( 0 / 180 ); // Use the default, although there are likely other issues if we get here.
        break;
}
}

As a side note, since I want the method called when ever the device is rotated, and I've turned off automatic rotation, I have the following in my viewDidLoad method.

[[UIDevice currentDevice] beginGeneratingDeviceOrientationNotifications];
[[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(detectOrientation) name:@"UIDeviceOrientationDidChangeNotification" object:nil];

That's a tip I found in this SOF Q&A.

---

Related Topics