iOS Avfoundation: Setting Orientation of Video

How do I save the video in the correct orientation?

Apparently at first I did the wrong thing in this answer.

I just took out the connection and orientation as global fields of the whole class and addressed them.

Now I have added the following code and this solves my problem.

var currentOrientation: AVCaptureVideoOrientation?
var videoConnection: AVCaptureConnection?

captureSession.addOutput(videoOutput!)

            videoConnection = videoOutput!.connection(with: .video)

            if (videoConnection!.isVideoOrientationSupported) {
                Logger.Log("connection!.isVideoOrientationSupported")
                if let capOr = AVCaptureVideoOrientation(rawValue: UIDevice.current.orientation.rawValue) {
                    Logger.Log("connection!.videoOrientation = \(capOr.rawValue)")
                    currentOrientation = capOr
                    videoConnection!.videoOrientation = currentOrientation!
                } else {
                    currentOrientation = .portrait
                    videoConnection!.videoOrientation = currentOrientation!
                }
            }

cameraPreviewLayer?.connection?.videoOrientation = currentOrientation!

override func viewWillTransition(to size: CGSize, with coordinator: UIViewControllerTransitionCoordinator) {
        coordinator.animate(alongsideTransition: nil) { _ in UIView.setAnimationsEnabled(true) }
        UIView.setAnimationsEnabled(false)
        super.viewWillTransition(to: size, with: coordinator)

        Logger.Log("VIEW WILL TRANSITION")
        if let videoOrientation = AVCaptureVideoOrientation(rawValue: UIDevice.current.orientation.rawValue) {
            Logger.Log("videoOrientation updated = \(videoOrientation.rawValue)")

            currentOrientation = videoOrientation
            videoConnection?.videoOrientation = currentOrientation!
            cameraPreviewLayer?.connection?.videoOrientation = currentOrientation!
        }

        cameraPreviewLayer?.frame.size = size
    }

iOS how to correctly handle orientation when capturing video using AVAssetWriter

Video orientation is handled by the AVAssetWriterInput.transform, looks like the getVideoTransform() implementation is not correct - CGAffineTransform expects the rotation angle to be in radians, so need to change to something like this:

private func getVideoTransform() -> CGAffineTransform {
    switch UIDevice.current.orientation {
        case .portrait:
            return .identity
        case .portraitUpsideDown:
            return CGAffineTransform(rotationAngle: .pi)
        case .landscapeLeft:
            return CGAffineTransform(rotationAngle: .pi/2)
        case .landscapeRight:
            return CGAffineTransform(rotationAngle: -.pi/2)
        default:
            return .identity
        }
    }

From Apple Technical Q&A:
https://developer.apple.com/library/archive/qa/qa1744/_index.html

If you are using an AVAssetWriter object to write a movie file, you
can use the transform property of the associated AVAssetWriterInput to
specify the output file orientation. This will write a display
transform property into the output file as the preferred
transformation of the visual media data for display purposes. See the
AVAssetWriterInput.h interface file for the details.

iOS AVFoundation Video Capture Orientation Options

To answer your question, yes, the image sensor is just oriented that way. The video camera is an approx 1-megapixel "1080p" camera that has a fixed orientation. The 5MP (or 8MP for 4S, etc) still camera also has a fixed orientation. The lenses themselves don't rotate nor do any of the other camera bits, and hence the feed itself has a fixed orientation.

"But wait!", you say, "pictures I take with the camera app (or API) get rotated correctly. Why is that?" That's cuz iOS takes a look at the orientation of the phone when a picture is taken and stores that information with the picture (as an Exif attachment). Yet video isn't so flagged -- and each frame would have to be individually flagged, and then there's issues about what to do when the user rotates the phone during video....

So, no, you can't ask a video stream or a still image what orientation the phone was in when the video was captured. You can, however, directly ask the phone what orientation it is in now:

UIDeviceOrientation currentOrientation = [UIDevice currentDevice].orientation;

If you do that at the start of video capture (or when you grab a still image from a video feed) you can then use that information to do your own rotation of playback.

How to change AVCaptureMovieFileOutput video orientation during running session?

Try adding this before you start your session:

[_movieFileOutput setRecordsVideoOrientationAndMirroringChanges:YES asMetadataTrackForConnection:movieFileOutputConnection];

The header file documentation for this method makes it sound very much like what you're looking for:

Controls whether or not the movie file output will create a timed metadata track that records samples which
reflect changes made to the given connection's videoOrientation and videoMirrored properties during
recording.

There's more interesting information there, I'd read it all.

However, this method doesn't actually rotate your frames, it uses timed metadata to instruct players to do it at playback time, so it's possible that not all players will support this feature. If that's a deal breaker, then you can abandon AVCaptureMovieFileOutput in favour of the lower level AVCaptureVideoDataOutput + AVAssetWriter combination, where your videoOrientation changes actually rotate the frames, resulting in files that will playback correctly in any player:

If an AVCaptureVideoDataOutput instance's connection's videoOrientation or videoMirrored properties are set to
non-default values, the output applies the desired mirroring and orientation by physically rotating and or flipping
sample buffers as they pass through it.

p.s. I don't think you need the beginConfiguration/commitConfiguration pair if you're only changing one property as that's for batching multiple modifications into one atomic update.

How do I set the orientation for a frame-by-frame-generated video using AVFoundation?

Your post opened my eyes about how Apples Videos app plays back video. I recorded several items with my app with the device in the four orientations. They all played back properly oriented. I just noticed that the Videos app doesn't support rotation like the player in the Photos Album app. The Videos app expects you to hold the device (at least my iPod touch) in landscape. I did some portrait recordings, added them to iTunes, and all, including the one created with Apple's camera app, did not rotate when rotating the device to portrait orientation.

Anyway...

My app is a time lapse app that does not do any extra processing on the frames like you are doing, so YMMV on the following. I have my app set up so that it does not rotate the window as the device is rotated. This way I'm always dealing with one orientation of the device. I use AVFoundation to grab every Nth frame from the video stream and write that out.

As I set up for recording, I do the following.

inputWriterBuffer = [AVAssetWriterInput assetWriterInputWithMediaType: AVMediaTypeVideo outputSettings: outputSettings];
    // I call this explicitly before recording starts. Video plays back the right way up.
[self detectOrientation];
inputWriterBuffer.transform = playbackTransform;

That detectOrientation calls the following method. I've reduced the actual code for clarity here. In my app I also rotate some buttons, so notice they do not get the same transformation. The thing to pay attention to is how I'm setting up the playbackTransform ivar.

-(void) detectOrientation {
CGAffineTransform buttonTransform;

switch ([[UIDevice currentDevice] orientation]) {
    case UIDeviceOrientationUnknown:
        NULL;
    case UIDeviceOrientationFaceUp:
        NULL;
    case UIDeviceOrientationFaceDown:
        NULL;
        break;
    case UIDeviceOrientationPortrait:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( 0 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];            

        playbackTransform = CGAffineTransformMakeRotation( ( 90 * M_PI ) / 180 );
        break;
    case UIDeviceOrientationLandscapeLeft:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( 90 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];            

        // Transform depends on which camera is supplying video
        if (theProject.backCamera == YES) playbackTransform = CGAffineTransformMakeRotation( 0 / 180 );
        else playbackTransform = CGAffineTransformMakeRotation( ( -180 * M_PI ) / 180 );

        break;
    case UIDeviceOrientationLandscapeRight:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( -90 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];

        // Transform depends on which camera is supplying video
        if (theProject.backCamera == YES) playbackTransform = CGAffineTransformMakeRotation( ( -180 * M_PI ) / 180 );
        else playbackTransform = CGAffineTransformMakeRotation( 0 / 180 );

        break;
    case UIDeviceOrientationPortraitUpsideDown:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( 180 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];

        playbackTransform = CGAffineTransformMakeRotation( ( -90 * M_PI ) / 180 );
        break;
    default:
        playbackTransform = CGAffineTransformMakeRotation( 0 / 180 ); // Use the default, although there are likely other issues if we get here.
        break;
}
}

As a side note, since I want the method called when ever the device is rotated, and I've turned off automatic rotation, I have the following in my viewDidLoad method.

[[UIDevice currentDevice] beginGeneratingDeviceOrientationNotifications];
[[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(detectOrientation) name:@"UIDeviceOrientationDidChangeNotification" object:nil];

That's a tip I found in this SOF Q&A.

iOS rotate video AVAsset avfoundation

Solved the rotation converting from the code below:

AVMutableVideoComposition rotated video captured in portrait mode

Now having issues with exporting in question below if anyone knows:

~~https://stackoverflow.com/questions/35233766/avasset-failing-to-export~~

Video rotated after applying AVVideoComposition

What worked for me at the end:

private func filterVideo(with filter: Filter?) {
    guard let player = playerLayer?.player, let playerItem = player.currentItem else { return }

    let videoComposition = AVVideoComposition(asset: playerItem.asset, applyingCIFiltersWithHandler: { (request) in
        if let filter = filter {
            if let filteredImage = filter.filterImage(request.sourceImage) {
                let output = filteredImage.cropping(to: request.sourceImage.extent)
                request.finish(with: output, context: nil)
            } else {
                printError("Image not filtered")
                request.finish(with: RenderError.couldNotFilter)
            }
        } else {
            let output = request.sourceImage.cropping(to: request.sourceImage.extent)
            request.finish(with: output, context: nil)
        }
    })

    playerItem.videoComposition = videoComposition
}

This is the filterImage function of Filter, which is just a nice little wrapper for CIFilter:

func filterImage(_ ciImage: CIImage) -> CIImage? {
    guard let filter = ciFilter else { return nil }
    filter.setDefaults()
    filter.setValue(ciImage, forKey: kCIInputImageKey)
    guard let filteredImageData = filter.value(forKey: kCIOutputImageKey) as? CIImage else { return nil }
    return filteredImageData
}