Realitykit as a Framework to Build 3D Nonar Apps

RealityKit as a framework to build 3D nonAR apps

Apple has been preparing a replacement for SceneKit, and it's definitely a Reality family – RealityKit, RealityFoundation, Reality Composer (iOS/macOS), Reality Converter, QuickLook, etc. Using RealityKit 2.0, you can create both AR and VR apps. As a tool for prototyping 3D scenes, you need to use Reality Composer. And even despite the lack of such an important feature as .allowsCameraControl, you are now able to make VR apps using RealityKit.

Workaround

Nonetheless, use my code as a starting point to create your own camera control for VR in RealityKit:

import UIKit
import RealityKit

class ViewController: UIViewController {
    @IBOutlet var arView: ARView!
    let buildings = try! Experience.loadScene()
    let camera = PerspectiveCamera()
    var current_X_Angle: Float = 0.0
    var current_Y_Angle: Float = 0.0

    override func viewDidLoad() {
        super.viewDidLoad()
        arView.environment.background = .color(.systemCyan)
        arView.scene.anchors.append(buildings)
        self.camera.position.z = 1
        buildings.children[0].addChild(self.camera)
        self.gestureRecognizer()
    }
    func gestureRecognizer() {
        for gestureRecognizer in [UIPanGestureRecognizer.self,
                                  UIPinchGestureRecognizer.self] {
            if gestureRecognizer == UIPinchGestureRecognizer.self {
                let r = UIPinchGestureRecognizer(target: self,
                               action: #selector(allowCameraControl_01))
                arView.addGestureRecognizer(r)
            }
            if gestureRecognizer == UIPanGestureRecognizer.self {
                let r = UIPanGestureRecognizer(target: self,
                               action: #selector(allowCameraControl_02))
                arView.addGestureRecognizer(r)
            }
        }
    }
    @objc func allowCameraControl_01(recognizer: UIPinchGestureRecognizer) {
        switch recognizer.state {
            case .changed, .ended:
                self.camera.position.z *= 1 / Float(recognizer.scale)
                recognizer.scale = 1.0
            default: break
        }
    }
    @objc func allowCameraControl_02(recognizer: UIPanGestureRecognizer) {
        switch recognizer.state {
            case .changed, .ended:
                let translate = recognizer.translation(in: recognizer.view)
                let angle_X = Float(translate.y / 300) * .pi / 180.0
                let angle_Y = Float(translate.x / 100) * .pi / 180.0
                self.current_X_Angle += angle_X
                self.current_Y_Angle += angle_Y                
                camera.setOrientation(Transform(pitch: current_X_Angle,
                                                  yaw: current_Y_Angle,
                                                 roll: .zero).rotation,
                                      relativeTo: buildings.anchor)
            default: break
        }
    }
}

Using RealityKit and SceneKit together

SceneKit and RealityKit are incompatible due to a complete dissimilarity – difference in scenes' hierarchy, difference in renderer and physics engines, difference in component content. What's stopping you from using SceneKit + ARKit (ARSCNView class)?

ARKit 6.0 has a built-in Depth API (the same API is available in RealityKit) that uses a LiDAR scanner to more accurately determine distances in a surrounding environment, allowing us to use plane detection, raycasting and object occlusion more efficiently.

For that, use sceneReconstruction instance property and ARMeshAnchors.

import ARKit
import SceneKit

class ViewController: UIViewController {

    @IBOutlet var sceneView: ARSCNView!

    override func viewDidLoad() {
        super.viewDidLoad()

        sceneView.scene = SCNScene()
        sceneView.delegate = self

        let config = ARWorldTrackingConfiguration()
        config.sceneReconstruction = .mesh
        config.planeDetection = .horizontal
        sceneView.session.run(config)
    }
}

Delegate's method:

extension ViewController: ARSCNViewDelegate {

    func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, 
                                                 for anchor: ARAnchor) {

        guard let meshAnchor = anchor as? ARMeshAnchor else { return }
        let meshGeo = meshAnchor.geometry

        // logic ...

        node.addChildNode(someModel)
    }
}

P. S.

This post will be helpful for you.

SceneKit's allowsCameraControl equivalent in RealityKit

At the moment RealityKit 2.0 has no SceneKit's equivalent called .allowsCameraControl for moving and rotating virtual camera. Cupertino engineers consider that there's no need to have such a control because RealityKit is rather AR-sentric framework than VR-sentric one.

For further details, look at this post.

---

Related Topics