Arkit - How to Display the Feed from a Virtual Scncamera Placed on Scnplane

SceneKIt - how to assign ARSCNView video feed as a texture to a SCNGeometry

Create a fresh (out of the box) ARKit SceneKit based App
(the one with the SpaceShip)

Then you modify your viewWillAppear section like this:

Add the DispatchQueue stuff below.

Important: Add a short delay (here 5.0 seconds) to give the AR Subsystem enough time to initialise the Camera.

override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)
    
    // Create a session configuration
    let configuration = ARWorldTrackingConfiguration()

    // Run the view's session
    sceneView.session.run(configuration)
    
    DispatchQueue.main.asyncAfter(deadline: .now() + 5.0) {
        let shipMesh = self.sceneView.scene.rootNode.childNode(withName: "shipMesh", recursively: true)
        shipMesh?.geometry?.firstMaterial?.diffuse.contents = self.sceneView.scene.background.contents
    }
    
}

I hope, this is what you were looking for.

ARKit and Vision – How to place a SCNPlane on a found QR-Code?

Works for me using the center point instead of the origins:

let center2 = CGPoint(x: result.boundingBox.midX, y: result.boundingBox.midY)

let hitTestResults2 = frame.hitTest(center2, types: [.featurePoint/*, .estimatedHorizontalPlane, .existingPlane, .existingPlaneUsingExtent*/] )

Is there any other way to display UIView into ARKit?

From Apples docs on SCNMaterialProperty.contents:

SceneKit cannot use a layer that is already being displayed elsewhere (for example, the backing layer of a
UIView
object)

Personally the easiest way of displaying information on a plane is using a SpriteKit Scene.

ARKit - Center SCNode on camera

To keep your node always in the center of view, you have to calculate the center position of world position.

extension ARSCNView {
    // create real world position of the point
    func realWorldPosition(for point: CGPoint) -> SCNVector3? {
        let result = self.hitTest(point, types: [.featurePoint])
        guard let hitResult = result.last else { return nil }
        let hitTransform = SCNMatrix4(hitResult.worldTransform)

        // m4x -> position ;; 1: x, 2: y, 3: z
        let hitVector = SCNVector3Make(hitTransform.m41, hitTransform.m42, hitTransform.m43)

        return hitVector
    }
}

// MARK: - ARSCNViewDelegate
func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
    DispatchQueue.main.async {
        let center = self.view.center
        guard let realWorldPosition = self.sceneView.realWorldPosition(for: center) else { return }
        self.NodeCompass.position = realWorldPosition
    }
}

ARKit – Get current position of ARCamera in a scene

Set yourself as the ARSession.delegate. Than you can implement session(_:didUpdate:) which will give you an ARFrame for every frame processed in your session. The frame has an camera property that holds information on the cameras transform, rotation and position.

func session(_ session: ARSession, didUpdate frame: ARFrame) {
    // Do something with the new transform
    let currentTransform = frame.camera.transform
    doSomething(with: currentTransform)
}

---

As rickster pointed out you always can get the current ARFrame and the camera position through it by calling session.currentFrame.
This is useful if you need the position just once, eg to move a node where the camera has been but you should use the delegate method if you want to get updates on the camera's position.

---

Related Topics