How to Implement a Billboard Effect (Lookat Camera) in Realitykit

How can I get an entity to constantly look at the camera in RealityKit similar to Billboard Constraint from SceneKit

I was able to figure out an answer to my question. Adding the following block of code allowed the entity to constantly look at the camera:

func session(_ session: ARSession, didUpdate frame: ARFrame) {
    planeModelEntity.look(at: arView.cameraTransform.translation, from: planeModelEntity.position(relativeTo: nil), relativeTo: nil)
}

In SceneKit, how do you make camera look at a specific face of a node's geometry?

No.

That would require movement of the camera, in addition to re-aiming it.

Imagine I'm in front of my house. I have a great view of the front and can just barely see the side to my left. In my Scene I tap the side of the house. A LookAt constraint would merely change the angle of the camera. It would not be aligned with the normal of that barely visible side.

To align with the normal, I'd have to walk around the house until I can stare at the house and be perpendicular to the side I tapped. At what radius? What path? You have to figure that out yourself.

Depending on what effect you're trying for, you might want to rotate the model instead of moving the camera. Rotate the tapped node locally (or as a child of an invisible parent) so that its minus-Z axis points out the tapped face, and keep a lookAtConstraint on the node, not the camera. This approach will change the look of the object, though: you will see it rotating, and the shading changing appropriately.

SceneKit Node To Follow Camera

There is actually a type of constraint for this specific purpose.
It's called SCNBillboardConstraint and it always orients the node it is assigned to towards the current camera. You don't even have to provide a target when initializing.

ARKit billboarding effect with SceneKit

You were almost there!

Just modify the text's node's pivot to rotate it by 180 degrees (Obj-C).

node.pivot = SCNMatrix4MakeRotation(M_PI, 0, 1, 0);

Position a SceneKit object in front of SCNCamera's current orientation

(Swift 4)

Hey, you can use this simpler function if you want to put the object a certain position relative to another node (e.g. the camera node) and also in the same orientation as the reference node:

func updatePositionAndOrientationOf(_ node: SCNNode, withPosition position: SCNVector3, relativeTo referenceNode: SCNNode) {
    let referenceNodeTransform = matrix_float4x4(referenceNode.transform)

    // Setup a translation matrix with the desired position
    var translationMatrix = matrix_identity_float4x4
    translationMatrix.columns.3.x = position.x
    translationMatrix.columns.3.y = position.y
    translationMatrix.columns.3.z = position.z

    // Combine the configured translation matrix with the referenceNode's transform to get the desired position AND orientation
    let updatedTransform = matrix_multiply(referenceNodeTransform, translationMatrix)
    node.transform = SCNMatrix4(updatedTransform)
}

If you'd like to put 'node' 2m right in front of a certain 'cameraNode', you'd call it like:

let position = SCNVector3(x: 0, y: 0, z: -2)
updatePositionAndOrientationOf(node, withPosition: position, relativeTo: cameraNode)

Edit: Getting the camera node

To get the camera node, it depends if you're using SceneKit, ARKit, or other framework. Below are examples for ARKit and SceneKit.

With ARKit, you have ARSCNView to render the 3D objects of an SCNScene overlapping the camera content. You can get the camera node from ARSCNView's pointOfView property:

let cameraNode = sceneView.pointOfView

For SceneKit, you have an SCNView that renders the 3D objects of an SCNScene. You can create camera nodes and position them wherever you want, so you'd do something like:

let scnScene = SCNScene()
// (Configure scnScene here if necessary)
scnView.scene = scnScene
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(0, 5, 10)  // For example
scnScene.rootNode.addChildNode(cameraNode)

Once a camera node has been setup, you can access the current camera in the same way as ARKit:

let cameraNode = scnView.pointOfView

SceneKit background too bright when applying camera frame MTLTexture as a background

It turns out what was causing the washed out effect in the texture was the pixel format. It should be .bgra8Unorm_srgb (not .bgra8Unorm).

---

Related Topics