Realitykit - Keep Object Always in Front of Screen

RealityKit – Keep object always in front of screen

The best solution in this case is to use trackedRaycast method:

let query: ARRaycastQuery = .init(origin: SIMD3<Float>(),
direction: SIMD3<Float>(),
allowing: .estimatedPlane,
alignment: .horizontal)

let repeated = arView.session.trackedRaycast(query) { results in

guard let result: ARRaycastResult = results.first
else { return }

let anchor = AnchorEntity(world: result.worldTransform)
anchor.addChild(model)
arView.scene.anchors.append(anchor)
}

repeated?.stopTracking()

Also, you have to implement ray(through:) instance method:

@MainActor func ray(through screenPoint: CGPoint) -> (origin: SIMD3<Float>, 
direction: SIMD3<Float>)?

How to set entity in front of screen with Reality kit?

If you want an entity to follow the camera and always be in front of the camera, the simplest way to achieve this is using an AnchorEntity:

    let box = ModelEntity(
mesh: MeshResource.generateBox(size: 0.05),
materials: [SimpleMaterial(color: .red, isMetallic: true)]
)

let cameraAnchor = AnchorEntity(.camera)
cameraAnchor.addChild(box)
arView.scene.addAnchor(cameraAnchor)

// Move the box in front of the camera slightly, otherwise
// it will be centered on the camera position and we will
// be inside the box and not be able to see it
box.transform.translation = [0, 0, -0.5]

However if you want to use the cameraTransform property, this seemed to work fine for me:

var c: Cancellable?
var boxAnchor: AnchorEntity?

struct ARViewContainer: UIViewRepresentable {

func makeUIView(context: Context) -> ARView {

let arView = ARView(frame: .zero)

let box = ModelEntity(
mesh: MeshResource.generateBox(size: 0.05),
materials: [SimpleMaterial(color: .red, isMetallic: true)]
)

boxAnchor = AnchorEntity(world: [0,0,0])
arView.scene.addAnchor(boxAnchor!)
boxAnchor!.addChild(box)

c = arView.scene.subscribe(to: SceneEvents.Update.self) { (event) in
guard let boxAnchor = boxAnchor else {
return
}

// Translation matrix that moves the box 1m in front of the camera
let translate = float4x4(
[1,0,0,0],
[0,1,0,0],
[0,0,1,0],
[0,0,-1,1]
)

// Transformed applied right to left
let finalMatrix = arView.cameraTransform.matrix * translate

boxAnchor.setTransformMatrix(finalMatrix, relativeTo: nil)

}

return arView

}

func updateUIView(_ uiView: ARView, context: Context) {}
}

Position object in front of camera in RealityKit

First, add forward in an extension to float4x4 that gives the forward-facing directional vector of a transform matrix.

extension float4x4 {
var forward: SIMD3<Float> {
normalize(SIMD3<Float>(-columns.2.x, -columns.2.y, -columns.2.z))
}
}

Then, implement the following 4 steps:

func updateCursorPosition() {

let cameraTransform: Transform = arView.cameraTransform

// 1. Calculate the local camera position, relative to the sceneEntity
let localCameraPosition: SIMD3<Float> = sceneEntity.convert(position: cameraTransform.translation, from: nil)

// 2. Get the forward-facing directional vector of the camera using the extension described above
let cameraForwardVector: SIMD3<Float> = cameraTransform.matrix.forward

// 3. Calculate the final local position of the cursor using distanceFromCamera
let finalPosition: SIMD3<Float> = localCameraPosition + cameraForwardVector * distanceFromCamera

// 4. Apply the translation
cursorEntity.transform.translation = finalPosition

}

RealityKit – Can't rotate object properly

Use *= compound multiply operator instead of += compound addition.

Here is the explanation.

Adding a Welcome ViewController screen in front of ViewController ARSession

First, you can drag the arrow in the storyboard to set your initial view controller.
drag the arrow

Then, make a new UIViewController class for the welcome screen.

class WelcomeViewController: UIViewController {

}

Then, inside the storyboard, set the welcome view controller to WelcomeViewController class.

set the class

Connect the "Let's go" button to WelcomeViewController via an @IBAction.
add ibaction

Inside the @IBAction, do this:

@IBAction func goPressed(_ sender: Any) {
let storyboard = UIStoryboard(name: "Main", bundle: nil)
if let viewController = storyboard.instantiateViewController(withIdentifier: "ViewController") as? ViewController {
self.present(viewController, animated: true, completion: nil) /// present the view controller (the one with the ARKit)!
}
}

Then, go back to the storyboard, and set the Storyboard ID to the main view controller (the one with the ARSCNView) to ViewController:

set storyboard ID

That's it!

Position a SceneKit object in front of SCNCamera's current orientation

(Swift 4)

Hey, you can use this simpler function if you want to put the object a certain position relative to another node (e.g. the camera node) and also in the same orientation as the reference node:

func updatePositionAndOrientationOf(_ node: SCNNode, withPosition position: SCNVector3, relativeTo referenceNode: SCNNode) {
let referenceNodeTransform = matrix_float4x4(referenceNode.transform)

// Setup a translation matrix with the desired position
var translationMatrix = matrix_identity_float4x4
translationMatrix.columns.3.x = position.x
translationMatrix.columns.3.y = position.y
translationMatrix.columns.3.z = position.z

// Combine the configured translation matrix with the referenceNode's transform to get the desired position AND orientation
let updatedTransform = matrix_multiply(referenceNodeTransform, translationMatrix)
node.transform = SCNMatrix4(updatedTransform)
}

If you'd like to put 'node' 2m right in front of a certain 'cameraNode', you'd call it like:

let position = SCNVector3(x: 0, y: 0, z: -2)
updatePositionAndOrientationOf(node, withPosition: position, relativeTo: cameraNode)

Edit: Getting the camera node

To get the camera node, it depends if you're using SceneKit, ARKit, or other framework. Below are examples for ARKit and SceneKit.

With ARKit, you have ARSCNView to render the 3D objects of an SCNScene overlapping the camera content. You can get the camera node from ARSCNView's pointOfView property:

let cameraNode = sceneView.pointOfView

For SceneKit, you have an SCNView that renders the 3D objects of an SCNScene. You can create camera nodes and position them wherever you want, so you'd do something like:

let scnScene = SCNScene()
// (Configure scnScene here if necessary)
scnView.scene = scnScene
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(0, 5, 10) // For example
scnScene.rootNode.addChildNode(cameraNode)

Once a camera node has been setup, you can access the current camera in the same way as ARKit:

let cameraNode = scnView.pointOfView

How to use Raycast methods in RealityKit?

Simple Ray-Casting

If you want to find out how to position a model made in Reality Composer into a RealityKit scene (that has a detected horizontal plane) using Ray-Casting method, use the following code:

import RealityKit
import ARKit

class ViewController: UIViewController {

@IBOutlet var arView: ARView!
let scene = try! Experience.loadScene()

@IBAction func onTap(_ sender: UITapGestureRecognizer) {

scene.steelBox!.name = "Parcel"

let tapLocation: CGPoint = sender.location(in: arView)
let estimatedPlane: ARRaycastQuery.Target = .estimatedPlane
let alignment: ARRaycastQuery.TargetAlignment = .horizontal

let result: [ARRaycastResult] = arView.raycast(from: tapLocation,
allowing: estimatedPlane,
alignment: alignment)

guard let rayCast: ARRaycastResult = result.first
else { return }

let anchor = AnchorEntity(world: rayCast.worldTransform)
anchor.addChild(scene)
arView.scene.anchors.append(anchor)

print(rayCast)
}
}

Pay attention to a class ARRaycastQuery. This class comes from ARKit, not from RealityKit.

Convex-Ray-Casting

A Convex-Ray-Casting methods like raycast(from:to:query:mask:relativeTo:) is the op of swiping a convex shapes along a straight line and stopping at the very first intersection with any of the collision shape in the scene. Scene raycast() method performs a hit-tests against all entities with collision shapes in the scene. Entities without a collision shape are ignored.

You can use the following code to perform a convex-ray-cast from start position to end:

import RealityKit

let startPosition: SIMD3<Float> = [0, 0, 0]
let endPosition: SIMD3<Float> = [5, 5, 5]
let query: CollisionCastQueryType = .all
let mask: CollisionGroup = .all

let raycasts: [CollisionCastHit] = arView.scene.raycast(from: startPosition,
to: endPosition,
query: query,
mask: mask,
relativeTo: nil)

guard let rayCast: CollisionCastHit = raycasts.first
else { return }

print(rayCast.distance) /* The distance from the ray origin to the hit */
print(rayCast.entity.name) /* The entity's name that was hit */

A CollisionCastHit structure is a hit result of a collision cast and it lives in RealityKit's scene.

P.S.

When you use raycast(from:to:query:mask:relativeTo:) method for measuring a distance from camera to entity it doesn't matter what an orientation of ARCamera is, it only matters what its position is in world coordinates.



Related Topics



Leave a reply



Submit