RealityKit – Keep object always in front of screen
The best solution in this case is to use trackedRaycast
method:
let query: ARRaycastQuery = .init(origin: SIMD3<Float>(),
direction: SIMD3<Float>(),
allowing: .estimatedPlane,
alignment: .horizontal)
let repeated = arView.session.trackedRaycast(query) { results in
guard let result: ARRaycastResult = results.first
else { return }
let anchor = AnchorEntity(world: result.worldTransform)
anchor.addChild(model)
arView.scene.anchors.append(anchor)
}
repeated?.stopTracking()
Also, you have to implement ray(through:)
instance method:
@MainActor func ray(through screenPoint: CGPoint) -> (origin: SIMD3<Float>,
direction: SIMD3<Float>)?
How to set entity in front of screen with Reality kit?
If you want an entity to follow the camera and always be in front of the camera, the simplest way to achieve this is using an AnchorEntity:
let box = ModelEntity(
mesh: MeshResource.generateBox(size: 0.05),
materials: [SimpleMaterial(color: .red, isMetallic: true)]
)
let cameraAnchor = AnchorEntity(.camera)
cameraAnchor.addChild(box)
arView.scene.addAnchor(cameraAnchor)
// Move the box in front of the camera slightly, otherwise
// it will be centered on the camera position and we will
// be inside the box and not be able to see it
box.transform.translation = [0, 0, -0.5]
However if you want to use the cameraTransform property, this seemed to work fine for me:
var c: Cancellable?
var boxAnchor: AnchorEntity?
struct ARViewContainer: UIViewRepresentable {
func makeUIView(context: Context) -> ARView {
let arView = ARView(frame: .zero)
let box = ModelEntity(
mesh: MeshResource.generateBox(size: 0.05),
materials: [SimpleMaterial(color: .red, isMetallic: true)]
)
boxAnchor = AnchorEntity(world: [0,0,0])
arView.scene.addAnchor(boxAnchor!)
boxAnchor!.addChild(box)
c = arView.scene.subscribe(to: SceneEvents.Update.self) { (event) in
guard let boxAnchor = boxAnchor else {
return
}
// Translation matrix that moves the box 1m in front of the camera
let translate = float4x4(
[1,0,0,0],
[0,1,0,0],
[0,0,1,0],
[0,0,-1,1]
)
// Transformed applied right to left
let finalMatrix = arView.cameraTransform.matrix * translate
boxAnchor.setTransformMatrix(finalMatrix, relativeTo: nil)
}
return arView
}
func updateUIView(_ uiView: ARView, context: Context) {}
}
Position object in front of camera in RealityKit
First, add forward
in an extension to float4x4 that gives the forward-facing directional vector of a transform matrix.
extension float4x4 {
var forward: SIMD3<Float> {
normalize(SIMD3<Float>(-columns.2.x, -columns.2.y, -columns.2.z))
}
}
Then, implement the following 4 steps:
func updateCursorPosition() {
let cameraTransform: Transform = arView.cameraTransform
// 1. Calculate the local camera position, relative to the sceneEntity
let localCameraPosition: SIMD3<Float> = sceneEntity.convert(position: cameraTransform.translation, from: nil)
// 2. Get the forward-facing directional vector of the camera using the extension described above
let cameraForwardVector: SIMD3<Float> = cameraTransform.matrix.forward
// 3. Calculate the final local position of the cursor using distanceFromCamera
let finalPosition: SIMD3<Float> = localCameraPosition + cameraForwardVector * distanceFromCamera
// 4. Apply the translation
cursorEntity.transform.translation = finalPosition
}
RealityKit – Can't rotate object properly
Use *=
compound multiply operator instead of +=
compound addition.
Here is the explanation.
Adding a Welcome ViewController screen in front of ViewController ARSession
First, you can drag the arrow in the storyboard to set your initial view controller.
Then, make a new UIViewController
class for the welcome screen.
class WelcomeViewController: UIViewController {
}
Then, inside the storyboard, set the welcome view controller to WelcomeViewController
class.
Connect the "Let's go" button to WelcomeViewController
via an @IBAction
.
Inside the @IBAction
, do this:
@IBAction func goPressed(_ sender: Any) {
let storyboard = UIStoryboard(name: "Main", bundle: nil)
if let viewController = storyboard.instantiateViewController(withIdentifier: "ViewController") as? ViewController {
self.present(viewController, animated: true, completion: nil) /// present the view controller (the one with the ARKit)!
}
}
Then, go back to the storyboard, and set the Storyboard ID
to the main view controller (the one with the ARSCNView
) to ViewController
:
That's it!
Position a SceneKit object in front of SCNCamera's current orientation
(Swift 4)
Hey, you can use this simpler function if you want to put the object a certain position relative to another node (e.g. the camera node) and also in the same orientation as the reference node:
func updatePositionAndOrientationOf(_ node: SCNNode, withPosition position: SCNVector3, relativeTo referenceNode: SCNNode) {
let referenceNodeTransform = matrix_float4x4(referenceNode.transform)
// Setup a translation matrix with the desired position
var translationMatrix = matrix_identity_float4x4
translationMatrix.columns.3.x = position.x
translationMatrix.columns.3.y = position.y
translationMatrix.columns.3.z = position.z
// Combine the configured translation matrix with the referenceNode's transform to get the desired position AND orientation
let updatedTransform = matrix_multiply(referenceNodeTransform, translationMatrix)
node.transform = SCNMatrix4(updatedTransform)
}
If you'd like to put 'node' 2m right in front of a certain 'cameraNode', you'd call it like:
let position = SCNVector3(x: 0, y: 0, z: -2)
updatePositionAndOrientationOf(node, withPosition: position, relativeTo: cameraNode)
Edit: Getting the camera node
To get the camera node, it depends if you're using SceneKit, ARKit, or other framework. Below are examples for ARKit and SceneKit.
With ARKit, you have ARSCNView to render the 3D objects of an SCNScene overlapping the camera content. You can get the camera node from ARSCNView's pointOfView property:
let cameraNode = sceneView.pointOfView
For SceneKit, you have an SCNView that renders the 3D objects of an SCNScene. You can create camera nodes and position them wherever you want, so you'd do something like:
let scnScene = SCNScene()
// (Configure scnScene here if necessary)
scnView.scene = scnScene
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(0, 5, 10) // For example
scnScene.rootNode.addChildNode(cameraNode)
Once a camera node has been setup, you can access the current camera in the same way as ARKit:
let cameraNode = scnView.pointOfView
How to use Raycast methods in RealityKit?
Simple Ray-Casting
If you want to find out how to position a model made in Reality Composer into a RealityKit scene (that has a detected horizontal plane) using Ray-Casting method, use the following code:
import RealityKit
import ARKit
class ViewController: UIViewController {
@IBOutlet var arView: ARView!
let scene = try! Experience.loadScene()
@IBAction func onTap(_ sender: UITapGestureRecognizer) {
scene.steelBox!.name = "Parcel"
let tapLocation: CGPoint = sender.location(in: arView)
let estimatedPlane: ARRaycastQuery.Target = .estimatedPlane
let alignment: ARRaycastQuery.TargetAlignment = .horizontal
let result: [ARRaycastResult] = arView.raycast(from: tapLocation,
allowing: estimatedPlane,
alignment: alignment)
guard let rayCast: ARRaycastResult = result.first
else { return }
let anchor = AnchorEntity(world: rayCast.worldTransform)
anchor.addChild(scene)
arView.scene.anchors.append(anchor)
print(rayCast)
}
}
Pay attention to a class ARRaycastQuery
. This class comes from ARKit, not from RealityKit.
Convex-Ray-Casting
A Convex-Ray-Casting methods like
raycast(from:to:query:mask:relativeTo:)
is the op of swiping a convex shapes along a straight line and stopping at the very first intersection with any of the collision shape in the scene. Sceneraycast()
method performs a hit-tests against all entities withcollision shapes
in the scene. Entities without a collision shape are ignored.
You can use the following code to perform a convex-ray-cast from start position to end:
import RealityKit
let startPosition: SIMD3<Float> = [0, 0, 0]
let endPosition: SIMD3<Float> = [5, 5, 5]
let query: CollisionCastQueryType = .all
let mask: CollisionGroup = .all
let raycasts: [CollisionCastHit] = arView.scene.raycast(from: startPosition,
to: endPosition,
query: query,
mask: mask,
relativeTo: nil)
guard let rayCast: CollisionCastHit = raycasts.first
else { return }
print(rayCast.distance) /* The distance from the ray origin to the hit */
print(rayCast.entity.name) /* The entity's name that was hit */
A CollisionCastHit
structure is a hit result of a collision cast and it lives in RealityKit's scene.
P.S.
When you use
raycast(from:to:query:mask:relativeTo:)
method for measuring a distance from camera to entity it doesn't matter what an orientation of ARCamera is, it only matters what its position is in world coordinates.
Related Topics
Way to Check If Up or Down Button Is Pressed with Nsstepper
Swiftui Published Updates Not Refreshing
Swift Compile Error, Subclassing Nsvalue, Using Super.Init(Nonretainedobject:)
Root Class of All Classes in Swift
How to Check Dictionary Value Is Empty or Not
How to Display an Alert Controller When Nsdate == to a Time (For Example 12:00 Am) in Swift
Swift Method Parameters in Documentation
Why Is This Predicate Format Being Turned into '= Nil'
Cannot Use Mutating Member ... Because Append
Swift - Connect Delegate to Custom Xib Cell
How to Fill Triangle in Swift Using Core Graphics
Need Clarification on Typealias Syntax in Swift
Swift: Switch Statement Fallthrough Behavior
List Scroll Freeze on Catalyst Navigationview
Removing Scheduled Local Notification
Cannot Divide and Assign Int64 Value
iOS-Charts Error: Thread1: Exc_Bad_Access (Code=2, Address=0X2A0C220)