What Is Aranchor Exactly

What is ARAnchor exactly?

Updated: July 10, 2022.

TL;DR


ARAnchor

ARAnchor is an invisible null-object that holds a 3D model at anchor's position. Think of ARAnchor just like it's a parent transform node with local axis (you can translate, rotate and scale it). Every 3D model has a pivot point, right? So this pivot point must meet an ARAnchor in ARKit.

If you're not using anchors in ARKit app (in RealityKit it's impossible not to use anchors because they are part of a scene), your 3D models may drift from where they were placed, and this will dramatically impact app’s realism and user experience. Thus, anchors are crucial elements of any AR scene.

According to ARKit 2017 documentation:

ARAnchor is a real-world position and orientation that can be used for placing objects in AR Scene. Adding an anchor to the session helps ARKit to optimize world-tracking accuracy in the area around that anchor, so that virtual objects appear to stay in place relative to the real world. If a virtual object moves, remove the corresponding anchor from the old position and add one at the new position.

ARAnchor is a parent class of other 10 anchors' types in ARKit, hence all those subclasses inherit from ARAnchor. Usually you do not use ARAnchor directly. I must also say that ARAnchor and Feature Points have nothing in common. Feature Points are rather special visual elements for tracking and debugging.

ARAnchor doesn't automatically track a real world target. When you need automation, you have to use renderer() or session() instance methods that can be implemented in case you comformed to ARSCNViewDelegate or ARSessionDelegate protocols, respectively.

Here's an image with visual representation of plane anchor. Keep in mind: you can neither see a detected plane nor its corresponding ARPlaneAnchor, by default. So, if want to see the anchor in your scene, you may "visualize" it using three thin SCNCylinder primitives. Each color of the cylinder represents a particular axis: so RGB is XYZ.


Sample Image


In ARKit you can automatically add ARAnchors to your scene using different scenarios:

  • ARPlaneAnchor

    • If horizontal and/or vertical planeDetection instance property is ON, ARKit is able to add ARPlaneAnchors in the running session. Sometimes enabled planeDetection considerably increases a time required for scene understanding stage.
  • ARImageAnchor (conforms to ARTrackable protocol)

    • This type of anchors contains information about a position, orientation and scale of a detected image (anchor is placed at image's center) on world-tracking or image-tracking configuration. To activate image tracking, use detectionImages instance property. In ARKit 2.0 you can totally track up to 25 images, in ARKit 3.0 / 4.0 – up to 100 images, respectively. But, in both cases, not more than just 4 images simultaneously. However, it was promised, that in ARKit 5.0 / 6.0, you can detect and track up to 100 images at a time (but it's still not implemented yet).
  • ARBodyAnchor (conforms to ARTrackable protocol)

    • You can enable body tracking by running a session based on ARBodyTrackingConfig(). You'll get ARBodyAnchor at a Root Joint of CG Skeleton or, in other words, at pelvis position of a tracked character.
  • ARFaceAnchor (conforms to ARTrackable protocol)

    • Face Anchor stores information about head's topology, pose and face expression. You can track ARFaceAnchor with a help of the front TrueDepth camera. When face is detected, Face Anchor will be attached slightly behind a nose, in the center of a face. In ARKit 2.0 you can track just one face, in ARKit 3.0 and higher – up to 3 faces, simultaneously. However, the number of tracked faces depends on presence of a TrueDepth sensor and processor version: gadgets with TrueDepth camera can track up to 3 faces, gadgets with A12+ chipset, but without TrueDepth camera, can also track up to 3 faces.
  • ARObjectAnchor

    • This anchor's type keeps an information about 6 Degrees of Freedom (position and orientation) of a real-world 3D object detected in a world-tracking session. Remember that you need to specify ARReferenceObject instances for detectionObjects property of session config.
  • AREnvironmentProbeAnchor

    • Probe Anchor provides environmental lighting information for a specific area of space in a world-tracking session. ARKit's Artificial Intelligence uses it to supply reflective shaders with environmental reflections.
  • ARParticipantAnchor

    • This is an indispensable anchor type for multiuser AR experiences. If you want to employ it, use true value for isCollaborationEnabled property in ARWorldTrackingConfig. Then import MultipeerConnectivity framework.
  • ARMeshAnchor

    • ARKit and LiDAR subdivide the reconstructed real-world scene surrounding the user into mesh anchors with corresponding polygonal geometry. Mesh anchors constantly update their data as ARKit refines its understanding of the real world. Although ARKit updates a mesh to reflect a change in the physical environment, the mesh's subsequent change is not intended to reflect in real time. Sometimes your reconstructed scene can have up to 30-40 anchors or even more. This is due to the fact that each classified object (wall, chair, door or table) has its own personal anchor. Each ARMeshAnchor stores data about corresponding vertices, one of eight cases of classification, its faces and vertices' normals.
  • ARGeoAnchor (conforms to ARTrackable protocol)

    • In ARKit 4.0+ there's a geo anchor (a.k.a. location anchor) that tracks a geographic location using GPS, Apple Maps and additional environment data coming from Apple servers. This type of anchor identifies a specific area in the world that the app can refer to. When a user moves around the scene, the session updates a location anchor’s transform based on coordinates and device’s compass heading of a geo anchor. Look at the list of supported cities.
  • ARAppClipCodeAnchor (conforms to ARTrackable protocol)

    • This anchor tracks the position and orientation of App Clip Code in the physical environment in ARKit 4.0+. You can use App Clip Codes to enable users to discover your App Clip in the real world. There are NFC-integrated App Clip Code and scan-only App Clip Code.

Sample Image


There are also other regular approaches to create anchors in AR session:

  • Hit-Testing methods

    • Tapping on the screen, projects a point onto a invisible detected plane, placing ARAnchor on a location where imaginary ray intersects with this plane. By the way, ARHitTestResult class and its corresponding hit-testing methods for ARSCNView and ARSKView will be deprecated in iOS 14, so you have to get used to a Ray-Casting.
  • Ray-Casting methods

    • If you're using ray-casting, tapping on the screen results in a projected 3D point on an invisible detected plane. But you can also perform Ray-Casting between A and B positions in 3D scene. So, ray-casting can be 2D-to-3D and 3D-to-3D. When using the Tracked Ray-Casting, ARKit can keep refining the ray-cast as it learns more and more about detected surfaces.
  • Feature Points

    • Special yellow points that ARKit automatically generates on a high-contrast margins of real-world objects, can give you a place to put an ARAnchor on.
  • ARCamera's transform

    • iPhone's or iPad's camera position and orientation simd_float4x4 can be easily used as a place for ARAnchor.
  • Any arbitrary World Position

    • Place a custom ARWorldAnchor anywhere in your scene. You can generate ARKit's version of world anchor like AnchorEntity(.world(transform: mtx)) found in RealityKit.


This code snippet shows you how to use an ARPlaneAnchor in a delegate's method: renderer(_:didAdd:for:):

func renderer(_ renderer: SCNSceneRenderer, 
didAdd node: SCNNode,
for anchor: ARAnchor) {

guard let planeAnchor = anchor as? ARPlaneAnchor
else { return }

let grid = Grid(anchor: planeAnchor)
node.addChildNode(grid)
}



AnchorEntity

AnchorEntity is alpha and omega in RealityKit. According to RealityKit documentation 2019:

AnchorEntity is an anchor that tethers virtual content to a real-world object in an AR session.

RealityKit framework and Reality Composer app were announced at WWDC'19. They have a new class named AnchorEntity. You can use AnchorEntity as the root point of any entities' hierarchy, and you must add it to the Scene anchors collection. AnchorEntity automatically tracks real world target. In RealityKit and Reality Composer AnchorEntity is at the top of hierarchy. This anchor is able to hold a hundred of models and in this case it's more stable than if you use 100 personal anchors for each model.

Let's see how it looks in a code:

func makeUIView(context: Context) -> ARView {

let arView = ARView(frame: .zero)
let modelAnchor = try! Experience.loadModel()
arView.scene.anchors.append(modelAnchor)
return arView
}

AnchorEntity has three components:

  • Anchoring component
  • Transform component
  • Synchronization component

To find out the difference between ARAnchor and AnchorEntity look at THIS POST.

Here are nine AnchorEntity's cases available in RealityKit 2.0 for iOS:

// Fixed position in the AR scene
AnchorEntity(.world(transform: mtx))

// For body tracking (a.k.a. Motion Capture)
AnchorEntity(.body)

// Pinned to the tracking camera
AnchorEntity(.camera)

// For face tracking (Selfie Camera config)
AnchorEntity(.face)

// For image tracking config
AnchorEntity(.image(group: "GroupName", name: "forModel"))

// For object tracking config
AnchorEntity(.object(group: "GroupName", name: "forObject"))

// For plane detection with surface classification
AnchorEntity(.plane([.any], classification: [.seat], minimumBounds: [1, 1]))

// When you use ray-casting
AnchorEntity(raycastResult: myRaycastResult)

// When you use ARAnchor with a given identifier
AnchorEntity(.anchor(identifier: uuid))

// Creates anchor entity on a basis of ARAnchor
AnchorEntity(anchor: arAnchor)

And here are only two AnchorEntity's cases available in RealityKit 2.0 for macOS:

// Fixed world position in VR scene
AnchorEntity(.world(transform: mtx))

// Camera transform
AnchorEntity(.camera)

Also it’s not superfluous to say that you can use any subclass of ARAnchor for AnchorEntity needs:

func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {

guard let faceAnchor = anchors.first as? ARFaceAnchor
else { return }

arView.session.add(anchor: faceAnchor)

self.anchor = AnchorEntity(anchor: faceAnchor)
anchor.addChild(model)
arView.scene.anchors.append(self.anchor)
}


Reality Composer's anchors:

At the moment (February 2022) Reality Composer has just 4 types of AnchorEntities:

Sample Image

// 1a
AnchorEntity(plane: .horizontal)

// 1b
AnchorEntity(plane: .vertical)

// 2
AnchorEntity(.image(group: "GroupName", name: "forModel"))

// 3
AnchorEntity(.face)

// 4
AnchorEntity(.object(group: "GroupName", name: "forObject"))


AR USD Schemas

And of course, I should say a few words about preliminary anchors. There are 3 preliminary anchoring types (July 2022) for those who prefer Python scripting for USDZ models – these are plane, image and face preliminary anchors. Look at this code snippet to find out how to implement a schema pythonically.

def Cube "ImageAnchoredBox"(prepend apiSchemas = ["Preliminary_AnchoringAPI"])
{
uniform token preliminary:anchoring:type = "image"
rel preliminary: imageAnchoring:referenceImage = <ImageReference>

def Preliminary_ReferenceImage "ImageReference"
{
uniform asset image = @somePicture.jpg@
uniform double physicalWidth = 45
}
}

If you want to know more about AR USD Schemas, read this story on Meduim.


Visualizing AnchorEntity

Here's an example of how to visualize anchors in RealityKit (mac version).

import AppKit
import RealityKit

class ViewController: NSViewController {

@IBOutlet var arView: ARView!
var model = Entity()
let anchor = AnchorEntity()

fileprivate func visualAnchor() -> Entity {

let colors: [SimpleMaterial.Color] = [.red, .green, .blue]

for index in 0...2 {

let box: MeshResource = .generateBox(size: [0.20, 0.005, 0.005])
let material = UnlitMaterial(color: colors[index])
let entity = ModelEntity(mesh: box, materials: [material])

if index == 0 {
entity.position.x += 0.1

} else if index == 1 {
entity.transform = Transform(pitch: 0, yaw: 0, roll: .pi/2)
entity.position.y += 0.1

} else if index == 2 {
entity.transform = Transform(pitch: 0, yaw: -.pi/2, roll: 0)
entity.position.z += 0.1
}
model.scale *= 1.5
self.model.addChild(entity)
}
return self.model
}

override func awakeFromNib() {
anchor.addChild(self.visualAnchor())
arView.scene.addAnchor(anchor)
}
}

Sample Image

What's the difference between ARAnchor and AnchorEntity?

Updated: July 10, 2022.


ARAnchor and AnchorEntity classes were both made for the same divine purpose – to tether 3D models to your real-world objects.

RealityKit AnchorEntity greatly extends the capabilities of ARKit ARAnchor. The most important difference between these two is that AnchorEntity automatically tracks a real world target, but ARAnchor needs session(...) instance method (or SceneKit's renderer(...) instance method) to accomplish this. Take into consideration that the collection of ARAnchors is stored in the ARSession object and the collection of AnchorEntities is stored in the Scene.

In addition, generating ARAchors requires a manual Session config, while generating AnchorEntities requires minimal developer's involvement.

Hierarchical differences:

Sample Image

The main advantage of RealityKit is the ability to use different AnchorEntities at the same time, such as .plane, .body or .object. There's automaticallyConfigureSession instance property in RealityKit. When enabled, the ARView automatically runs an ARSession with a config that will get updated depending on your camera mode and scene anchors. When disabled, the session needs to be run manually with your own config.

arView.automaticallyConfigureSession = true           // default

In ARKit, as you know, you can run just one config in the current session: World, Body, or Geo. There is an exception in ARKit, however - you can run two configs together - FaceTracking and WorldTracking (one of them has to be a driver, and the other one – driven).

let config = ARFaceTrackingConfiguration()
config.isWorldTrackingEnabled = true
arView.session.run(config)


Apple Developer documentation says:

In RealityKit framework you use an AnchorEntity instance as the root of an entity hierarchy, and add it to the anchors collection for a Scene instance. This enables ARKit to place the anchor entity, along with all of its hierarchical descendants, into the real world. In addition to the components the anchor entity inherits from the Entity class, the anchor entity also conforms to the HasAnchoring protocol, giving it an AnchoringComponent instance.

AnchorEntity has three building blocks:

  • Transform component (transformation matrix containing translate, rotate and scale)
  • Synchronization component (entity's synchronization data for multiuser experience)
  • Anchoring component (allows choose a type of anchor – world, body or image)


All entities have Synchronization component that helps organise collaborative sessions.

Sample Image


AnchorEntity has nine specific anchor types for nine different purposes:

  • ARAnchor
    • helps implement 10 ARKit anchors, including ARGeoAnchor and ARAppClipCodeAnchor
  • body
  • camera
  • face
  • image
  • object
  • plane
  • world
  • raycastResult



You can simultaneously use both classes ARAnchor and AnchorEntity in your app. Or you can use just AnchorEntity class because it's all-sufficient one.

For additional info about ARAnchor and AnchorEntity, please look at THIS POST.

RealityKit – What's the difference between Anchor and Entity translation?

In RealityKit, both AnchorEntity and ModelEntity are subclasses of their parental Entity class, which means that both of these subclasses have a Transform component. This implies that both objects can be moved, rotated and scaled.

The difference between ModelEntity and AnchorEntity is that an anchor is a special object that is automatically tracked in RealityKit. Model or several models need to be attached to the anchor, to prevent models from drifting. Thus, you can move the anchor itself (together with the models attached to it) and each model individually, if you need to.


Answering your question:

For an anchor with a single child, is there any difference between repositioning the anchor and repositioning the child entity?

In that particular case there's no difference. The only thing you need to think of is THIS.

What's the difference between ARSession delegate methods?

Foreword

I agree with you that Apple's documentation is often unclear.

ARKit and/or RealityKit apps are built on a running ARSession object that is a key AR element based on a specific configuration. Each configuration type in ARKit allows you to generate specific ARAnchors (ARPlaneAnchor, ARImageAnchor, ARFaceAnchor, etc). In ARKit, we must use an explicit (manual) configuration, while in RealityKit, a configuration is automatic, due to the fact it's set according to the AnchorEntity type (.plane, .image, .face, etc).

ARKit anchors can be tracked by implementing the session(...) or renderer(...) delegate methods. ARKit anchors may be accessed through each ARFrame (60 frames per second).

arView.session.currentFrame?.anchors

RealityKit anchors are automatically (implicitly) tracked by the application. RealityKit anchors' collection can be accessed through the Scene object.

arView.scene.anchors

The ARKit and RealityKit frameworks can work together or separately from each other. RealityKit's AnchorEntity is capable of using ARKit's ARAnchor transform to place a model into a scene:

self.anchorEntity = AnchorEntity(anchor: arAnchor)

Use case

ARSessionDelegate protocol has 4 optional session(...) methods that you can use with both frameworks. The first pair of session(...) methods are used quite often. session(_:didAdd:) instance method is used to add/extract specific anchors to/from an ARSession under certain conditions. However, session(_:didUpdate:) instance method allows you to update the content, based on a specific ARAnchor's data. It can be accomplished this way:

func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {

guard let faceAnchor = anchors.first as? ARFaceAnchor
else { return }

self.geometry.update(from: faceAnchor.geometry)
self.facialExpression(anchor: faceAnchor)
}


THIS POST shows you how to use both methods together.

And read this post to find out how to use session(_:didUpdate:) method alone.


What is difference between “real-world object” , surface, AR anchors?

My intuition is that whoever wrote Apple’s docs kept things ambiguous because a) you can use those methods for multiple kinds of hit tests, and b) ARKit doesn’t really know what it’s looking at.

If you do a hit test for any of the plane-related types (existingPlane, estimatedHorizontalPlane, etc), you’re looking for real-world flat surfaces. Or rather, you’re looking for things that ARKit “thinks” look like flat horizontal (or vertical, in iOS 11.3 and later) surfaces. Those might or might not accurately reflect the shape of the real world at any given moment, but they’re ARKit’s best guess. Which of the plane-related types you search for determines whether you get an existing ARAnchor.

(Note false negatives are more common than false positives. For example, you might find no hit result at a point where a corner of tabletop hasn’t been mapped by ARKit, but you’re unlikely to find a plane hit result with no corresponding real-world flat surface.)

If you do a featurePoint hit test, you’re testing against ARKit’s sparse map of the user’s environment. If you turn on the showFeaturePoints option in ARSCNView you can see this map — in each frame of video, it finds tens to small-hundreds of points that are visually interesting enough (well, “interesting” from a particular algorithm’s point of view) that it can correlate their 2D positions between frames and use parallax differences to estimate their distances from the camera and positions in 3D space. (In turn, that informs ARKit’s idea of where the device itself is in 3D space.)

Because a “feature point” can be any small, high-contrast region in the camera image, it doesn’t really correlate to any specific kind of real-world thing. If you’re looking at a desk with a nice wood-grain pattern, you’ll see a lot of feature points along the plane of the desktop. If you’re looking at a desk with, say, a potted plant on it, you’ll see some points on the desktop and some on the pot and some on the leaves... not enough points that you (or a CV algorithm) can really intuit the shape of the plant. But enough that, if the user were to tap on one of those points, your app could put some 3D object there and it might convincingly appear to stick to the plant.

So, in the most general sense, ARKit hit testing is looking for “objects” of some sort in the “real world” (as perceived by ARKit), but unless you’re looking for planes (surfaces) one can’t really be more specific about what the “objects” might be.

How to determine that ARObjectAnchor was removed from the scene?

Interpretation of informations from Apple TSI (id 731233593 if anyone wants to refer to this issue in his own TSI):

ARObjectAnchors are not necessarily removed when the object goes offscreen. It is not a behaviour that should be relied on. This applies to AR(SCNView|Session)Delegate.didRemove callbacks and to the content of ARFrame.anchors. The callbacks are definitely called only if the client code removes the relevant anchor programmatically. I was unable to squeeze a better explanation of the official doc line "ARKit may automatically remove anchors" despite asking explicitly. "You shouldn’t worry about why this happened."

Timing out on didAdd or didUpdate callbacks is an official method. The evidence is Apple's official project for creating object scans, file Controllers/TestRun.swift method startNoDetectionTimer. Using 5 seconds timeout.

Demands of ARObjectAnchor to implement ARTrackable are encouraged to fill an enhancement request at Feedback Assistant. I personally was encouraged to investigate alternatives and found that CoreML is unexpectedly friendly and fits my use case much better.

Thanks @Andy for pushing me this far.

RealityKit – Anchor is ignoring parent anchor transform

You don't need so many nested AnchorEntities. RealityKit's scene can contain any hierarchical structure of nested ModelEntities (also known as Parent-Child pairs). Nevertheless, in order to automatically track a position and orientation of every 3D element in this structure – just ONE anchor is needed (that's a regular way of working with anchors in any AR framework).

let container = Entity()
let modelA = ModelEntity()
let modelB = ModelEntity()

container.addChild(modelA)
modelB.setParent(modelA)

container.position.x = -1.2
modelA.position.x = 0.5 // adjust transform independently
modelB.position.y = 1

let anchor = AnchorEntity() // object being tracked
anchor.addChild(container)
arView.scene.addAnchor(anchor)

Where is the .camera AnchorEntity located?

Question I. Is the .camera anchor actually located right where the physical iPad / iPhone camera is located or is it located further back (perhaps where the user would normally hold the iPad / iPhone)?

Answer to first question

In RealityKit and ARKit frameworks ARCamera has a pivot point like other entities (nodes) have, and it's located at the point where lens is attached to the camera body (at bayonet level). This pivot can tether AnchorEntity(.camera). In other words, virtual camera and real-world camera have that pivot point approximately at the same place.

So, if you attach RealityKit's AnchorEntity to a camera's pivot, you place it to the coordinates where camera's bayonet is located. And this AnchorEntity(.camera) will be tracked automatically without a need to implement session(_:didUpdate:) method.

However, if attach ARKit's ARAnchor to the camera's pivot, you have to implement session(_:didUpdate:) method to constantly update a position and orientation of that anchor for every ARFrame.

Sample Image


Question II. How do you get a child entity of the AnchorEntity(.camera) to move as the iPad / camera moves in real space?

Answer to second question

If you want to constantly update model's position in RealityKits at 60 fps (when ARCamera moves and rotates) you need to use the following approach:

import ARKit
import RealityKit

class ViewController: UIViewController {

@IBOutlet var arView: ARView!

override func viewDidLoad() {
super.viewDidLoad()

let box = MeshResource.generateBox(size: 0.25)
let material = SimpleMaterial(color: .systemPink, isMetallic: true)
let boxEntity = ModelEntity(mesh: box, materials: [material])

let cameraAnchor = AnchorEntity(.camera) // ARCamera anchor
cameraAnchor.addChild(boxEntity)
arView.scene.addAnchor(cameraAnchor)

boxEntity.transform.translation = [0, 0,-1] // Box offset 1 m
}
}

...Or you could implement ARKits currentFrame property inside session(_:didUpdate:) method:

extension ViewController: ARSessionDelegate {

func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {

guard let transform = arView.session.currentFrame?.camera.transform
else { return }

let arkitAnchor = ARAnchor(transform: transform)
arView.session.add(anchor: arkitAnchor) // add to session

let anchor = AnchorEntity(anchor: arkitAnchor)
anchor.addChild(boxEntity)
arView.scene.addAnchor(anchor) // add to scene
}
}

class ViewController: UIViewController {

@IBOutlet var arView: ARView!
var boxEntity = ModelEntity(...)

override func viewDidLoad() {
super.viewDidLoad()
arView.session.delegate = self // Session's delegate
}
}


Leave a reply



Submit