Programmatically Setting Texture in Scene Generated by Reality Composer

Programmatically setting texture in Scene generated by Reality Composer

You forgot to set a boxComponent which need to be stored on the entity.

For this use set() instance method.

In your code it should look like this:

anchor.cylinder!.components.set(cylinderModelComponent)

In my code it looks like this:

anchor.steelBox!.components.set(boxComponent)

Here are my full code version (I tested it in macOS app):

import AppKit
import RealityKit

class GameViewController: NSViewController {

    @IBOutlet var arView: ARView!

    override func awakeFromNib() {

        let anchor = try! Experience.loadBox()
        anchor.steelBox?.scale = SIMD3(x: 9, y: 9, z: 9)
        anchor.steelBox?.orientation = simd_quatf(angle: -Float.pi/4,
                                                   axis: SIMD3(x: 1, y: 1, z: 0))      

        let boxEntity: Entity = anchor.steelBox!.children[0]
        var boxComponent: ModelComponent = boxEntity.components[ModelComponent].self!

        let paths: NSArray = NSSearchPathForDirectoriesInDomains(.documentDirectory, 
                                                                 .userDomainMask, 
                                                                  true) as NSArray

        // If you're testing it in macOS app – place your Image.png here:
        // /Users/<UserName>/Library/Containers/<ApplicationName>/Data/Documents/

        let path: NSString = paths.object(at: 0) as! NSString
        let filePath: NSString = path.strings(byAppendingPaths: ["Image.png"])[0] as NSString
        let url = URL(fileURLWithPath: filePath as String)

        var material = SimpleMaterial()
        material.tintColor = NSColor.yellow
        material.baseColor = try! MaterialColorParameter.texture(TextureResource.load(contentsOf: url))
        material.roughness = MaterialScalarParameter(floatLiteral: 0.1)
        material.metallic = MaterialScalarParameter(floatLiteral: 0.1)    
        boxComponent.materials = [material]

        anchor.steelBox!.components.set(boxComponent)
        arView.scene.anchors.append(anchor)
    }
}

How to Add Material to ModelEntity programatically in RealityKit?

Updated: June 14, 2022

RealityKit materials

There are 6 types of materials in RealityKit 2.0 and RealityFoundation at the moment:

• SimpleMaterial
• UnlitMaterial
• OcclusionMaterial (read this post to find out how to setup SceneKit occlusion shader)
• VideoMaterial (look at this post to find out how to setup it)
• PhysicallyBasedMaterial
• CustomMaterial

To apply these materials use the following logic:

import Cocoa
import RealityKit

class ViewController: NSViewController {        
    @IBOutlet var arView: ARView!
    
    override func awakeFromNib() {
        let box = try! Experience.loadBox()
        
        var simpleMat = SimpleMaterial()
        simpleMat.color = .init(tint: .blue, texture: nil)
        simpleMat.metallic = .init(floatLiteral: 0.7)
        simpleMat.roughness = .init(floatLiteral: 0.2)
        
        var pbr = PhysicallyBasedMaterial()
        pbr.baseColor = .init(tint: .green, texture: nil) 

        let mesh: MeshResource = .generateBox(width: 0.5, 
                                             height: 0.5, 
                                              depth: 0.5, 
                                       cornerRadius: 0.02, 
                                         splitFaces: true)

        let boxComponent = ModelComponent(mesh: mesh,
                                     materials: [simpleMat, pbr])

        box.steelBox?.children[0].components.set(boxComponent)
        box.steelBox?.orientation = Transform(pitch: .pi/4, 
                                                yaw: .pi/4, 
                                               roll: 0).rotation
        arView.scene.anchors.append(box)
    }
}

Read this post to find out how to load a texture for RealityKit's shaders.

How to create RealityKit's shaders similar to SceneKit's shaders

We know that in SceneKit there are 5 different shading models, so we can use RealityKit's SimpleMaterial, PhysicallyBasedMaterial and UnlitMaterial to generate all these five shaders that we've been accustomed to.

Let's see how it looks like:

SCNMaterial.LightingModel.blinn           – SimpleMaterial(color: . gray,
                                                        roughness: .float(0.5),
                                                       isMetallic: false)

SCNMaterial.LightingModel.lambert         – SimpleMaterial(color: . gray,
                                                        roughness: .float(1.0),
                                                       isMetallic: false)
  
SCNMaterial.LightingModel.phong           – SimpleMaterial(color: . gray,
                                                        roughness: .float(0.0),
                                                       isMetallic: false)

SCNMaterial.LightingModel.physicallyBased – PhysicallyBasedMaterial()

// all three shaders (`.constant`, `UnlitMaterial` and `VideoMaterial `) 
// don't depend on lighting
SCNMaterial.LightingModel.constant        – UnlitMaterial(color: .gray)
                                          – VideoMaterial(avPlayer: avPlayer)

Is there a way to programmatically change the material of an Entity that was created in Reality Composer?

Model entity is stored deeper in RealityKit's hierarchy, and as you said, it's Entity, not ModelEntity. So use downcasting to access mesh and materials:

import UIKit
import RealityKit

class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        let boxScene = try! Experience.loadBox()
        print(boxScene)
        
        let modelEntity = boxScene.steelBox?.children[0] as! ModelEntity
        let material = SimpleMaterial(color: .green, isMetallic: false)
        modelEntity.model?.materials = [material]
        
        let anchor = AnchorEntity()
        anchor.scale = [5,5,5]
        modelEntity.setParent(anchor)
        arView.scene.anchors.append(anchor)
    }
}

RealityKit – How to access the property in a Scene programmatically?

Of course, you need to look for the required ModelEntity in the depths of the model's hierarchy.

Use this SwiftUI solution:

struct ARViewContainer: UIViewRepresentable {
    
    func makeUIView(context: Context) -> ARView {
        
        let arView = ARView(frame: .zero)
        let pictureScene = try! Experience.loadPicture()
        pictureScene.children[0].scale *= 4
        
        print(pictureScene)
        
        let edgingModel = pictureScene.masterpiece?.children[0] as! ModelEntity

        edgingModel.model?.materials = [SimpleMaterial(color: .brown,
                                                  isMetallic: true)]
        
        var mat = SimpleMaterial()

        // Here's a great old approach for assigning a texture in iOS 14.5
        mat.baseColor = try! .texture(.load(named: "MonaLisa", in: nil))
        
        let imageModel = pictureScene.masterpiece?.children[0]
                                                  .children[0] as! ModelEntity
        imageModel.model?.materials = [mat]
        
        arView.scene.anchors.append(pictureScene)
        return arView
    }

    func updateUIView(_ uiView: ARView, context: Context) { }
}

How to preserve behaviors from Reality Composer?

The problem is: in RealityKit 2.0 we don't have access to the Reality Composer's behaviors (except notification), and the anchor type – we know that it's AnchorEntity(.image) – you assigned in Reality Composer works the same way as ARKit's image anchor in ARImageTrackingConfig, i.e. it uses local tracking instead of global tracking. In other words, hardly you get such a scenario with current Reality Composer scene.

Try ARKit and its ARWorldTrackingConfig for image tracking. All the "behaviors" must be recreated from scratch in RealityKit or in SceneKit.

Move the centre of the force within Reality Composer scene

Physics forces are applied to the model's pivot position. At the moment, neither RealityKit nor Reality Composer has the ability to change the location of an object's pivot point.

In addition to the above, you've applied Add Force behavior that pushes an object along a specific vector with the definite velocity, however, user's taps occur along the local -Z axis of the screen. Do these vectors match?

And one more note: within the Reality-family, only rigid body dynamics is possible, not soft body.

---

Related Topics