How to Add Material to Modelentity Programatically in Realitykit

How to Add Material to ModelEntity programatically in RealityKit?

Updated: June 14, 2022

RealityKit materials

There are 6 types of materials in RealityKit 2.0 and RealityFoundation at the moment:

• SimpleMaterial
• UnlitMaterial
• OcclusionMaterial (read this post to find out how to setup SceneKit occlusion shader)
• VideoMaterial (look at this post to find out how to setup it)
• PhysicallyBasedMaterial
• CustomMaterial

To apply these materials use the following logic:

import Cocoa
import RealityKit

class ViewController: NSViewController {        
    @IBOutlet var arView: ARView!
    
    override func awakeFromNib() {
        let box = try! Experience.loadBox()
        
        var simpleMat = SimpleMaterial()
        simpleMat.color = .init(tint: .blue, texture: nil)
        simpleMat.metallic = .init(floatLiteral: 0.7)
        simpleMat.roughness = .init(floatLiteral: 0.2)
        
        var pbr = PhysicallyBasedMaterial()
        pbr.baseColor = .init(tint: .green, texture: nil) 

        let mesh: MeshResource = .generateBox(width: 0.5, 
                                             height: 0.5, 
                                              depth: 0.5, 
                                       cornerRadius: 0.02, 
                                         splitFaces: true)

        let boxComponent = ModelComponent(mesh: mesh,
                                     materials: [simpleMat, pbr])

        box.steelBox?.children[0].components.set(boxComponent)
        box.steelBox?.orientation = Transform(pitch: .pi/4, 
                                                yaw: .pi/4, 
                                               roll: 0).rotation
        arView.scene.anchors.append(box)
    }
}

Read this post to find out how to load a texture for RealityKit's shaders.

How to create RealityKit's shaders similar to SceneKit's shaders

We know that in SceneKit there are 5 different shading models, so we can use RealityKit's SimpleMaterial, PhysicallyBasedMaterial and UnlitMaterial to generate all these five shaders that we've been accustomed to.

Let's see how it looks like:

SCNMaterial.LightingModel.blinn           – SimpleMaterial(color: . gray,
                                                        roughness: .float(0.5),
                                                       isMetallic: false)

SCNMaterial.LightingModel.lambert         – SimpleMaterial(color: . gray,
                                                        roughness: .float(1.0),
                                                       isMetallic: false)
  
SCNMaterial.LightingModel.phong           – SimpleMaterial(color: . gray,
                                                        roughness: .float(0.0),
                                                       isMetallic: false)

SCNMaterial.LightingModel.physicallyBased – PhysicallyBasedMaterial()


// all three shaders (`.constant`, `UnlitMaterial` and `VideoMaterial `) 
// don't depend on lighting
SCNMaterial.LightingModel.constant        – UnlitMaterial(color: .gray)
                                          – VideoMaterial(avPlayer: avPlayer)

Is there a way to programmatically change the material of an Entity that was created in Reality Composer?

Model entity is stored deeper in RealityKit's hierarchy, and as you said, it's Entity, not ModelEntity. So use downcasting to access mesh and materials:

import UIKit
import RealityKit

class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        let boxScene = try! Experience.loadBox()
        print(boxScene)
        
        let modelEntity = boxScene.steelBox?.children[0] as! ModelEntity
        let material = SimpleMaterial(color: .green, isMetallic: false)
        modelEntity.model?.materials = [material]
        
        let anchor = AnchorEntity()
        anchor.scale = [5,5,5]
        modelEntity.setParent(anchor)
        arView.scene.anchors.append(anchor)
    }
}

Add UIImage as texture to a Plane in RealityKit

Currently, you cannot use UIImage or CIImage as a shader's texture in RealityKit 2.0. In both versions of RealityKit, the texture must be loaded via the String type parameter of the load() method.

RealityKit 2.0

To assign a texture to a shader in RealityKit 2.0 use the following approach:

let mesh: MeshResource = .generatePlane(width: 0.45, depth: 0.45)
        
var material = SimpleMaterial()
material.color = .init(tint: .white.withAlphaComponent(0.999),
                    texture: .init(try! .load(named: "texture.png")))
material.metallic = .float(1.0)
material.roughness = .float(0.0)

let model = ModelEntity(mesh: mesh, materials: [material])

RealityKit 1.0

To assign a texture to a shader in RealityKit 1.0 use this approach:

let scene = try! Experience.loadMyScene()

var material = SimpleMaterial()
material.baseColor = try! .texture(.load(named: "texture.png"))
material.metallic = MaterialScalarParameter(floatLiteral: 1.0)
material.roughness = MaterialScalarParameter(floatLiteral: 0.0)
material.tintColor = UIColor.white

let mesh: MeshResource = .generatePlane(width: 0.45, depth: 0.45)
let component = ModelComponent(mesh: mesh, materials: [material])

scene.myFavoriteScene?.children[0].components.set(component)
arView.scene.anchors.append(scene)

CGImage

Nonetheless, you can create a texture resource from an in-memory Core Graphics image:

static func generate(from: CGImage, 
                 withName: String?, 
                  options: TextureResource.CreateOptions) -> TextureResource

Also, you can use a URL parameter:

material.color.texture = .init(try! .load(contentsOf: url))  // RealityKit 2.0

RealityKit – How to add motion to a loaded ModelEntity from USDZ file?

Add to your code .generateCollisionShapes(recursive:) instance method for every participant (entity) that not only creates collision's shapes but also allows you to simulate physics.

import SwiftUI
import RealityKit

struct ARViewContainer: UIViewRepresentable {
    
    let boxx = ModelEntity(mesh: .generateBox(size: 0.5))
    let ball = ModelEntity(mesh: .generateSphere(radius: 0.25))
    let anchor = AnchorEntity()
    
    func makeUIView(context: Context) -> ARView {
        let arView = ARView(frame: .zero)
        // BALL
        ball.physicsBody = .init()
        ball.physicsMotion = .init()
        ball.physicsMotion?.linearVelocity = [10,0,0]
        ball.position.x = -3
        ball.generateCollisionShapes(recursive: true)
        anchor.addChild(ball)
        // BOX
        boxx.physicsBody = .init()
        boxx.physicsMotion = .init()
        boxx.physicsMotion?.linearVelocity = [0,2,0]
        boxx.generateCollisionShapes(recursive: true)
        anchor.addChild(boxx)
        // Anchor
        anchor.position.z = -3
        arView.scene.addAnchor(anchor)
        return arView
    }
    func updateUIView(_ uiView: ARView, context: Context) { }
}

...

struct ContentView : View {
    var body: some View {
        return ARViewContainer().edgesIgnoringSafeArea(.all)
    }
}

For more details, read THIS post.

RealityKit – How to access the property in a Scene programmatically?

Of course, you need to look for the required ModelEntity in the depths of the model's hierarchy.

Use this SwiftUI solution:

struct ARViewContainer: UIViewRepresentable {
    
    func makeUIView(context: Context) -> ARView {
        
        let arView = ARView(frame: .zero)
        let pictureScene = try! Experience.loadPicture()
        pictureScene.children[0].scale *= 4
        
        print(pictureScene)
        
        let edgingModel = pictureScene.masterpiece?.children[0] as! ModelEntity

        edgingModel.model?.materials = [SimpleMaterial(color: .brown,
                                                  isMetallic: true)]
        
        var mat = SimpleMaterial()

        // Here's a great old approach for assigning a texture in iOS 14.5
        mat.baseColor = try! .texture(.load(named: "MonaLisa", in: nil))
        
        let imageModel = pictureScene.masterpiece?.children[0]
                                                  .children[0] as! ModelEntity
        imageModel.model?.materials = [mat]
        
        arView.scene.anchors.append(pictureScene)
        return arView
    }

    func updateUIView(_ uiView: ARView, context: Context) { }
}

---

Related Topics