How to Create a Hud on Top of My Scenekit.Scene

How can I overlay a SKScene over a SCNScene in Swift?

The SpriteKit overlay goes on the SceneKit view, not on the SceneKit scene. This is a bit confusing because you're overlaying a scene on a view.

I see several possible error sources:

self.sceneView = MainScene(view: self.view)) 

as defined returns an SCNScene. You're assigning that to a property that expects an SCNView.

The line

scnView = view as! SCNView

will crash unless view returns a properly connected SCNView instance. But the init definition you've written expects a UIView.

Somewhere, you need to have your view be an SCNView. That view, because it conforms to protocol SCNSceneRenderer, will have an overlaySKScene property on it (overlaySKScene comes from that protocol, not from SCNView). That's where you can assign your SKScene instance.

If you have done that, then your code would look something like

scnView.scene = self 
scnView.overlaySKScene = theSKScene

I have a simple example of an SKScene overlaid on an SCNView at https://github.com/halmueller/ImmersiveInterfaces/tree/master/Tracking%20Overlay

See also How do I create a HUD on top of my Scenekit.scene.

How to create game menu ui (i.e. buttons, scroll views) with Scene Kit?

Depending on your requirements for a title screen, pause and even HUD elements, UIKit could work. I would not recommend UIKit if you need any fancy stuff like shaders, particle effects or a lot of heavy animations. In that case I would go with Sprite Kit as suggested by David Rönnqvist.

But if your needs are menu screens, simple HUD elements, dialogs etc. there are no problems too use UIKit with any needed custom views, and just treat the SCNView like any other UIKit view. The up side is of course that UIKit does a great job for common UI tasks and user interaction, and is very easy and simple to use. Sprite Kit can some times feel a bit heavy for UI in my opinion.

Recursion:: How to create a mini-view of 3d scenekit of self on top of self?

Here is how I did it:

You can simply add another camera to the scene, and render to a SCNLayer. Then, you can either use this layer within the scene as a material, or add it as a custom view on top of your scene.

SCNScene *scene2 =[SCNScene scene];

// We duplicate the scene by cloning the root node. 
// I found that you cannot share the scene if you 
// use the layer within it.
[[scene2 rootNode] addChildNode:[root clone]];

// Create a SCNLayer, set the scene and size
SCNLayer *scnlayer = [SCNLayer layer];
scnlayer.scene = scene2;
scnlayer.frame = CGRectMake(0, 0, 600, 800);

// "Layer" should be the name of your second camera
scnlayer.pointOfView = [scene.rootNode childNodeWithName:@"Layer" recursively:YES];

// Make sure it gets updated
scnlayer.playing = YES;

// Make a parent layer with a black background
CALayer *backgroundLayer = [CALayer layer];
backgroundLayer.backgroundColor = CGColorGetConstantColor(kCGColorBlack);
backgroundLayer.frame = CGRectMake(0, 0, 600, 800);

// Add the SCNLayer
[backgroundLayer addSublayer:scnlayer];

// Set the layer as the emissive material of an object
SCNMaterial *material = plane.geometry.firstMaterial;
material.emission.contents = scnlayer;

I'm pretty sure this is not a great way to do it, but it worked for me.

SceneKit and SpriteKit together

I am not answering your two questions directly, instead I will deal with your larger issue which is that you want a HUD on your game. A user interface. I never made a Scenekit game, but I did make a Spritekit game and the HUD is a clever trick. Let me explain.

Imagine you are standing behind a camera and looking through to see what the camera is seeing through its lens. You see the scene in front of it. Now take a piece of paper, cut out a rectangular box in the middle and draw on the remaining periphery. Tape this piece of paper to the front of the camera and look through. You still see the scene (a little smaller now), but now you also see the paper with the drawings. This is how HUDs work for the game.

I did some searching and both SpriteKit and SceneKit have a camera. In sceneKit you create a camera node and set the point of view of the scene to that camera. This represents the player looking out into the scene.

I am not sure if it works the same way in SceneKit, but in SpriteKit you can add nodes to the camera that will be locked in place. These become your HUD elements (buttons, score, etc). Since these are all child nodes of the camera they move when you move the camera so it feels like they are a static HUD. This is how you achieve a HUD. Read up on the documentation for SceneKit cameras as this will help you understand how to achieve the same effect.

I once also thought about doing it the way you are going now and it proved to be a very painful experience to mix SceneKit and SpriteKit so I would caution against it.

Also, while you can mix Interface Builder and SpriteKit I would caution against it. It basically ruined my first game into a buggy mess. If you do decide to use Interface Builder, try to limit it to menu screens and options screens. But, I found that programmatically creating everything gives you a much smoother, cleaner, and bug free experience.

Hope this helps!

Adding an SKLabelNode to Scene Kit View

The typical way to do a HUD for a SceneKit scene is to create a SpriteKit SKScene and set it as the overlaySKScene of your SceneKit view. Then it renders at full resolution and always at the same view-relative size and position.

SpriteKit and UIScrollView and HUD

I answered a similar question here: What is the right way to create scrollable table in SKSpriteKit?

To summarize - I suggest building upon ScrollKit. On top of that I recommend flipping the UIScrollView Y-coordinates to match SpriteKit's coordinate system, as well as using a custom SKNode (example of such a node here) that allows to set its contentOffset.

SwiftUI with SceneKit: How to use button action from view to manipulate underlying scene

I found a solution using @EnvironmentalObject but I am not completely sure, if this is the right approach. So comments on this are appreciated.

First, I moved the SCNScene into it’s own class and made it an OberservableObject:

class Scene: ObservableObject {
    @Published var scene: SCNScene

    init(_ scene: SCNScene = SCNScene()) {
        self.scene = scene
        self.scene = setup(scene: self.scene)
    }

    // code omitted which deals with setting the scene up and adding/removing the box

    // method used to determine if the box node is present in the scene -> used later on 
    func boxIsPresent() -> Bool {
        return scene.rootNode.childNode(withName: "box", recursively: true) != nil
    }
}

I inject this Scene into the app as an .environmentalObject(), so it is available to all views:

class SceneDelegate: UIResponder, UIWindowSceneDelegate {

    func scene(_ scene: UIScene, willConnectTo session: UISceneSession, options connectionOptions: UIScene.ConnectionOptions) {

        // Create the SwiftUI view that provides the window contents.
        let sceneKit = Scene()
        let mainView = MainView().environmentObject(sceneKit)

        // Use a UIHostingController as window root view controller.
        if let windowScene = scene as? UIWindowScene {
            let window = UIWindow(windowScene: windowScene)
            window.rootViewController = UIHostingController(rootView: mainView)
            self.window = window
            window.makeKeyAndVisible()
        }
    }
}

MainView is slightly altered to call SceneView (a UIViewRepresentable) with the separate Scene for the environment:

struct MainView: View {
    @EnvironmentObject var scene: Scene

    var body: some View {
        ZStack {
            SceneView(scene: self.scene.scene)
            HUDView()
        }
    }
}

Then I make the Scene available to the HUDView as an @EnvironmentalObject, so I can reference the scene and its methods and call them from the Button action. Another effect is, I can query the Scene helper method to determine, if a Button should be active or not:

struct HUDView: View {
    @EnvironmentObject var scene: Scene
    @State private var canAddBox: Bool = false
    @State private var canRemoveBox: Bool = true

    var body: some View {
        VStack {
            HStack(alignment: .center, spacing: 0) {
                Spacer ()

                ButtonView(
                    action: {
                        self.scene.addBox()
                        if self.scene.boxIsPresent() {
                            self.canAddBox = false
                            self.canRemoveBox = true
                        }
                    },
                    icon: "plus.square.fill",
                    isActive: $canAddBox
                )

                ButtonView(
                    action: {
                        self.scene.removeBox()
                        if !self.scene.boxIsPresent() {
                            self.canRemoveBox = false
                            self.canAddBox = true
                        }
                    },
                    icon: "minus.square.fill",
                    isActive: $canRemoveBox
                )

            }
            .background(Color.white.opacity(0.2))

            Spacer()
        }
    }
}

Here is the ButtonView code, which used a @Bindingto set its active state (not sure about the correct order for this with the@State property inHUDView`):

struct ButtonView: View {
    let action: () -> Void
    var icon: String = "square"
    @Binding var isActive: Bool

    var body: some View {
        Button(action: action) {
            Image(systemName: icon)
                .font(.title)
                .accentColor(self.isActive ? Color.white : Color.white.opacity(0.5))
        }
        .frame(width: 44, height: 44)
        .disabled(self.isActive ? false: true)

    }
}

Anyway, the code works now. Any thoughts on this?

---

Related Topics