Scenekit Ar Game Fps Getting Low and The Device Getting Hot with Use

SceneKit AR game fps getting low and the device getting hot with use

Using ARKit's 6 DoF tracking at 60 fps along with SceneKit's or RealityKit's rendering (of poly geometry, PBR shaders and shadows) at 60 fps is extremely heavy burden for the processor and graphical unit. It also drains your battery very fast and makes the phone generate a lot of heat. For AR, VR and AI apps it's a common issue though.

To lower an energy impact follow these recommendations:

• Keep the total number of polygons of your model not more than 10K
• Use not more than 50K polygons per a scene (use low-poly models)
• Don't use ray-traced shadows (use pre-baked or fake shadows)
• Try not to use many PBR shaders, like metallic or reflective materials
• Try to lower a tracking frame rate and, if needed, a rendering frame rate
• Take into consideration that physics consumes much processing power
• Do not use high-resolution JPEG or PNG textures for your 3D models

Low Framerate in SceneKit

The issue was that Physics was being called on every frame, then performing some logic. The player was hitting the floor, used to center the block, every frame. I changed the contact bit mask and all the issues went away.

Get CVPixelBuffer (camera frame along with ar models) in a performant manner

You can create your own CVPixelBuffer, then get a MTLTexture from it and finally have SCNRenderer render into that texture. This won't involve the CPU cost of generating a UIImage from the snapshot API.

The key part here is to use a CVMetalTextureCache to obtain a Metal texture from a pixel buffer.

Run and Pause an ARSession in a specified period of time

I don't think the run() and pause() strategy is the way to go because the DispatchQueue API is not designed for realtime accuracy. Which means there will be no guarantee that the pause will be 16ms every time. On top of that, restarting a session might not be immediate and could add more delay.

Also, the code you shared will at most capture only one image and as session.run(configuration) is asynchronous will probably capture no frame.

As you're not using ARSCNView/ARSKView the only way is to implement the ARSession delegate to be notified of every captured frame.

Of course the delegate will most likely be called every 16ms because that's how the camera works. But you can decide which frames you are going to process. By using the timestamp of the frame you can process a frame every 32ms and drop the other ones. Which is equivalent to a 30 fps processing.

Here is some code to get you started, make sure that dispatchQueue is not concurrent to process your buffers sequentially:

var lastProcessedFrame: ARFrame?

func session(_ session: ARSession, didUpdate frame: ARFrame) {
  dispatchQueue.async {
    self.updateCoreML(with: frame)
  }
}

private func shouldProcessFrame(_ frame: ARFrame) -> Bool {
  guard let lastProcessedFrame = lastProcessedFrame else {
    // Always process the first frame
    return true
  }
  return frame.timestamp - lastProcessedFrame.timestamp >= 0.032 // 32ms for 30fps
}

func updateCoreML(with frame: ARFrame) {

  guard shouldProcessFrame(frame) else {
    // Less than 32ms with the previous frame
    return
  }
  lastProcessedFrame = frame
  let pixelBuffer = frame.capturedImage
  let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:])
  do {
    try imageRequestHandler.perform(self.visionRequests)
  } catch {
    print(error)
  }
}

What does the SceneKit statistics window tell us?

Here's what you see (note: I know most of them from experience so feel free to correct me in comments if I'm wrong):

• The red and grey bar is kind of a performance review. It's based on the FPS and you should do everything to keep it green and full. Right now, it's pretty bad!
• GL tells you which rendering engine you're using
• 6FPS is your framerate. That means how many times your screen is updated in one second. Your target should be 60, (it's the maximum, and what is expected from modern games), but 30 is acceptable.
• That diamond with the 6 is the node count, i.e. how many nodes are in your scene graph.
• 40.3k is the polycount, or the number of polygons in your scene. This seems pretty high considering you only have 6 nodes and can explain the low FPS.
• That donut chart is what each frame spends its time doing. In your case, you can see most of the time is spent on rendering (the section on the right explains the meaning for each color)
• 0.2s is the time spent to render each frame. It's directly linked to the framerate.

---

Related Topics