Arkit - How to Put 3D Object on Qrcode

ARKit – How to put 3D Object on QRCode?

You could use Apple's Vision API to detect the QR code and place an anchor.

To start detecting QR codes, use:

 var qrRequests = [VNRequest]()
var detectedDataAnchor: ARAnchor?
var processing = false

func startQrCodeDetection() {
// Create a Barcode Detection Request
let request = VNDetectBarcodesRequest(completionHandler: self.requestHandler)
// Set it to recognize QR code only
request.symbologies = [.QR]
self.qrRequests = [request]
}

In ARSession's didUpdate Frame

public func session(_ session: ARSession, didUpdate frame: ARFrame) {
DispatchQueue.global(qos: .userInitiated).async {
do {
if self.processing {
return
}
self.processing = true
// Create a request handler using the captured image from the ARFrame
let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: frame.capturedImage,
options: [:])
// Process the request
try imageRequestHandler.perform(self.qrRequests)
} catch {

}
}
}

Handle the Vision QR request and trigger the hit test

func requestHandler(request: VNRequest, error: Error?) {
// Get the first result out of the results, if there are any
if let results = request.results, let result = results.first as? VNBarcodeObservation {
guard let payload = result.payloadStringValue else {return}
// Get the bounding box for the bar code and find the center
var rect = result.boundingBox
// Flip coordinates
rect = rect.applying(CGAffineTransform(scaleX: 1, y: -1))
rect = rect.applying(CGAffineTransform(translationX: 0, y: 1))
// Get center
let center = CGPoint(x: rect.midX, y: rect.midY)

DispatchQueue.main.async {
self.hitTestQrCode(center: center)
self.processing = false
}
} else {
self.processing = false
}
}

func hitTestQrCode(center: CGPoint) {
if let hitTestResults = self.latestFrame?.hitTest(center, types: [.featurePoint] ),
let hitTestResult = hitTestResults.first {
if let detectedDataAnchor = self.detectedDataAnchor,
let node = self.sceneView.node(for: detectedDataAnchor) {
let previousQrPosition = node.position
node.transform = SCNMatrix4(hitTestResult.worldTransform)

} else {
// Create an anchor. The node will be created in delegate methods
self.detectedDataAnchor = ARAnchor(transform: hitTestResult.worldTransform)
self.sceneView.session.add(anchor: self.detectedDataAnchor!)
}
}
}

Then handle adding the node when the anchor is added.

func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {

// If this is our anchor, create a node
if self.detectedDataAnchor?.identifier == anchor.identifier {
let sphere = SCNSphere(radius: 1.0)
sphere.firstMaterial?.diffuse.contents = UIColor.redColor()
let sphereNode = SCNNode(geometry: sphere)
sphereNode.transform = SCNMatrix4(anchor.transform)
return sphereNode
}
return nil
}

Source

ARKit : Translate a 3D Object on camera view

You need to pass the coordinates of the object in a SCNMatrix4 form as follows:

let translationMatrix = SCNMatrix4Translate(theNode.worldTransform, 0.1, 0.1, 0.1) //tx, ty, tz are translations in each axis i
and then theNode.transform = translation matrix

How to place a 3D model on top of a reference image with RealityKit?

Solution I

To make your code work properly, remove this line:

anchor.transform.matrix = position

Sample Image

@TimLangner – "...But I want to make the model appear on top of the reference image and not at any other location..."

As you can see, my test sphere has appeared on top of the reference image. When changing its position, remember that the Y-axis of the image anchor is directed towards the camera if the QR is vertical, and is directed up if the QR code is on a horizontal surface.

Make sure, a pivot point is located where it should.

In my case (you can see that I'm using AppClipCode), to move sphere 30 cm up, I have to move along negative Z direction.

entity.position.z = anchor.position.z - 0.3


Solution II

In my opinion, the most simple and productive solution would be to use the RealityKit's native AnchorEntity(.image(...)), without the need of implementing ARImageAnchor in delegate's method.

AnchorEntity(.image(group: "GroupName", name: "forModel"))

Here's a code:

import UIKit
import RealityKit

class ViewController: UIViewController {

@IBOutlet var arView: ARView!

override func viewDidLoad() {
super.viewDidLoad()

let entity = ModelEntity(mesh: .generateSphere(radius: 0.1))
let anchor = AnchorEntity(.image(group: "AR Resources",
name: "appClipCode"))
anchor.addChild(entity)

entity.position.z = anchor.position.z - 0.3
arView.scene.anchors.append(anchor)
}
}


Related Topics



Leave a reply



Submit