ARKit – How to put 3D Object on QRCode?
You could use Apple's Vision API to detect the QR code and place an anchor.
To start detecting QR codes, use:
var qrRequests = [VNRequest]()
var detectedDataAnchor: ARAnchor?
var processing = false
func startQrCodeDetection() {
// Create a Barcode Detection Request
let request = VNDetectBarcodesRequest(completionHandler: self.requestHandler)
// Set it to recognize QR code only
request.symbologies = [.QR]
self.qrRequests = [request]
}
In ARSession
's didUpdate Frame
public func session(_ session: ARSession, didUpdate frame: ARFrame) {
DispatchQueue.global(qos: .userInitiated).async {
do {
if self.processing {
return
}
self.processing = true
// Create a request handler using the captured image from the ARFrame
let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: frame.capturedImage,
options: [:])
// Process the request
try imageRequestHandler.perform(self.qrRequests)
} catch {
}
}
}
Handle the Vision QR request and trigger the hit test
func requestHandler(request: VNRequest, error: Error?) {
// Get the first result out of the results, if there are any
if let results = request.results, let result = results.first as? VNBarcodeObservation {
guard let payload = result.payloadStringValue else {return}
// Get the bounding box for the bar code and find the center
var rect = result.boundingBox
// Flip coordinates
rect = rect.applying(CGAffineTransform(scaleX: 1, y: -1))
rect = rect.applying(CGAffineTransform(translationX: 0, y: 1))
// Get center
let center = CGPoint(x: rect.midX, y: rect.midY)
DispatchQueue.main.async {
self.hitTestQrCode(center: center)
self.processing = false
}
} else {
self.processing = false
}
}
func hitTestQrCode(center: CGPoint) {
if let hitTestResults = self.latestFrame?.hitTest(center, types: [.featurePoint] ),
let hitTestResult = hitTestResults.first {
if let detectedDataAnchor = self.detectedDataAnchor,
let node = self.sceneView.node(for: detectedDataAnchor) {
let previousQrPosition = node.position
node.transform = SCNMatrix4(hitTestResult.worldTransform)
} else {
// Create an anchor. The node will be created in delegate methods
self.detectedDataAnchor = ARAnchor(transform: hitTestResult.worldTransform)
self.sceneView.session.add(anchor: self.detectedDataAnchor!)
}
}
}
Then handle adding the node when the anchor is added.
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
// If this is our anchor, create a node
if self.detectedDataAnchor?.identifier == anchor.identifier {
let sphere = SCNSphere(radius: 1.0)
sphere.firstMaterial?.diffuse.contents = UIColor.redColor()
let sphereNode = SCNNode(geometry: sphere)
sphereNode.transform = SCNMatrix4(anchor.transform)
return sphereNode
}
return nil
}
Source
ARKit : Translate a 3D Object on camera view
You need to pass the coordinates of the object in a SCNMatrix4 form as follows:
let translationMatrix = SCNMatrix4Translate(theNode.worldTransform, 0.1, 0.1, 0.1) //tx, ty, tz are translations in each axis i
and then theNode.transform = translation matrix
How to place a 3D model on top of a reference image with RealityKit?
Solution I
To make your code work properly, remove this line:
anchor.transform.matrix = position
@TimLangner – "...But I want to make the model appear on top of the reference image and not at any other location..."
As you can see, my test sphere has appeared on top of the reference image. When changing its position, remember that the Y-axis of the image anchor is directed towards the camera if the QR is vertical, and is directed up if the QR code is on a horizontal surface.
Make sure, a pivot point is located where it should.
In my case (you can see that I'm using AppClipCode), to move sphere 30 cm up, I have to move along negative Z direction.
entity.position.z = anchor.position.z - 0.3
Solution II
In my opinion, the most simple and productive solution would be to use the RealityKit's native AnchorEntity(.image(...)), without the need of implementing ARImageAnchor
in delegate's method.
AnchorEntity(.image(group: "GroupName", name: "forModel"))
Here's a code:
import UIKit
import RealityKit
class ViewController: UIViewController {
@IBOutlet var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
let entity = ModelEntity(mesh: .generateSphere(radius: 0.1))
let anchor = AnchorEntity(.image(group: "AR Resources",
name: "appClipCode"))
anchor.addChild(entity)
entity.position.z = anchor.position.z - 0.3
arView.scene.anchors.append(anchor)
}
}
Related Topics
Nsuserdefaults in Swift - Implementing Type Safety
Swift: How to Fully Strip Internal/Inline Symbols
Get the Current Position of Scrollview in Swiftui
How to Show an Alert from Another Class in Swift
Swift - How to Save Audio from Avaudioengine, or from Audioplayernode? If Yes, How
Why Do Some Types (E.G. Float80) Have a Memory Alignment Bigger Than Word Size
Swiftui MACos Xcode Style Toolbar
Scenekit - Get Direction of Camera
Property Cannot Be Declared Public Because Its Type Uses an Internal Type
Convert String to Cgfloat in Swift
Value for Swift_Version Cannot Be Empty
Convert Swift Dictionary to String
How to Update Swift Dependencies in Xcode
Class Level or Struct Level Method in Swift Like Static Method in Java
Why Is My Swift Loop Failing with Error "Can't Form Range with End < Start"