Accessibility (Voice Over) with Sprite Kit

Accessibility (Voice Over) with Sprite Kit

I've searched in vain for a description of how to implement VoiceOver in Swift using SpriteKit, so I finally figured out how to do it. Here's some working code that converts a SKNode to an accessible pushbutton when added to a SKScene class:

// Add the following code to a scene where you want to make the SKNode variable named “leave” an accessible button
// leave must already be initialized and added as a child of the scene, or a child of other SKNodes in the scene
// screenHeight must already be defined as the height of the device screen, in points

// Accessibility

private var accessibleElements: [UIAccessibilityElement] = []

private func nodeToDevicePointsFrame(node: SKNode) -> CGRect {

    // first convert from frame in SKNode to frame in SKScene's coordinates

    var sceneFrame = node.frame
    sceneFrame.origin = node.scene!.convertPoint(node.frame.origin, fromNode: node.parent!)

    // convert frame from SKScene coordinates to device points
    // sprite kit scene origin is in lower left, accessibility device screen origin is at upper left
    // assumes scene is initialized using SKSceneScaleMode.Fill using dimensions same as device points

    var deviceFrame = sceneFrame
    deviceFrame.origin.y = CGFloat(screenHeight-1) - (sceneFrame.origin.y + sceneFrame.size.height)
    return deviceFrame
}

private func initAccessibility() {
    if accessibleElements.count == 0 {
        let accessibleLeave = UIAccessibilityElement(accessibilityContainer: self.view!)
        accessibleLeave.accessibilityFrame = nodeToDevicePointsFrame(leave)
        accessibleLeave.accessibilityTraits = UIAccessibilityTraitButton
        accessibleLeave.accessibilityLabel = “leave” // the accessible name of the button
        accessibleElements.append(accessibleLeave)
    }
}
override func didMoveToView(view: SKView) {
    self.isAccessibilityElement = false
    leave.isAccessibilityElement = true
}

override func willMoveFromView(view: SKView) {
    accessibleElements = []
}

override func accessibilityElementCount() -> Int {
    initAccessibility()
    return accessibleElements.count
}

override func accessibilityElementAtIndex(index: Int) -> AnyObject? {
    initAccessibility()
    if (index < accessibleElements.count) {
        return accessibleElements[index] as AnyObject
    } else {
        return nil
    }
}

override func indexOfAccessibilityElement(element: AnyObject) -> Int {
    initAccessibility()
    return accessibleElements.indexOf(element as! UIAccessibilityElement)!
}

UIAccessibilityElement with button trait adding one of one after voice over speaks button

I can't figure out where "one of one" is coming from. Any idea what is causing Voice Over to add "one of one" after speaking the word button?

Every UIControl button you're creating in a cell with the .button trait will be vocalized the way you mentioned.

Whatever the number of created buttons in a tableviewcell, all will be vocalized with the same suffix indicating the cell they belong to and the total number of cells in the section.

"one of one" in your example means that your button is in the first cell and that you have only one cell in your section.

For instance, if you create two buttons in the third cell of a section containing ten cells, you will hear the suffix "three of ten" for your two buttons.

I hope this explanation is clear enough to understand where your "one of one" is coming from as desired.

Is it possible to use Xcode UI Testing on an app using SpriteKit?

The main idea is to create the accessibility material for elements that you want to UI test. That's mean:

1. List all accessible elements contained in the scene

2. Configure settings for each of these elements, especially framedata.

Step by Step

This answer is for Swift 3 and is mainly based on Accessibility (Voice Over) with Sprite Kit

Let's say I want to make the SpriteKit button named tapMe accessible.

List of accessible elements.

Add an array of UIAccessibilityElementto the Scene.

 var accessibleElements: [UIAccessibilityElement] = []

Scene's cycle life

I need to update two methods: didMove(to:)and willMove(from:).

override func didMove(to view: SKView) {
    isAccessibilityElement       = false
    tapMe.isAccessibilityElement = true
}

As scene is the accessibility controller, documentation stated it must return False to isAccessibilityElement.

And:

override func willMove(from view: SKView) {
    accessibleElements.removeAll()
}

Override UIAccessibilityContainer methods

3 methods are involved: accessibilityElementCount(), accessibilityElement(at index:) and index(ofAccessibilityElement. Please allow me to introduce an initAccessibility() method I'll describe later.

override func accessibilityElementCount() -> Int {
    initAccessibility()
    return accessibleElements.count
}

override func accessibilityElement(at index: Int) -> Any? {

    initAccessibility()
    if (index < accessibleElements.count) {
        return accessibleElements[index]
    } else {
        return nil
    }
}

override func index(ofAccessibilityElement element: Any) -> Int {
    initAccessibility()
    return accessibleElements.index(of: element as! UIAccessibilityElement)!
}

Initialize accessibility for the Scene

func initAccessibility() {

    if accessibleElements.count == 0 {

        // 1.
        let elementForTapMe   = UIAccessibilityElement(accessibilityContainer: self.view!)

        // 2.
        var frameForTapMe = tapMe.frame

        // From Scene to View
        frameForTapMe.origin = (view?.convert(frameForTapMe.origin, from: self))!

        // Don't forget origins are different for SpriteKit and UIKit:
        // - SpriteKit is bottom/left
        // - UIKit is top/left
        //              y
        //  ┌────┐       ▲
        //  │    │       │   x
        //  ◉────┘       └──▶
        //
        //                   x
        //  ◉────┐       ┌──▶
        //  │    │       │
        //  └────┘     y ▼
        //
        // Thus before the following conversion, origin value indicate the bottom/left edge of the frame.
        // We then need to move it to top/left by retrieving the height of the frame.
        //

        frameForTapMe.origin.y = frameForTapMe.origin.y - frameForTapMe.size.height

        // 3.
        elementForTapMe.accessibilityLabel   = "tap Me"
        elementForTapMe.accessibilityFrame   = frameForTapMe
        elementForTapMe.accessibilityTraits  = UIAccessibilityTraitButton

        // 4.
        accessibleElements.append(elementForTapMe)

    }
}

1. Create UIAccessibilityElement for tapMe
2. Compute frame data on device's coordinates. Don't forget that frame's origin is the top/left corner for UIKit
3. Set data for UIAccessibilityElement
4. Add this UIAccessibilityElement to list of all accessible elements in scene.

Now tapMe is accessible from UI testing perspective.

References

• Session 406, UI Testing in Xcode, WWDC 2015

• eyes off eyes on — Voiceover accessibility in SpriteKit

• How do I support VoiceOver in a SpriteKit game? | Apple Developer Forums

• swift - Accessibility (Voice Over) with Sprite Kit - Stack Overflow

iOS: Is there a way to replace the default accessibility / voice over double tap action on an element with a custom one?

Absolutely! Override bool accessibilityActivate() to implement a custom default action.

iOS VoiceOver - Cannot capture touches in ARKit

If you want to get any feedback of user touches, you must let VoiceOver know that your view can interpret touch events directly by setting its accessibilityTraits to UIAccessibilityTraitAllowsDirectInteraction.

You should get gestures notifications for this particular view telling your app is in charge of touch interception, not Voiceover.

If you want to insert accessible child views inside your screen view, I suggest you disable its accessibility flag because if a parent view is accessible, its children aren't.

---

Related Topics