How to Catch Accessibility Focus Changed

How to catch accessibility focus changed?

I searched and tried accessibilityElementDidBecomeFocused but didn't trigger after cursor changed.

If you want to use the UIAccessibilityFocus informal protocol methods, you MUST override them in your object directly and not in the view controller (see this answer). Create a subclass of UIButton for instance:

import UIKit

class FlashButton: UIButton {}

extension FlashButton {

    override open func accessibilityElementDidBecomeFocused() {
    // Actions to be done when the element did become focused.
    }
}

If I change the cursor and come back to the flash button, it should say change the flash status again.

Whatever the number of times you select the flash button, the desired status should be always read out first before saying to change its value.

An example is provided in the code hereafter:

class FlashButton: UIButton {

    private var intStatus: Bool = false
    private var a11yLabelStatus: String = "off"

    var status: Bool {
        get { return intStatus }
        set(newValue) { intStatus = newValue }
    }
}

extension FlashButton {

    override open func accessibilityElementDidBecomeFocused() {
    // Actions to be done when the element did become focused.
    }

    override open func accessibilityElementDidLoseFocus() {
    // Actions to be done when the element did lose focus.
    }

    //This native a11y function replaces your defined IBAction to change and read out the flash status.
    override func accessibilityActivate() -> Bool {

        intStatus = (intStatus == true) ? false : true
        a11yLabelStatus = (intStatus == true) ? "on" : "off"

        UIAccessibility.post(notification: .announcement,
                             argument: "flash status is " + a11yLabelStatus)
        return true
    }

    override var accessibilityLabel: String? {
        get { return "the flash status is " + a11yLabelStatus }
        set { }
    }

    override var accessibilityHint: String? {
        get { return "double tap to change the value." }
        set { }
    }
}

How can I achieve this?

Either you try the UIAccessibilityFocus informal protocol to catch the focus change or you just use the code snippet above in order to change the status of your accessibility element: you can combine both, it's up to you to adapt these concepts in your coding environment. ;o)

If it isn't enough, take a look at this site dedicated to a11y developers where code snippets and illustrations are available to find out another solution for your implementation.

Is there a way to detect VoiceOver focus change / user interaction at the `UIApplication` or `UIWindow` level?

So I figured out the solution here and I'm now kicking myself.

UIAccessibility.elementFocusedNotification is the key (https://developer.apple.com/documentation/uikit/uiaccessibility/1620210-elementfocusednotification).

So a solution could look something like:

NotificationCenter.default.addObserver(self,
                                       selector: #selector(self.accessibilityElementFocussed(notification:)),
                                       name: UIAccessibility.elementFocusedNotification,
                                       object: nil)

@objc private func accessibilityElementFocussed(notification: NSNotification) {
    // Reset your idle timer here.
}

Hopefully this helps someone out - I've seen a lot of solutions for idle timers online that don't take into account VoiceOver users.

iOS Accessibility - is there a way to tell when VoiceOver has changed focus?

You can use the UIAccessibilityFocus protocol to detect changes in focus by accessibility clients (including VoiceOver). Note that UIAccessibilityFocus is an informal protocol that each accessibility element must implement independently.

That said, for your use case, Aaron is right to suggest returning a different accessibilityLabel under each condition.

ADB Accessibility Focus Change

My answer here is going to be as succinct as possible. My full code is available on GitHub.

As far as I am aware, a developer cannot perform an accessibility action via ADB, they would have to create an Accessibility service in order to act on behalf of an Accessibility user and create a Broadcast Receiver so that it can take input via the ADB. This is two thirds of the answer, requiring one more component as developers cannot activate accessibility services via the ADB! This has to be done manually each time accessibility is toggled or through accessibility shortcuts - of which there can be only one. EDIT I figured this bit out as well :)

Accessibility Service

The developer documentation provides a mechanism for an Accessibility Service:

An accessibility service is an application that provides user interface enhancements to assist users with disabilities, or who may temporarily be unable to fully interact with a device. For example, users who are driving, taking care of a young child or attending a very loud party might need additional or alternative interface feedback.

I followed the Google Codelab to construct a service that could take actions on the part of a user. Here is a snippet from the service, for swiping left and right (user navigation):

    fun swipeHorizontal(leftToRight: Boolean) {
        val swipePath = Path()
        if (leftToRight) {
            swipePath.moveTo(halfWidth - quarterWidth, halfHeight)
            swipePath.lineTo(halfWidth + quarterWidth, halfHeight)
        } else {
            swipePath.moveTo(halfWidth + quarterWidth, halfHeight)
            swipePath.lineTo(halfWidth - quarterWidth, halfHeight)
        }
        val gestureBuilder = GestureDescription.Builder()
        gestureBuilder.addStroke(StrokeDescription(swipePath, 0, 500))
        dispatchGesture(gestureBuilder.build(), GestureResultCallback(baseContext), null)
    }

Broadcast receiver.

It's important to note that the Receiver is registered when the service is enabled:

    override fun onServiceConnected() {
        IntentFilter().apply {
            addAction(ACCESSIBILITY_CONTROL_BROADCAST_ACTION)

            priority = 100

            registerReceiver(accessibilityActionReceiver, this)           
        }
   // REMEMBER TO DEREGISTER!

Then the receiver can respond to intents and invoke the service:

    override fun onReceive(context: Context?, intent: Intent?) {
        require(intent != null) { "Intent is required" }
        val serviceReference = //get a reference to the service somehow

        intent.getStringExtra(ACCESSIBILITY_ACTION)?.let {
            serviceReference.apply {
                when (it) {
                    ACTION_NEXT -> swipeHorizontal(true)
                    ACTION_PREV -> swipeHorizontal(false)
                    //...

example usage on the adb:

adb shell am broadcast -a com.balsdon.talkback.accessibility -e ACTION "ACTION_PREV"

EDIT

Starting the service via adb:

Thanks to this comment

TALKBACK="com.google.android.marvin.talkback/com.google.android.marvin.talkback.TalkBackService"
ALLYDEV="com.balsdon.accessibilityBroadcastService/.AccessibilityBroadcastService"
adb shell settings put secure enabled_accessibility_services $TALKBACK:$ALLYDEV

Accessibility shortcut

** EDIT: NO LONGER NEEDED BUT I'LL LEAVE IT HERE ANYWAY **

Possibly the most annoying thing is that every time accessibility is toggled, the service is turned off. So I added my service as a shortcut with the VOLUME_UP and VOLUME_DOWN keys pressed. Thanks to this question.

  INPUT_DEVICE="/dev/input/event1" # VOLUME KEYS EVENT FILE
  VOLUME_DOWN=114 #0x0072
  VOLUME_UP=115   #0x0073
  BLANK_EVENT="sendevent $INPUT_DEVICE 0 0 0"

  INST_DN="sendevent $INPUT_DEVICE 1 $VOLUME_DOWN 1 && $BLANK_EVENT && sendevent $INPUT_DEVICE 1 $VOLUME_UP 1 && $BLANK_EVENT"
  INST_UP="sendevent $INPUT_DEVICE 1 $VOLUME_DOWN 0 && $BLANK_EVENT && sendevent $INPUT_DEVICE 1 $VOLUME_UP 0 && $BLANK_EVENT"
  adb shell "$INST_DN"
  sleep 3
  adb shell "$INST_UP"

I already have a script for toggling accessibility on/off, so I just tacked this on top of that and I get my service running every time.

EDIT 2

I put an issue on AOSP regarding the fact that a developer needs to write a "service that swipes" as opposed to a "service that navigates". This is problematic as the gestures can be modified and then my system will not behave as expected. Instead I should be able to call the particular action I want to do NAVIGATE TO NEXT ELEMENT or NAVIGATE TO PREVIOUS ELEMENT as opposed to "SWIPE RIGHT" and "SWIPE LEFT" - having read the WCAG guidelines I do not believe that this is violating the principles.

---

Related Topics