Macos Accessibility API Not Working After Change in Code and New Build (Xcode/Swift Simulating Keyboard Input)

OS X: How do I leverage Apple's Assitive Tools API to watch keyboard input?

I just came across this blog post from January 2008, which mentions it would be easy to write a key logger using the Accessibility API.

That functionality should be half of the basic functionality you need (the other half would be to inject the expanded text into the current application).

Maybe you could contact the blog author for further details.

Swift macOS: How to paste into another application

At its simplest you need to do two things:

1. Subscribe to notifications of the current application changing so you know what application you should send events to.

2. Simulate a Cmd+V keypress using System Events.

However, the bad news is, as you say you need to do this in an App Store submitted app, your app needs to be inside the sandbox and requires the temporary entitlement com.apple.security.temporary-exception.apple-events to send events to System Events directly quoting the Apple documentation:

requesting the necessary apple-events temporary exception entitlements for the Finder and System Events will likely result in rejection during the app review process, because granting access to these processes gives your app free rein over much of the operating system. For system-level tasks, use other methods, as discussed above.

There is also a way to do the step #2 using assistive technologies, but that too will not work inside the sandbox. Fundamentally, what you are trying to do (to activate an external arbitrary app, and to control it) is pretty much bang on what the sandbox is designed to prevent you from doing.

Fortunately, since macOS 10.8 there is now NSUserAppleScriptTask, which is executed outside of the sandbox.

There is a twist to NSUserAppleScriptTask: scripts executed with NSUserAppleScriptTask need to be placed inside the application's script directory, and your application cannot write to it, only read from it.

This objc.io article shows an example of how you can request for security scoped writing access to the scripting directory to be able to write your script to it; Annoyingly to yourself and your user you need to bring up an open dialog on the first time, therefore, and you need to store the security scoped bookmark so you don't need to repeat the exercise, but that's the best you'll be able to do inside the sandbox.

Your only other route beside jumping the NSUserAppleScriptTask hoops to my knowledge is to convince Apple that it's a really good idea to accept your app with the temporary entitlement that allows it to script System Events (I would not hold my breath).

Simulate Media-Key press in MacOS

The suggested duplicate question emulate media key press on Mac identified by https://stackoverflow.com/users/4244136/willeke in comments above includes a python solution to my question. I translated it to Swift and posted the working code there. Thanks!

---

Related Topics