So I made a voice controller for MCOC for the disabled

Blizzard25

The only problem is, Kabam you have to enable Accessibility Services for it to actually work. I have another version running an adb shell that works but that's not realistic. Any chance you want to add this service to your code to allow potentially disabled people to play your game with their voice? You don't even need to add my code to yours, mine will run over top of the app. All you have to do is enable the service to run.

Long story short, this is just a controller that simulates motions on your screen. Ex. saying "swipe left" would simulate a swipe left action on your screen (so in MCOC, dexing/swiping backward) and so on. You can add any number of commands that you want. It may not seem like something that should be added, but accessibility services were designed with this use case in mind, helping the handicapped use things on their devices they otherwise couldn't.

Couple of screenshots to show you there's nothing nefarious in terms of permissions being required and that it is actually an accessibility service.

DNA3000

I am not an Android developer, so I am genuinely curious. Hypothetically speaking how would the MCOC client block someone else from using these features to essentially bot the game? If someone were to make a service that appeared to be an accessibility service but actually used that capability to simply send inputs to the game without human intervention that could be problematic. Is there a way to enable something like this that would prevent it from being used in any way other than the intended voice translation accessibility inputs?

Blizzard25

DNA3000 said:

I am not an Android developer, so I am genuinely curious. Hypothetically speaking how would the MCOC client block someone else from using these features to essentially bot the game? If someone were to make a service that appeared to be an accessibility service but actually used that capability to simply send inputs to the game without human intervention that could be problematic. Is there a way to enable something like this that would prevent it from being used in any way other than the intended voice translation accessibility inputs?

Well for voice inputs that would be pretty rough to bot. You'd have to have a camera watching the screen, then voice outputs that matched what the camera was picking up....not impossible, but not really practical. And you can whitelist applications that are able to use accessibility services so no one else can. So to your point, application security on my side and whitelisting on their side.

Blizzard25

https://developer.android.com/reference/android/accessibilityservice/AccessibilityService

DNA3000

Blizzard25 said:

DNA3000 said:

I am not an Android developer, so I am genuinely curious. Hypothetically speaking how would the MCOC client block someone else from using these features to essentially bot the game? If someone were to make a service that appeared to be an accessibility service but actually used that capability to simply send inputs to the game without human intervention that could be problematic. Is there a way to enable something like this that would prevent it from being used in any way other than the intended voice translation accessibility inputs?

Well for voice inputs that would be pretty rough to bot. You'd have to have a camera watching the screen, then voice outputs that matched what the camera was picking up....not impossible, but not really practical. And you can whitelist applications that are able to use accessibility services so no one else can. So to your point, application security on my side and whitelisting on their side.

That would seem to imply that they would need to explicitly white list your service directly; there's no way to, say, categorically white list in such a way to limit to services of similar type. That could get sticky: asking the developers to enable a generic Android API would be one thing, but opening the door to having to individually approve and police user-contributed accessibility services would be a lot more problematic.

If they could enable this in such a way that they could somehow enable all accessibility services that "look like" actual accessibility services, that *might* be something they'd consider. Is there such a way to apply some sort of more generic safety limits around this?

Blizzard25

DNA3000 said:

Blizzard25 said:

DNA3000 said:

I am not an Android developer, so I am genuinely curious. Hypothetically speaking how would the MCOC client block someone else from using these features to essentially bot the game? If someone were to make a service that appeared to be an accessibility service but actually used that capability to simply send inputs to the game without human intervention that could be problematic. Is there a way to enable something like this that would prevent it from being used in any way other than the intended voice translation accessibility inputs?

Well for voice inputs that would be pretty rough to bot. You'd have to have a camera watching the screen, then voice outputs that matched what the camera was picking up....not impossible, but not really practical. And you can whitelist applications that are able to use accessibility services so no one else can. So to your point, application security on my side and whitelisting on their side.

That would seem to imply that they would need to explicitly white list your service directly; there's no way to, say, categorically white list in such a way to limit to services of similar type. That could get sticky: asking the developers to enable a generic Android API would be one thing, but opening the door to having to individually approve and police user-contributed accessibility services would be a lot more problematic.

If they could enable this in such a way that they could somehow enable all accessibility services that "look like" actual accessibility services, that *might* be something they'd consider. Is there such a way to apply some sort of more generic safety limits around this?

Well generic access is exactly what you want to stay away from for exactly the reasons you described. Industry best practice (for whatever that's worth) is white listing accessibility services

DNA3000

Blizzard25 said:

DNA3000 said:

Blizzard25 said:

DNA3000 said:

I am not an Android developer, so I am genuinely curious. Hypothetically speaking how would the MCOC client block someone else from using these features to essentially bot the game? If someone were to make a service that appeared to be an accessibility service but actually used that capability to simply send inputs to the game without human intervention that could be problematic. Is there a way to enable something like this that would prevent it from being used in any way other than the intended voice translation accessibility inputs?

Well for voice inputs that would be pretty rough to bot. You'd have to have a camera watching the screen, then voice outputs that matched what the camera was picking up....not impossible, but not really practical. And you can whitelist applications that are able to use accessibility services so no one else can. So to your point, application security on my side and whitelisting on their side.

That would seem to imply that they would need to explicitly white list your service directly; there's no way to, say, categorically white list in such a way to limit to services of similar type. That could get sticky: asking the developers to enable a generic Android API would be one thing, but opening the door to having to individually approve and police user-contributed accessibility services would be a lot more problematic.

If they could enable this in such a way that they could somehow enable all accessibility services that "look like" actual accessibility services, that *might* be something they'd consider. Is there such a way to apply some sort of more generic safety limits around this?

Well generic access is exactly what you want to stay away from for exactly the reasons you described. Industry best practice (for whatever that's worth) is white listing accessibility services

That's what I figured. The security model, barring some weirdness I'm not aware of, is mostly designed to protect the user of the device, less so the applications running on the device. I'll find out what I can, but I'm not sure this is something Kabam would want to tackle long-term.