Synchronous API?

[dhardy]

Out of curiosity, what made you choose to design AccessKit around an internal tree shadowing the UI, instead of as a synchronous API over an existing widget/text node/... tree?

The synchronous approach has some drawbacks (e.g. I'm not sure how well it would integrate with immediate-mode UIs like egui), but does have some major advantages:

• The ability to request details as required, instead of requiring all possible details (down to the level of glyph positions) ahead-of-time
• Support test-driven-development: users may write todo!() impls and fill these in as required.
• The ability to build a shadow tree (like AccessKit's current API) as an external adapter over the synchronous API. (The reverse is not possible.)

[DataTriny]

What you describe is effectively how OS accessibility APIs work today. We call it pull-based. Whether you are on Windows, macOS or Linux, current assistive technologies only query the information they need, when they need it. The accessibility stack is mostly just a proxy between the assistive technology and your application.

For me the biggest disadvantage of this approach is that it require a lot of inter-process communication, which is still expensive today. It wouldn't make sense for your GPU to ask you the color of each pixel, you prepare your frame and send it at once.

Supporting immediate mode rendering UIs is not really possible with a pull-based approach. Even if they are not the most popular, we must ensure that they can be made accessible.

Ultimately we may need an hybrid approach, like the web. Send what you can upfront and let the client ask details on regions that change too frequently. But I don't see currently how this could fit nicely into the model. To be clear, since no software with huge UIs have ever been built with AccessKit, we still need to prove that our solution is viable.

Your input can help shape the future of AccessKit, don't refrain from pointing out rough edges or even design flaws. Thank you!

[dhardy]

Supporting immediate mode rendering UIs is not really possible with a pull-based approach.

This is not true. It does require building a tree of nodes (like AccessKit currently does), but this can be done by the UI library or an intermediary library. This is my last point above: it is possible to build a shadow-tree based API over a pull-based API, but not vice-versa.

Ultimately we may need an hybrid approach, like the web.

A hybrid approach could also end up being the worst of all options depending on how it works. Synchronous (pull-based) integration requires that the adapter run in the UI thread with some form of access to UI state (unless run over a shadow tree as above). If you have this limitation but also have an internal shadow-tree/cache — that just sounds like extra complexity and redundancy.

Unless you mean an asynchronous pull-based model; this could work (but also adds complexity).

---

FYI I'm aiming to add additional tree inspection support to Kas sufficient for (read-only) pull- or push-based a11y integration. I'll ping you later.