electronShell and screenreader accessibility


#1

Hi,

I am a fully blind computer science student specializing in web development. I first got in contact with the ElectronShell through the Atom editor …which seemed to be completely inaccessible to screenreader users.
Skip ahead a few months to the release of Visual Studio Code. This seemed like a good idea, a lightweight editor with loads of languages supported. I downloaded it …only to find it was completely screenreader inaccessible.
Skip ahead again to a month or so back when I got Kitematic on my screen …which was completely screenreader inaccessible as well.
I did some digging and found the Electron Shell is the common core of all these applications. I have two questions:

  • Is this purely coincidental? Somehow, I doubt it.
  • If the previous question can be answered with a ‘no’, what can we do to fix this glaring issue that excludes literally thousands of users from using these applications?
    I’d like to start a dialogue here in finding the problem and discussing possible solutions to what could be a crippling accessibility issue that, in my opinion, shouldn’t even exist anymore in 2015.

Really looking forward to replies on this.

Regards,
Florian


#2

You’re right, to my knowledge, no real work has been done to make it easy for Electron applications to be screenreader accessible. I think two things need to happen: education and an infrastructure to make accessibility easier. If you can educate us on what it takes to make an application or webpage screenreader accessible, perhaps someone here can take on creating tools to make the process easier for application developers?


#3

Hi,

I’m perfectly willing to help there :slight_smile: he problem is that something is going on that I don’t know the cause for, namely:
Usually, making a web page accessible isn’t very difficult for simple pages. Just make sure you either use standard semantic HTML or write the correct WAI-ARIA markup to render things like snazzy JS clickable divs into properly tagged operable elements.
However, in the applications I named earlier, the web content doesn’t seem to be exposed to the screenreader at all, rendering all this useless.
Let’s take another example, the current rendition of the Spotify application for Windows is also web-based. However, this web content is exposed to screenreaders so they can at least make some sense of it. It’s not ideal, but usable.
This makes me think that not the html and js itself, but the wrapper itself in the ElectronShell is the biggest culprit here. How to fix? I have no idea whatsoever. For that I’d need to know how these wrappers work under the hood, how they communicate with the OS. Can someone give me a hand in figuring that one out? :smile:
Florian


#4

I know one may think, accessibility of web page has greatly advanced, application is made of html, so it must be easy to make accessible.

However applicaiton made of web technology have to work much harder to have a native like performance.

So we have situation like the Atom text editor, where lines of text are grouped into tile to only have dom for the part of the document that is visible. The tiles themself are sorted by … chronological order of the scroling history. There’s text decoration, but in markup is too slow. So there’s computation and clever trick to align XY pixel position. And instead of using top, left positioning, advanced CSS like 3d transform is used in order to make the redraw happens on the video card.

So we don’t have a very nice accessible markup, only kept secret by an electron wall.
We have markup, optimized for fast display. And most coherence exist as spacial relationship on the XY plane.

And please be sure in no way, I am here to discourage you from improving accessibility wich is important.
Simply for me it looks like Electron shell is probably the first step instead of the “Biggest Culprit”


#5

I think that this is the key issue. Electron applications are rarely simple web pages. Perhaps you know of some guides for making Angular, React or other web framework based applications accessible? Is the Slack (for Windows) application accessible? It is also based off of Electron, to my knowledge.


#6

@jean,
This is certainly true to a degree. Redrawing using 3d-transform renders the process inaccessible to screenreaders since no markup is actually being used to specify the textual content of the application. This may be fast but only works if you have the ability to read the screen without issue. People who are blind, partially sighted, colorblind, dyslexic etc. will find all their assistive technologies failing due to bypassing the structural HTML layer.
I have no quick solution ready to fix this, oter than rendering a secondary virtual DOM based on the 3d-transformed CSS content. However, this will mean a serious performance hit. I’d love others to chip in with their ideas about this one.

@lee:
ReactJS, AngularJS, EmberJS, Meteor etc. are fancy frameworks that give you a whole lot of functionality out of the box and let you build apps quickly. These applications can certainly be made accessible; the ng-aria plugin for Anguular and the recently released React-accessibility API built by Facebook are ways of doing this.
However, as I said earlier, it seems right now no web content at all is being communicated back to screenreaders. So I’m wondering to what degree these modifications would actually help as long as that remains the case. Jeancroy’s explanation of rendering everything through CSs 3d-transform would explain this to a certain degree; if the initial drawing of the UI uses this same technology you essentially paint a visual layer on screen without actually using structural markup. If I look at this from an accessibility audit point of view, this would be a breach of several major success criteria of the WCAG 2.0-specification. Again though, I have no idea how one would go about fixing this without significantly hindering performance. You’d almost be forced to render either the visual UI, or an accessible version I think. As I said, if others have better ideas please feel fre to chip in.
Florian


#7

I’m not just trying to puzzle out how to make Atom accessible because you brought this up with regard to Electron. So the question I’m considering is, “How do we make it so that it is as easy as possible to make any Electron-based application accessible?”

The reason why I asked about Angular and its ilk is because they are web-based frameworks to help build apps quickly, similar to Electron … though they focus on the UI aspect whereas Electron is more for the interaction with the OS. Seeing how they solve the problem, if they do, could be instructive on how to make accessibility for Electron-based applications easier. Or perhaps Electron, since it does not focus on the UI as much, could simply point out these resources for others to build more accessible applications.

Once some tools are available or identified for Electron, those can be forward-ported to Atom and work can begin to make Atom more accessible.


#8

A first step could be to make a very accessible web page. Either because simple, or a benchmark with various aria-* features.

Then package that webpage into a electron application.

Then see what screenreader do of that.

If it fail, see what Chrome have in term of accessibility API, wheter those exist in Chromium, and wheter we could implement/activate those in electron

After the simple webpage works, then I guess it’s time to explore JS framework support.


#9

@lee:
I think what you are referring to is for example something like this:


I am not aiming at Atom specifically either, I am aiming at the exactly the same behavior I am noticing in various Electron apps, one being Atom and another being VS Code. This behavior being the following:

  • Menubar menus seem to work reasonably well, these seem to at least try to approximate native controls and have similar accessibility exposure to screenreaders. I am wondering if its regular win32 API calls that do this behind the scenes, but again I don’t know how the packaging of an app works exactly.
  • The actual area where main content of the app is shown (code editor, but also things like preferences windows) appears to be a fully blank web view to screenreaders. Screenreaders usually contain a mode for rendering web content , often in what is a sort of virtual buffer where the DOM order is being used to construct a top-to-bottom version of the web page. CSS is often ignored in this virtual buffer, so the reading order is dependent on the DOM order.
    The virtual buffer in these applications however, is completely blank. Screenreaders usually take the information they require from a browser’s accessibility tree, in which the DOM is also included. It almost seems like these webviews do not contain this tree, which makes me wonder if a 100% accessible application for as far as a browser is concerned would even render in an Electron packaging.

@jeancroy:
Yes, I think that would be good. I am very much wondering where in the packaging toolchain this problem occurs. See my explanation above for what SHOULD happen and what DOES happen for some pointers. For a test page, we could use something like this:

Additional Javascript required here to make this button act like a proper button
This link should not show up for screenreaders

I am a heading level 2 and am breaking WCAG since I have no paent heading 1

...

You can go as crazy as you want with this, maybe add a few labeledby and describedby elements if you want to go all out. Trap focus in a modal, all sorts of stuff can be done. At this point however, I am wondering if a simple heading 1 saying “Hello world” would even render.
Florian


#10

@zersiax

What do you think of the accessibility improvements that were made to Atom in mid 2016? Does your screen reader now work with Atom?