How to access raw GPU textures/shaders?


#1

FYI, this will be a pretty low-level question about electron’s internals.

I would like to be able to use a custom graphics shader for rendering the output from multiple <webview>s. So, I think that I need access to the OpenGL context from the GPU process, and I am hoping the context will have textures (likely tiles) from all the renderer/helper processes being managed by electron. I will write the needed C++ and OpenGL necessary to stitch these textures together (i.e. web content from multiple different webviews can be composited together into a single view). In theory, if I found the default Chromium GLSL shaders, I could override it at that level – I won’t be adding vertices or geometry or new textures, just re-using what is already rendered.

I am prepared and able to fork electron… but I just can’t figure out where to put my patch in the electron architecture (which IIUC goes from higher-level to lower-level as follows):

  • electron
  • brightray
  • libchromiumcontent

From my understanding of the (modern) Chromium model, there is a single GPU process which is receiving textures from other processes (possibly zero-copy, and/or using a “mailbox” pattern where tiles get rasterized and copied to the GPU process). I can also see that there is only a single GL context created by chromium-based apps, but I can’t seem to find where this is initialized (via tracing). The multi-process model makes it hard to follow the OpenGL calls, and it seems that disabling the GPU process makes it much easier to debug, but then it uses a totally different code path (I suppose from a time before the GPU process existed?)

I have made two (unsatisfactory) prototypes which seemed more complicated than necessary:

  • A CEF3-based implementation where I rendered to PBOs (great, but it definitely introduced some latency, and there’s a long way to go to turn CEF into something featureful like Electron)
  • Patching electron’s offscreen renderer to blit the pixels to a manually created OS window with a new OpenGL context, and never actually creating a main electron window (didn’t work with retina + other code smells).

I know most people with similar designs are using WebGL, but for my purposes I would prefer to fork this into a custom electron build.

I suppose this question is targeted to the electron devs, or someone who might have an idea of where to start, or some idea of where definitely not to look (e.g. “not in brightray”).


#2

This is interesting to me as well, but unfortunately I don’t know the answer. @groundwater or @zcbenz?