Electron app with multiple Gstreamer videos


#1

I have been working on a live video production app in Python that takes multiple video sources, allows the user to overlay graphics, and generates a composite video output. It is heavily based on existing systems and the UI will be graphically intensive. Since I need many custom widgets, I had started off developing with Kivy since I was already in Python, and it looked tailored at custom UIs. However, the more I use it, the more jarring it becomes as there is no straightforward to access native UI styles for more common elements like file browsers and select drop-downs.

I have some experience building Ember desktop apps using Electron, and they seem like a good fit for composing fairly skeuomorphic controls. Additionally, the MVC architecture of Ember has some advantages over Kivy’s properties (change-event-binding) system to reduce boilerplate code.

The sticking point is that I need some way to render videos from a GStreamer pipeline to the electron app. There could be up to a dozen videos at a time in various matrix arrangements. Is there any efficient way to do this in an electron app? I found some tools like Webchimera, but they seem more like they are targeted at receiving and displaying an RTSP stream, and I’m not sure if there would be significant overhead from all of the streams?


#2

So far, the closest idea I’ve had is to use the BrowserWindow’s getNativeWIndowHandle() method along with the technique here on line 46. Unfortunately, the window handle returned is an 8 byte buffer. I’ve tried various combinations of passing that as 8 bytes, 4 bytes, little-endian, big-endian, etc. without getting the sink to draw onto the electron window. Is this something that should be possible, or is there some part of the rendering process that this would violate?