A retro-arcade cabinet is a perfect example of a low-powered machine that can be ran using IoT equipment such as a Raspberry Pi. The requirements for retro-games are particular given that all the game run via an emulation layer that typically only leverages the CPU which makes CPU cycles very precious for retro-gaming.

The software itself for the arcade cabinet has been Lakka, which is a minimalist Linux distribution with just the necessary installed to be able to run RetroArch and targeted to a specific machine such that all extra Linux support for hardware is removed. More than often, if the built-in streaming option for Lakka is used, then the games tend to lag and extra CPU power is required that makes the emulation considerably slow.

It seems clear that having the machine both run a game and also capture frames from the game to stream them remotely will be difficult if the same low-power profile is to be maintained.

The only option that seems to be both a fallback option and incidentally also the best option is to somehow intercept the video signal itself via some additional hardware device that will be able to capture the signal and then offer it up to a remote system somehow. The idea here would be that the machine running the game will not expend any CPU cycles to grab frames and stream to a remote server but that an additional device will intercept the signal and use its own processing power to process the signal and stream it remotely.

Incidentally, not all HDMI streamers are expensive and the illustrated device comes at about USD20 and is able to pass through a HDMI signal whilst also capturing the signal and making it available via USB. Internally, the device works like a video camera and has support for motion JPEG images as well as raw images, both of which can be streamed for further processing.

There are two ways to go about setting up streaming using an HDMI video capture device like the one illustrated given that the signal is offered via the USB port. One can loop-back the video video signal right into the same machine that is also running the game via USB or the USB cable can be pulled to a separate machine that will just handle the video streaming.

In principle, looping back the signal into the emulation machine is not a problem because the video capture device actually does the processing of grabbing the frames and making them available, such that the emulation machine would not have to expend any CPU cycles for streaming but rather additionally act as a streaming server offering the video. The second option is a bit more wasteful in terms of needing an additional machine and note that since the video capture device will be processing the video anyway, the video processing machine does not need any special hardware or support for video formats.

The software that runs on the arcade cabinet is Lakka, which is a reduced Linux distribution that contains just the bare minimal necessary to run RetroArch with some support for hardware. Using Lakka is elegant because the machine behaves like an arcade machine by booting directly into the menu seamlessly without having to first boot a Linux desktop and only then start RetroArch. The downside is that Lakka is mostly read-only and lacks all the necessary tools to be able to set something like this project up. In fact, Lakka curates the Linux modules such that plugging the HDMI capture device into Lakka via USB will have no effect and it will not be recognized at all.

With that said, Lakka needs recompiling in order to include the uvcvideo driver. Compiling Lakka involves pulling the github source with:

git clone https://github.com/libretro/Lakka-LibreELEC.git

after which it is a matter of entering the directory and issuing make in order to build the image.

This part depends on the targeted device, with different build options for each device. In this case, we used a Raspberry Pi 4, so the following command is the one to pick:

DISTRO=Lakka PROJECT=RPi DEVICE=RPi4 ARCH=aarch64 make image 

Here are some notes based on some experience with building Lakka its source:

• the build takes an extremely long amount of time to compile and you can expect the process to take a full day (12 hours),
• the build needs more than of RAM, perhaps might work better,
• the build system pulls in software from various URLs while it is building and those servers sometimes fail - in this case, the build might be okay and just re-issuing the make command to build again should be alright,
• when the build starts, the build system uses distribution-specific commands to install packages such as rsync or wget and it can happen that the build system misses a requirement such that the Lakka build will fail; the logs can be consulted and the necessary requirements installed manually in order to continue

It seems like the kernel configuration for a specific device is maintained inside projects/RPi/devices/DEVICE_NAME/linux where DEVICE_NAME is a device name like RPi4. For example, in this case the file containing the kernel configuration was projects/RPi/devices/RPi4/linux/linux.aarch64.conf.

The kernel can be configured manually with a text editor at this point by editing the configuration file and ensuring that the required hardware is compiled either as a module or built-in. The key is to set an option within the kernel configuration file to either m or y which correspond to module or to be compiled into the kernel. The choice between a modular or monolithic kernel is up to the choice of the user and it should perhaps only matter if the device that Lakka will be compiled for is extremely low-powered. Similarly, the amount of modules compiled only matter in terms of space with most devices being able to accommodate any number of modules.

The way to compile the module manually is to start from the needed module, in this case, uvcvideo which corresponds to the kernel configuration option CONFIG_USB_VIDEO_CLASS and then check what the configuration option CONFIG_USB_VIDEO_CLASS depends on in order to enable them as well. In this case a website called kernelconfig was used in order to enable CONFIG_USB_VIDEO_CLASS and everything that CONFIG_USB_VIDEO_CLASS depends on by following the depends section on the right. Usually, if this were a proper kernel compilation using menuconfig, then the configuration script would have handled all the dependencies but given this minimal environment, everything has to be done manually.

Lakka developers ship Lakka in its most minimalist form such that an override exists that will make it such that some modules will not compile on any platform. The override file can be found at distributions/Lakka/kernel_options_overrides. The overrides at the current time will override the necessary uvcvideo module and set it to not compile by setting CONFIG_USB_VIDEO_CLASS to n. It is possible to go through the override file and remove modules that are set to not compile relevant to uvcvideo by following a dependency tree but it is way easier to just remove all overrides by making a backup of the distributions/Lakka/kernel_options_overrides file and then creating an empty file instead at distributions/Lakka/kernel_options_overrides. Again, the idea here is that Lakka offers minimalist builds but kernel modules should not be a show stopper because they only occupy space such that compiling them all as-is should not be a problem.

Either way, after building Lakka there should be an .img.gz file created within the target/ directory from the Lakka root directory. The file can be transferred to the Lakka machine under /storage/.update and then the image will be flashed after the device reboots. After flashing the new image, the DHMI device should be plugged in and the following command can be issued on the command-line on the device (via SSH) in order to dermine whether uvcvideo is available:

lsmod | grep uvc

which should print out something along the lines of:

uvcvideo              155648  1
videobuf2_vmalloc      20480  1 uvcvideo
uvc                    12288  1 uvcvideo
videobuf2_v4l2         36864  1 uvcvideo
videodev              368640  3 videobuf2_v4l2,uvcvideo
videobuf2_common       81920  4 videobuf2_vmalloc,videobuf2_v4l2,uvcvideo,videobuf2_memops
mc                     94208  5 videodev,snd_usb_audio,videobuf2_v4l2,uvcvideo,videobuf2_common
usbcore               409600  8 xhci_hcd,snd_usb_audio,usbhid,snd_usbmidi_lib,btmtk,uvcvideo,btusb,xhci_pci
usb_common             16384  3 xhci_hcd,usbcore,uvcvideo

It is mostly important for uvcvideo to show up in the list and also hopefully a new video device should exist at /dev/video0 that corresponds to the HDMI USB port.

Regardless of what setup option was used, either feeding back into the same machine that runs the games or whether a separate machine was added to just handle the stream, the tools and principles are mostly the same such that it is possible to start streaming right away. Perhaps the first command would be an ffmpeg command meant to inspect the video device in order to determine that video formats and resolutions it supports:

ffmpeg -f v4l2 -list_formats all -i /dev/video0

For the illustrated device, the following was reported:

[video4linux2,v4l2 @ 0x55bcca7748c0] Raw       :     yuyv422 :           YUYV 4:2:2 : 1920x1080 2560x1440 1360x768 1280x1024 1280x960 1280x720 1024x768 800x600 720x576 720x480 640x480
[video4linux2,v4l2 @ 0x55bcca7748c0] Compressed:       mjpeg :          Motion-JPEG : 1920x1080 2560x1440 1360x768 1280x1024 1280x960 1280x720 1024x768 800x600 720x576 720x480 640x480

which lists the resolutions that the HDMI capture device can capture at along with the pixel format, either YUYV422 or MJPEG. The difference between YUYV422 and MJPEG is just size with the MJPEG stream being way "thinner" than raw YUYV422 in terms of transferred bytes.

ffmepg can also be used to stream right away without any necessary software and if the HDMI capture device is looped into the machine running Lakka, then it might be the only option without having to poke the Lakka build-system again in order to include additional software. The following command:

ffmpeg -f v4l2 -video_size 1280x720 -input_format mjpeg -i /dev/video0 -f flv -listen 1 rtmp://0.0.0.0:8000/

will:

• pull images off the HDMI capture device with a resolution of 1280x720,
• an input format of mjpeg ,
• from the video device /dev/video0,
• will package the stream in an FLV container, and,
• will listen for connections on any address 0.0.0.0 and port 8000

In case a dedicated machine was added, using ffmpeg is not necessary and there are "better" solution as an all-in-one that can capture and stream video using a dedicated server. For example, using Docker, the mpromonet/v4l2camera:latest image can be used to capture and stream by issuing:

/usr/bin/docker run -it --device=/dev/video0 -p 8554:8554 mpromonet/v4l2camera:latest

and a separate SystemD service file can be written just for that purpose:

[Unit]
Description=v4l2camera
After=docker.service
StartLimitIntervalSec=0

[Service]
Restart=always
RestartSec=5s
ExecStartPre=/bin/sh -c '/usr/bin/docker network create entertainment || true'
ExecStartPre=/bin/bash -c 'while [[ ! `docker pull mpromonet/v4l2camera:latest` ]]; do sleep 1; done'
ExecStart=/usr/bin/docker run --name=v4l2camera \
  --rm \
  --hostname v4l2camera \
  --net=entertainment \
  --user 0:0 \
  --device=/dev/video0 \
  -p 8554:8554 \
  mpromonet/v4l2camera:latest
ExecStop=/usr/bin/docker stop v4l2camera
ExecStop=/usr/bin/docker rm -f v4l2camera
TimeoutSec=3600

[Install]
WantedBy=multi-user.target

For network play when games might end up being competitive, it does not make much sense to have the same machine capture the stream. Even if the machine is very powerful, the additional concern might induce bugs that might disrupt gameplay or even be an additional task that the player will have to manage.

On the other hand, intercepting the signal externally to the system that is used to play the game on is a very clean solution and often also a cheap solution because the capture device does not really have to be too powerful. Even though a low-end capture device has been used, these solutions scale up to the tune of hardware acceleration via codec chips on the motherboards of the capture device. However, digital signals like HDMI do not really need all that much processing and interpreting the signals is pretty cheap.

It's interesting when you think about it, having RetroArch both play a game and stream, makes RetroArch grab the frames during the rendering process, which even directly imples an additional dedicated thread that will need to process the stream and also additionally stream it to a streaming endpoint like YouTube or Twitch. Those are a lot of expensive operations compared to capturing the signal externally and then streaming it directly. In reality, the difference will be that someone watching the stream that uses the HDMI capture device will get to see the menu and in-between games as well because Lakka only streams upon game start.

Even though the usage case seems particular it is a solution that should be kept in mind when lower-power devices are involved that must also offer "efficient streaming" that is usually very expensive in computational terms.