Streaming lossless VNC to a web browser with Quite OK image format

Lossless encoding

Kasm Workspaces allows an optional true Lossless mode to be enabled using an installation flag. Before enabling and using this mode, it is important to understand the bandwidth/compute requirements especially in shared environments.

Lossless encoding can and will consume all available bandwidth available to a client and is designed to be run on local networks. This means under even moderate motion and activity at 60 frames per second any resolution 1920x1080 and up will likely consume an entire gigabit connection. Bandwidth consumed depends heavily on the client’s capabilities, however, most quad core x86_64 CPUs produced after 2014 should be capable of decoding at a sustained 1000mbps.

In this document we will be reviewing how to enable and use this mode along with brief technical notes on how this was implemented to function in any modern web browser.

Enabling Lossless

Important

Lossless encoding is disabled by default and enabling it has consequences that can break certain configurations or integrations between Kasm and other systems. Lossless requires Cross-origin isolation, which blocks requests on the page to all external sites. Enabling Cross-origin isolation may break certain configurations such as branding or referencing external images for Workspace thumbnails. If you have integrated Kasm Workspaces with other systems, enabling Cross-origin isolation will need to be thoroughly tested with your environment.

During installation or upgrade of Kasm Workspaces 1.12.0 you can pass the flag --enable-lossless:

cd /tmp
curl -O https://kasm-static-content.s3.amazonaws.com/kasm_release_1.12.0.d4fd8a.tar.gz
tar -xf kasm_release*.tar.gz
sudo bash kasm_release/install.sh --enable-lossless

cd /tmp
curl -O https://kasm-static-content.s3.amazonaws.com/kasm_release_1.12.0.d4fd8a.tar.gz
tar -xf kasm_release*.tar.gz
sudo bash kasm_release/upgrade.sh --enable-lossless

During installation this will set the apropriate headers needed by Lossless mode and the software will detect the presence of these headers unlocking quality level 5 Lossless for all clients connecting to this Kasm Workspaces deployment.

Using Lossless

Enabling lossless is pretty straightforward once enabled by the installation. After connecting to a Workspaces session, navigate to Streaming Quality in the sidebar and move the quality slider all the way to the right (Quality level 5 Lossless). If this option is unavailable re-check your installation/upgrade settings and ensure --enable-lossless was passed.

With lossless enabled you should be able to play videos/games at near 60fps with very little latency.

Note

While Lossless mode will function on any modern web browser the best experience will be had on Chromium based browsers.

Technical Background

Lossless encoding uses The Quite OK Image Format which is a very fast to encode and decode lossless image format. While it is a great image format for delivering truly lossless image compression it has no native support in any modern web browser. Without native image decoding support it is necessary to pass the VNC rect images to worker threads to be decoded into raw image data that can be painted to the canvas presented to the end user.

The server side headers are needed to enable SharedArrayBuffer support, specifically known as Cross-origin isolation. This enables a high speed in memory methodology to pass the huge amounts data back and forth from the workers to the main browser javascript thread for rendering. These headers specifically in NGINX format:

add_header 'Cross-Origin-Embedder-Policy' 'require-corp';
add_header 'Cross-Origin-Opener-Policy' 'same-origin';
add_header 'Cross-Origin-Resource-Policy' 'same-site';

Because this is all happening manually leveraging WebAssembly and pure CPU decoding the overhead is significant in comparison with decoding native browser formats like JPEG and WebP. For comparative image delivery it is at least double CPU usage and 10x the bandwidth of near lossless JPEG as native image formats can leverage GPU and worker threads baked into the browser for rendering DOM elements. The main advantages of using this form of lossless compression are we are no longer bound by a single thread and the image is truly lossless. When leveraging workers this means we can achieve a higher framerate versus native browser decoding at the expense of bandwidth and CPU overhead.

As with anything lossless it is impossible to convey the quality and experience in a video or technical document, it is something that needs to be experienced.

from Hacker News https://ift.tt/oFO4pUC