HDR Support for GNOME

HDR it is about enabling monitors to display brigher highlights and darker colors as well. In other words, it is about extending the range of luminance. HDR is increasingly common in TVs, and is starting to appear in monitors as well. We want to be prepared for it.

For some background reading, and further references, see this xdc presentation from last year.

Terminology

HDR stands for stands for "High Dynamic Range".

scRGB is a variant of the sRGB color space we are using now, which allows components to go beyond the [0,1] range.

FP16 stands for a buffer format where every component is encoded as a 16bit floating point number, thus needing 64bit per pixel.

The Approach

The approach that is recommended in the presentation, and that we are like going to follow, is to standardize on using the scRGB color space and and FP16 buffer format for HDR content, and enable such content to be passed all the way from applications to the monitor without loss of information.

For best results, there needs to be an API for applications to get information about the color gamut of the monitor, so they can do application-specific compression of their HDR content's range to fit the monitor.

Affected Components

• Wayland
• GNOME shell / mutter
• GTK+
• GEGL and Babl
• Applications that potentially handle HDR content:
  • gnome-photos
  • gimp
  • eog

In Detail

The Wayland protocol needs to be extended to define a way to attach FP16 buffers to surfaces. And there should be an API to obtain information about the characteristics of the monitor wrt to color.

The compositor needs to be able to accept FP16 buffers, and combine them with traditional 32bit RGBA buffers into a FP16 framebuffer. A possible complication here is the shell's own chrome. Cogls ability to use shaders will probably be helpful for converting this to FP16.

The compositor also needs to implement the Wayland monitor information API when it is defined.

GTK+ needs to be able to attach FP16 buffers to wayland surfaces. This will probably be much easier in GTK+4 where all rendering is going through Vulkan or GL, and we may keep this as a GTK+4-only feature for that reason. GTK+ also needs to offer a way for applications to render HDR content in a suitable way.

GEGL and Babl are the image and pixel manipulation libraries used by gimp and gnome-photos. They are already prepared for FP16 content and should only need minor adjustment.

The listed applications should then use that capability to preserve HDR content, eg from raw images. I suggest to make one application the testcase for this, say gnome-photos. Eventually, applications could use the monitor information API to do custom color gamut compression, but that is starting to overlap with color management.

Other things to check for: How well does mesa support FP16 buffers ? What about other drivers ?

People

People who are likely going to be involved with this include Jonas Adahl (Wayland), Rui Matos (GNOME Shell), Benjamin Otte (GTK+), Debarshi Ray (GNOME Photos). But everybody else is more than welcome to chime in, comment and help out.