How Serialised GVariants Work

This is a short overview of the inner workings of serialised GVariant instances. It uses a short example.

Notation

Before starting, here is a quick overview of some of the visual notation used in the example.

A dotted arrow represents a strong reference held between two objects (ie: the referenced object may not disappear so long as the reference is held).

A forked arrow represents a "sequence pointer". This is a pointer not just to a single byte (as most pointers are) but to a sequence of bytes. It has a start and an end.

A red arrow is a normal C pointer.

A long green unlabelled rectangle represents a sequence of bytes somewhere in memory. Sequence pointers and C pointers can point here.

The smaller labelled boxes represent conceptual objects. The red ones are normal GObjects and the light blue ones are GVariant instances. Objects may be the source and the destination of references.

Example

In the example, we are going to open a GMappedFile that contains an array of strings, create a GVariant representing this array, then extract and inspect one of the strings from the middle of the array.

Step 1

First, we start by performing the normal operation of opening a GMappedFile. This gives us a GMappedFile object that refers to a file-backed virtual memory region (represented by the long green rectangle).

  MappedFile *mapped;

  mapped = g_mapped_file_new ("filename", FALSE, NULL);

We could just as easily use any other memory region as the source of our serialised data (for example, a buffer just read from a network socket or a memory region shared with another process).

Step 2

After presumably inspecting the header of the file we have determined that the file contains an array of strings. We want to create a GVariant instance to refer to this array of strings.

A GVariant instance itself is a very small (~20 bytes) proxy object that is allocated with GSlice. It merely refers to the array and does not actually contain it. As such, we need to ensure that the GMappedFile instance continues to exist for as long as our GVariant exists.

We create the GVariant instance with a call that looks something like the following:

  GVariant *array;

  array = g_variant_load_nocopy ("as",       /* array of strings */
                               pointer,    /* inside the mapped file */
                               length,     /* and not past the end */
                               (GDestroyNotify) g_mapped_file_free,
                               mapped);

array now holds a reference to the mapped file. When array stops existing, the mapped file will be closed. If we didn't want this, we could just give NULL for the destroy notify (or add refcounting to GMappedFile -- see bug #458796).

At this point, GVariant has not yet read a single byte from the mapped area.

Step 3

We query the number of strings that are held in the array.

  int count;

  count = g_variant_array_count (array);

This involves reading a single 32bit integer from the start of the byte sequence representing the array (see ../Serialisation). After discovering the number of items, we decide that we want to read item #3 from our array of strings.

  GVariant *string;

  string = g_variant_array_get (array, 3);

With the read of two more 32bit integers we discover the start and the end of the byte sequence corresponding to string #3 in the array.

We create a new proxy object and point it to this sequence.

Of course, so long as this new string object exists, the file contents need to continue to exist. The string object owns a reference on its parent.

Step 4

We can now obtain a C pointer to the start of the string (which is nil-terminated).

Because we know the length of the byte sequence corresponding to the string, we can also determine the length of the string without scanning it. This information is returned to the user. The API looks like:

   int length;
   char *ptr;

   ptr = g_variant_string_get (strval, &length);

but you can give NULL for the length return field.

When you obtain a pointer into a GVariant, the pointer remains valid so long as the GVariant continues to exist.

Step 5

There is a trade-off that was implicitly made to allow the zero copying that has so far been performed. What if we no longer care about the contents of the array as a whole, but want to keep this one string around? If we drop the reference we own on the array, it is still kept around by the reference owned on it by the string.

An effect of this is that the entire mapped file is kept in memory for as long as we have any value that existed anywhere inside of it. This might be a fantastic waste.

As a compromise, GVariant allows values to be "severed". This allows a value to drop any references it owns on its parent and to be recreated in its own right. Severing a GVariant will typically involve copying.

  string = g_variant_sever (string);

The API looks a little bit strange in order to stress the fact that, conceptually, the value is being destroyed and recreated (even if this isn't actually true in terms of pointer value).

Step 6

At this point, we can drop our own reference to array and it will be destroyed.

  g_variant_unref (array);

When the array is destroyed, it drops its reference on the GMappedFile (really, it calls the destroy notify). This results in the GMappedFile being closed and the memory region that it represented being unmapped.

There is a problem though. Once the mapped file is closed, the pointer we obtained in step 4 is rendered invalid.

In reality, the pointer was rendered invalid by our actions in step 5. Pointers are only valid so long as the GVariant whence they came continues to exist. In step 5 we "recreated" that value, thus invalidating the pointer.

Step 7

If we want to keep a valid pointer to the string, we need to request the pointer once again, after the value has been severed:

  ptr = g_variant_string_get (string, &length);