Threads/Concurrency with Python and the GNOME Platform

Operations which could potentially block should not be executed in the main loop. The main loop is in charge of input processing and drawing and blocking it results in the user interface freezing. For the user this means not getting any feedback and not being able to pause or abort the operation which causes the problem.

Such an operation might be:

The following examples show

Threads

The first example uses a Python thread to execute code in the background while still showing feedback on the progress in a window.

import threading
import time

from gi.repository import GLib, Gtk, GObject


def app_main():
    win = Gtk.Window(default_height=50, default_width=300)
    win.connect("delete-event", Gtk.main_quit)

    progress = Gtk.ProgressBar(show_text=True)
    win.add(progress)

    def update_progess(i):
        progress.pulse()
        progress.set_text(str(i))
        return False

    def example_target():
        for i in range(50):
            GLib.idle_add(update_progess, i)
            time.sleep(0.2)

    win.show_all()

    thread = threading.Thread(target=example_target)
    thread.daemon = True
    thread.start()


if __name__ == "__main__":
    # Calling GObject.threads_init() is not needed for PyGObject 3.10.2+
    GObject.threads_init()

    app_main()
    Gtk.main()

The example shows a simple window containing a progress bar. After everything is set up it constructs a Python thread, passes it a function to execute, starts the thread and the GTK+ main loop. After the main loop is started it is possible to see the window and interact with it.

In the background example_target() gets executed and calls GLib.idle_add() and time.sleep() in a loop. In this example time.sleep() represents the blocking operation. GLib.idle_add() takes the update_progess() function and arguments that will get passed to the function and asks the main loop to schedule its execution in the main thread. This is needed because GTK+ isn't thread safe; only one thread, the main thread, is allowed to call GTK+ code at all times.

Before using Python threads, or libraries using threads (GStreamer for example), you have to call GObject.threads_init(). Contrary to the naming, this function isn't provided by gobject but initializes thread support in PyGObject (it was called gobject.threads_init() in pygtk). Think of it as gi.threads_init().

Since PyGObject 3.10.2, calling GObject.threads_init() this is no longer needed.

Threads: The Deprecated Way

It is actually supported to call GTK+ from multiple threads, but this support is deprecated and will be removed eventually. Also note that this only works under X11 (Linux, BSD) and not under Windows, for example.

The following program spawns a thread which periodically calls GTK+ code. In addition, two timers are installed, which both change the window title periodically and a button which changes its label on a click event.

import threading
import time

from gi.repository import Gtk, GObject, Gdk, GLib


def app_main_deprecated():
    win = Gtk.Window(default_height=50, default_width=300)
    win.connect("delete-event", Gtk.main_quit)

    progress = Gtk.ProgressBar(show_text=True)

    def example_target():
        for i in range(50):
            Gdk.threads_enter()
            progress.pulse()
            progress.set_text(str(i))
            Gdk.threads_leave()
            time.sleep(0.2)

    def change_title_gdk(*args):
        win.set_title("change_title_gdk")
        return False

    Gdk.threads_add_timeout_seconds(
        GLib.PRIORITY_DEFAULT, 2, change_title_gdk, None)

    def change_title_glib(*args):
        Gdk.threads_enter()
        win.set_title("change_title_glib")
        Gdk.threads_leave()
        return False

    GLib.timeout_add_seconds(4, change_title_glib)

    def change_title_click(button):
        button.set_label("You clicked me")

    button = Gtk.Button(label="Click Me")
    button.connect("clicked", change_title_click)

    box = Gtk.Box()
    box.pack_start(button, False, True, 0)
    box.pack_start(progress, True, True, 0)
    win.add(box)
    win.show_all()

    thread = threading.Thread(target=example_target)
    thread.daemon = True
    thread.start()


if __name__ == "__main__":
    # Calling GObject.threads_init() is not needed for PyGObject 3.10.2+
    GObject.threads_init()

    Gdk.threads_init()
    Gdk.threads_enter()
    app_main_deprecated()
    Gtk.main()
    Gdk.threads_leave()

The way calling GTK+ from a thread works is that with Gdk.threads_init() a global lock gets initialized which protects any GTK+ code that isn't thread-safe. For this to work, anything that calls GTK+ code must acquire the lock by calling Gdk.threads_enter() before this critical section and release it with Gdk.threads_leave() afterwards. The lock itself is reentrant, so it can be locked multiple times by the same thread if it gets unlocked the same amount of time afterwards. This way, anytime a thread holds the lock, it can safely call GTK+ code.

Back to the example: Before Gtk.main() gets called and the window gets set up, the lock is initialized and locked. After that the main loop gets started and after it returns the lock is unlocked. The lock is not actually held the whole time until Gtk.main() returns, but will be released before the main loop starts and re-acquired before the function returns.

The thread will execute example_target() in which it periodically acquires the lock, calls GTK+ code and release it again. After it acquires the lock, it can be sure that GTK+ is in a valid state and no other context is currently calling GTK+, so it's safe to call GTK+ from there.

Now the question is, since the lock must be held in all threads including the main thread to achieve mutual exclusion, where else is it needed to acquire the lock during 'normal', non threading, usage?

After the main loop has started there are two ways your code can be called:

Threads: FAQ

        # [...]

        toggle_button = Gtk.ToggleButton()

        def signal_handler_in_thread():

            def function_calling_gtk(event, result):
                result.append(toggle_button.get_active())
                event.set()

            event = threading.Event()
            result = []
            GLib.idle_add(function_calling_gtk, event, result)
            event.wait()
            toggle_button_is_active = result[0]
            print(toggle_button_is_active)

        # [...]

Using Generators

Another way to get concurrent execution is by splitting the long running operation in short parts and execute them one by one when the main loop is idle i.e. there is nothing to draw and no user input to process.

import time

from gi.repository import GLib, Gtk


def main_app():
    win = Gtk.Window(default_height=50, default_width=200)
    win.connect("delete-event", Gtk.main_quit)

    progress = Gtk.ProgressBar(show_text=True)
    win.add(progress)
    win.show_all()

    def example_generator():
        for i in range(50):
            progress.pulse()
            progress.set_text(str(i))
            time.sleep(0.2)
            yield True

    def run_generator(function):
        gen = function()
        GLib.idle_add(lambda: next(gen, False), priority=GLib.PRIORITY_LOW)

    run_generator(example_generator)


if __name__ == "__main__":
    main_app()
    Gtk.main()

Instead of a thread this uses a generator which executes the long running operation but uses yield to pass control to the main loop in regular intervals.

The generator gets instantiated and the next() function, which resumes the execution of the generator, is scheduled using GLib.idle_add(). GLib.idle_add() will re-schedule the call to next() until it returns False which will happen if the generator is exhausted, meaning the function is finished and has returned. Every time the generator yields, GTK+ has time to process events and update the user interface.

If you run this example you will notice that interaction with the window isn't as smooth as with the previous examples using threads and in case you are running GTK+ 3.12+ that there is no smooth animation of the progress bar. The reason is that each step of example_generator() is executed in the main thread and during this time GTK+ can't react to your input.

Besides these shortcomings, this approach still can be useful if the steps are short enough or you just want to get things working without having to think about threads and shared state.

Asynchronous Operations

In addition to functions for blocking I/O glib also provides corresponding asynchronous versions, usually with the same name plus a _async suffix. These functions do the same operation as the synchronous ones but don't block during their execution. Instead of blocking they execute the operation in the background and call a callback once the operation is finished or got canceled.

The following example shows how to download a web page and display the source in a text field. In addition it's possible to abort the running operation.

import time

from gi.repository import Gio, GLib, Gtk


class DownloadWindow(Gtk.Window):

    def __init__(self):
        super(DownloadWindow, self).__init__(
            default_width=500, default_height=400, title="Async I/O Example")

        self.cancellable = Gio.Cancellable()

        self.cancel_button = Gtk.Button(label="Cancel")
        self.cancel_button.connect("clicked", self.on_cancel_clicked)
        self.cancel_button.set_sensitive(False)

        self.start_button = Gtk.Button(label="Load")
        self.start_button.connect("clicked", self.on_start_clicked)

        textview = Gtk.TextView()
        self.textbuffer = textview.get_buffer()
        scrolled = Gtk.ScrolledWindow()
        scrolled.add(textview)

        box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=6,
                      border_width=12)
        box.pack_start(self.start_button, False, True, 0)
        box.pack_start(self.cancel_button, False, True, 0)
        box.pack_start(scrolled, True, True, 0)

        self.add(box)

    def append_text(self, text):
        iter_ = self.textbuffer.get_end_iter()
        self.textbuffer.insert(iter_, "[%s] %s\n" % (str(time.time()), text))

    def on_start_clicked(self, button):
        button.set_sensitive(False)
        self.cancel_button.set_sensitive(True)
        self.append_text("Start clicked...")

        file_ = Gio.File.new_for_uri(
            "http://python-gtk-3-tutorial.readthedocs.org/")
        file_.load_contents_async(
            self.cancellable, self.on_ready_callback, None)

    def on_cancel_clicked(self, button):
        self.append_text("Cancel clicked...")
        self.cancellable.cancel()

    def on_ready_callback(self, source_object, result, user_data):
        try:
            succes, content, etag = source_object.load_contents_finish(result)
        except GLib.GError as e:
            self.append_text("Error: " + e.message)
        else:
            content_text = content[:100].decode("utf-8")
            self.append_text("Got content: " + content_text + "...")
        finally:
            self.cancellable.reset()
            self.cancel_button.set_sensitive(False)
            self.start_button.set_sensitive(True)


if __name__ == "__main__":
    win = DownloadWindow()
    win.show_all()
    win.connect("delete-event", Gtk.main_quit)

    Gtk.main()

The example uses the asynchronous version of Gio.File.load_contents() to load the content of an URI pointing to a web page, but first we look at the simpler blocking alternative:

We create a Gio.File instance for our URI and call Gio.File.load_contents(), which, if it doesn't raise an error, returns the content of the web page we wanted.

    file = Gio.File.new_for_uri("http://python-gtk-3-tutorial.readthedocs.org/")
    try:
        status, contents, etag_out = file.load_contents(None)
    except GLib.GError:
        print("Error!")
    else:
        print(contents)

In the asynchronous variant we need two more things:

The window contains two buttons for which we register clicked signal handlers:

Once the operation is finished, either because the result is available, an error occurred or the operation was canceled, on_ready_callback() will be called with the Gio.File instance and a Gio.AsyncResult instance which holds the result.

To get the result we now have to call Gio.File.load_contents_finish() which returns the same things as Gio.File.load_contents() except in this case the result is already there and it will return immediately without blocking.

After all this is done we call Gio.Cancellable.reset() so the Gio.Cancellable can be re-used for new operations and we can click the "Load" button again. This works since we made sure that only one operation can be active at any time by deactivating the "Load" button using Gtk.Widget.set_sensitive().

Known Issues

Projects/PyGObject/Threading (last edited 2014-03-26 23:47:00 by ChristophReiter)