Mail Threading

Camel API

The Camel API is blocking and synchronous, as such it should not be called directly from the main GUI thread, as it will block the user interface while it is processing.

Custom Threads

Two tasks inside Camel use threads internally in order to implement a cancellable api ontop of a non-cancellable one. This is the host lookup functions camel_getaddrinfo and camel_getnameinfo. The threading used in these functions is completely separate from other threading inside Camel or the mailer.

Mail-OPS

mail-ops.h contains asynchronous versions of varous Camel api's, and also more complex functionality which may involve multiple Camel calls. It is a mapping layer which converts the synchronous camel api's into a function which invokes a callback once the task is complete.

mail-mt.h

Mail-mg.h contains a few thread helpers including MailMsg, MailAsyncEvent, and mail_call_main.

MailMsg

The MailMsg api, in mail-mt.h, provides an api which helps the application manage processing across multiple threads. It is used where a task involves a Camel-related component which must run in another thread, and a GUI related component which runs afterwards. It is used directly by the Mail-OPS code to asynchronise Camel tasks.

The core of the api is the mail_msg_t which each user sub-classes in order to pass its own data around. After a message is created (mail_msg_new), additional fields are initialised by the client code, then the message is sent to an EThread for processing. The ops vtable is used to run each stage of the processing, until final cleanup.

• describe_msg() is used to obtain a description of the task. If supplied, this is used to create the Shell status button.

• receive_msg() is called in the target thread. This is where Camel operations will be carried out.

• reply_msg() is called back in the GUI thread. This is where the client code can update the user interface/invoke Gtk+.

• destroy_msg() is called to free any user data. The mail_msg implementation frees its own data.

In addition to the basic service of updating the Shell status button, which is based on the CamelOperation contained in the structure, the mail_msg code will automatically show an error box if the exception is set after the task has complete, and the describe_msg() call is supplied. This has the beneficial effect of not having to have separate error handling in every function, with the draw-back of not having control over the error messages either. Similarly, the CamelOperation is registered automatically in the target thread if the describe_msg() function is supplied.

 typedef struct _mail_msg {
        EMsg msg;               /* parent type */
        struct _mail_msg_op *ops; /* operation functions */
        unsigned int seq;       /* seq number for synchronisation */
        CamelOperation *cancel; /* a cancellation/status handle */
        CamelException ex;      /* an initialised camel exception, upto the caller to use this */
        struct _mail_msg_priv *priv; /* private for internal use */
 } mail_msg_t;

Client code will sub-class this structure and append its own fields as appropriate.

 /* callback functions for thread message */
 typedef struct _mail_msg_op {
        char *(*describe_msg)(struct _mail_msg *msg, int complete); 
 
        void (*receive_msg)(struct _mail_msg *msg);     /* message received */
        void (*reply_msg)(struct _mail_msg *msg);       /* message replied */
        void (*destroy_msg)(struct _mail_msg *msg);     /* finalise message */
 } mail_msg_op_t;

Thread Pools

There are 3 pre-defined thread pools which are used throughout Evolution mail.

• thread_new: A new thread is created for each message enqueued onto it. Up to a limit of 10 threads, after which the threads act as a pool, with any thread being able to process each message/job. This is used where the order of execution is not important, and the task will probably run quickly.
• thread_queued: A single worker thread is created, and every message is processed in order. This is used where order is important, such as loading a folder followed by retrieving a message.
• thread_queued_slow: A single worker thread is created, and every message is processed in order. This is used for longer-running tasks which should probably be run in order to avoid resource starvation.

CamelSession

CamelSession also includes a thread abstraction similar to Mail-MT. It is used by CamelFolder for handling client-side filtering, and CamelVeeFolder for handling updates.

It does not have a notion of a GUI thread, so it only has two callbacks, a processing callback and a free callback.

MailAsyncEvent

Because camel's events may come in from any thread at any time, MailAsyncEvent was born. It allows application code to redirect an event back to the GUI thread, or to another new thread for processing. It contains a completely separate queue for each instance, and lets application code clean up properly.

In a multi-threaded context, it may be possible to receive an event after the event has been unhooked. i.e. an event occurs in a thread, it is then proxied to the main thread. Before the proxied event can be handled, the event is unhooked, and resources freed. After this point the proxied event arrives; leading to inconsistent data and potential crashes. MailAsyncEvent handles this by ensuring that any outstanding events are fully processed before proceeding.

mail_call_main

mail_call_main is a simple function used to marshal a call from a given thread to the main gui thread, and wait for the results. This is used in a few special cases in order to call simple/quick functions that must run in the gui thread from any other thread. Its api is based on providing the callback signature form a fixed-set of signatures and the callback, and using varags.

Threading Primitives

A number of generally useful threading primitives were developed for Evolution and are available in libedataserver.

EThread

EThread is a simple abstraction which manages worker threads and thread pools. It has two major modes of operation, queued, where each job is processed in order, and new, where a new thread is launched for each job. An optional limit limits how many threads will be launched concurrently. This code pre-dates GThread, and its continued usefulness is up for debate.

EMsgPort

EMsgPort is a simple, re-usable, multi-thread safe, asynchronous message queue. It is used by Mail-MT, EThread, and various other parts of evolution to simplify inter-thread communications. It predates GAsyncQueue, and provides one cruicial benefit - the ability to poll-wait for messages to arrive.

In addition to the purely pthread-based wait function, you can obtain a file descriptor (either unix or NSPR) on which you can poll for READ events. When a READ event occurs on this file descriptor, a message has probably arrived, and the non-blocking get() interface can be used to check for it.

• put(): Enqueues a message into the message port. After it has been enqueued, the callee relinquishes any access to the message data.

• get(): Retrieves the oldest message in the queue. This is a non-blocking call and returns NULL if nothing is available yet.

• wait(): Waits using pthread primitives for a message to arrive. If using this interface, you can only wait on this queue.

• getfd(): Returns a file descriptor on which you can poll for IN/READ events to wait for messages to arrive. Using this mechanism you can wait on multiple ports simultaneously, or other I/O events, or the file descriptor can be passed to a GIOChannel, and waited for asynchronously using a main loop callback.

EMutex

EMutex is an abstracted mutex which provides different mutex options through the same api. A normal fast mutex and a custom recursive mutex is available. This code pre-dates GRecMutex, and apart from having the same api to access the mutex regardless of type, is probably not that useful anymore.