Multi-touch gestures in Mutter-based compositors
Contents
Introduction
Because of the recent work on moving Mutter to use XInput2 for event handling, it is possible to handle multi-touch gestures in Mutter plugins, even if still a bit cumbersome. This page documents what currently needs to be done in plugins in order to handle gestures.
Approach
It may be a good idea to restrict the gestures that the compositor handles to those with more than 3 fingers, so that the compositor can quickly decide if a gesture is intended for itself, or for applications. Otherwise it can get very complex (not just for our code, but also for the user!) and introduce noticeably delay.
The approach discussed below and implemented in the example consists in waiting for a given maximum amount of time since the first finger touches the screen until the minimum required number of fingers are down. If that amount of time has passed and not all the fingers are present, the compositor rejects the current touch sequences and the X server will deliver them to the applications. The example has this limit set to an arbitrary value of 100 milliseconds (GESTURE_TIMEOUT), but ideally it would be measured the typical and worst-case amount of time that passes between the first and last touches reach the compositor.
Instead of dealing with XInput events directly, the plugin uses subclasses of ClutterGestureAction such as ClutterZoomAction.
Implementation
Currently Mutter doesn't have any support specific to touch events, so there's a bit of preparation that needs to be done by plugins themselves. In future releases, once GNOME Shell starts using touch events, this preparation will possibly be done by Mutter.
Replace the window member in the event struct
Clutter expects that all input events that a stage will emit belong to the X window that it relates to, dropping any events that don't. In our case, we'll get touch events from the root window due to our grab, but the stage's window is a different one so Clutter would drop all of them.
To avoid that, we have to intercept all touch events and replace the root window with the window of the stage, as shown in the mutter_gesture_plugin_xevent_filter() function in the example.
Grab touch events
In order to start receiving all touch events to check for interesting gestures, the plugin needs to set a grab for touch events in the root window by means of the function XIGrabTouchBegin(). See the function grab_touch_events() in the example.
Listen to events from ClutterGestureAction
Create whatever subclass of ClutterGestureAction you are interested in, attach it to the stage and connect to the appropriate signal (ie. ClutterZoomAction::zoom).
Additionally, you need to listen for all the touch events in the stage with the ClutterActor::captured-event signal and start a timeout on the first TouchBegin event for rejecting the sequences in case that there are less touch points than the ones required for system gestures.
Reject unwanted touch sequences
As soon as we have decided that we don't want to handle the gesture that is going on, we must tell the X server that we don't want it, so it can replay the touch events on whatever application has to handle it.
This is done by rejecting ownership of the touch sequences with the XIAllowTouchEvents() function with XIRejectTouch as shown in the reject_touch_sequence() function in the example.
It's recommended to call XIAllowTouchEvents() with XIAcceptTouch for the touch sequences if we are interested in handling the gesture.