Empathy Research

• By Canonical, February, 2010

Purpose

This research serves many purposes: 1- Confirm that the changes made in Empathy in the new release have improved the use of Empathy 2- Look to further improve the usability of Empathy 3- Begin a reflection around the usability of the format of usability reports itself and transform reporting so it is easy to use in many contexts especially for developers

Methodology

A usability study of Empathy was conducted with 16 participants, half from the general public and half Empathy or Pidgin users.

A scenario of tasks was developed that took participants through sign-up, define the personal profile, interact with others, and quit.

Participants came to the Canonical office in London where we set a temporary lab, and conducted the tasks indicated by a moderator. The sessions lasted 60 minutes each.

Results

Participants responded positively to Empathy, stating that it is a useful and relatively easy application to work with.

They responded enthusiastically to the following features and functionalities:

Tabs: All the participants were thrilled by the tabs that allowed them to have parallel conversations with their contacts. They also were surprised and impressed by the interaction design of the tabs and the facility with which they could separate them and put them back together.

Calendar to find past conversations: Again, all the participants found the search calendar for past conversations exciting and very intuitive. They found the possibility of searching for specific conversations on specific very useful and sensible.

The usability report separated critical issues from medium and minor ones. Critical issues are usability issues that will prevent users from completing their tasks. Medium issues are those that will cause users to have to do extra work (like search under various menus) to accomplish their goal and this will decrease their satisfaction and enjoyment of the interface or applications. Minor issues are problems that mostly distract and slow down users without affecting their performance or satisfaction.

Where problems occurred:

• Participants did not always understand the implications of the network paradigm Participants expected information to be organised around people Participants needed more feedback on what the system was doing and if they had done the right action

Critical issues:

Issue 1: When participants signed up, they often didn't know if they were signed up or not because the screen showed confusing messages.

Issue 2: Participants did not know how their personal information appeared to others. The only information they saw about themselves was if they were available or not.

Issue 3: Participants were not able to figure out how to chat with more than one person. They thought they had to invite their contacts to join a group but then, could not figure out which server they should chose.

Issue 4: Participants did not understand what was happening when they received a file. They did not understand the icon and the flashing behaviour.

Issue 5: Many participants did not understand the network paradigm and were confused between actions that affected all networks, like status, and actions that affected only one network like changing alias.

Medium issues:

Issue 1: Participants expected the organisation of the information to be contextual, that is, the architecture of information to be structured around people. For example, they expected to see everything that has to do with a contact under the name of that contact; everything that has to do with themselves under personal information. As a consequence, when they were looking for specific information they were lost and had to open many tabs.

Issue 2: Sometimes, the process of doing a change involved more work than necessary. For example, to correct an error to a network when authentication has failed, participants needed to add a new network and go through the same process again. They could not, instead, modify the fields display on the screen.

Issue 3: Participants were puzzled by certain words used in Empathy. Many were not sure what an 'alias' or an 'identifier' meant. This could, eventually, compromise their use of Empathy.

Minor issues:

Empathy icon in panel not visible when you log on Many default settings were not what participants expected Offline icon not clear Clear password icon not clear Customise message icon not visiblefree Sidebar in video chat not understood Personal status not clear - some participants thought it was a filter Inconsistent language - log-in and save used interchangeably Same icon has 2 meanings: offline and invisible Search in the conversations has 2 behaviours: enter and click on 'find' Notification and sounds should be put together since they are 2 types of notification

Comments

Please feel free to add any comments about this study below. There are 2 types of comments that would be most useful.

First, on the findings.

Second, on the format of the report. As I mentioned in introduction, I am currently rethinking the format of usability testing reports. My goal is to make them easier for developers to use and enjoy. So any feedback on this would be most appreciated.

Comments

This report was presented at the Usability Hackfest in London. I want to thank everyone who was there for there input and helpful comments. Special thanks to Sjoerd Simons and Guillaume Desmottes who took a couple of hours reviewing the details of the report with me.