GNOME Control Center Usability Study - Dec 2008

Back to Main Page

(DRAFT: Last updated by Kristin Travis on December 22/08)

A system preferences usability study occurred Nov. 21-25 in MPK (contact Kristin or Jenya for additional details).

The primary goal of this study was to confirm the system preferences categories we have been considering for Open Solaris.

We structured the study into three sections, where 10 participants:

• Completed some common system preferences tasks on both a PC and Mac
• Performed a card sorting task to categorize preferences content
• Reviewed some initial wireframes.

Study Script available here

Top Level Findings

Preferences Tasks

• Participants said they did not frequently change their system preferences, or default preferences.
  • We confirmed that changes are made infrequently, but the implied expectation is that changes can be completed quickly (some participants gave up on finding content relatively quickly, after a minute or so)
• Participants did not have any difficulty getting to the 'system preferences' area to make changes. They used the Apple Menu, Apple dock, Windows Control Panel, and right clicked on the desktop to access preferences.
• Generally speaking, participants completed most tasks very quickly (usually less than a minute or two) on both Mac and PC, regardless of their experience with each platform.
  • In general, participants said task completion was "easy"
  • Participants had no real preference for PC vs. Mac operation

Card Sorting

• Participants did not have major issues sorting the cards that represented the system preference categories
  • Participants added some new categories, and restructured others (removed, etc.)
  • Participants were fairly confident about the categorizations overall
• Repeating the name of the item in a subcategory was helpful, even if the terminology wasn't clear, for example, participants grouped Accessibility and Keyboard Accessibility even if they didn't know what Accessibility meant
• The most 'problematic' categories were Media (Audio, Video, Voice, Storage), General Admin, and Removable Devices. These areas will need additional thought and work.
• Participants said the Display and Appearance categories had a lot of overlap: they sometimes combined these two categories.
• Most terminology was understood. Some terms need to be reviewed, such as Application Organizing, Window Behavior, Automatically Start (applications vs. the computer), and Accessibility.

Section 1: Preferences Tasks

Current Usage

Participants described which of the following tasks they completed and how often:

The frequencies were defined as:

• Often: once a week or more
• Sometimes: two-three times a month
• Rarely: once every six months or less

Note: More than one data point applied to some participants. For example, some participants said they performed the task "sometimes to rarely".

Study Tasks: Background

Each participant completed approximately three tasks on both platforms (NEED TO SPECIFY OS VERSION STILL **), such as changing fonts, screen resolution, mouse controls, power options, depending on the tasks with which they were familiar. The last two participants were instructed to complete specific tasks.

The order of the tasks varied. Half of the participants started with the Mac, and half started with the PC, alternately completing tasks on each.

Study Tasks: Results

Overall, there were no real issues with completing the tasks.

• Changing power options: The Energy Saver (Mac) and the 'Show Details' button (needed to see and change content) slowed down some participants
• There were a few comments on icons/pictures making it easier to find things.

• When asked if one platform seemed easier than the other for the tasks they just completed, participants did not have a strong preference for one platform over the other. Similarly, they did not have a strong preference for using Start-> Control Panel or Menus to access system preferences.

Details for each task per participant are also provided.

Section 2: Card Sorting

This image shows our starting point for categories. We hoped to confirm that participants would make similar categorizations.

After working through an example task with the moderator, participants individually categorized content for main and sub-categories of system preferences. This activity lasted approximately 12-20 minutes.

White cards represented main categories and blue cards represented items in the main categories. Participants also added (dark blue cards) or removed or duplicated categories at their discretion.

Note: Not all categories were included in the study, due to time constraints, and areas we wanted to focus on.

After participants were finished, a discussion about their categorizations followed. Participants provided feedback on any issues, definitions, re-structuring, etc. They also indicated their satisfaction/confidence level with the categories they sorted.

A few pictures of final categorizations are provided:

Most categorizations were consistently sorted. Details per participant are provided here.

Note: General Admin additions from P4 are still to be updated.

Participants had some issues with a few categories, resulting in more variability for the following categories:

Section 3: Comments on wireframes

Participants were shown early screens of our next steps. Some participants noted the rollover approach may obscure some information, while others were not concerned by this.

Based on comments, we will be updating this approach.

Back to Main Page

CategoryUsability