1. Usability Testing Suite

I want to do those with open source software for work and open source contributions. What tool can you suggest? If there's no equivalent tool, is there an easy way to consolidate different open source apps like gtk-recordmydesktop, the Sound Recorder, Cheese, etc, to make them work as an all-in-one user testing tool like Morae? (From Allan Caeg)

We would like to put together a tool or set of tools to make usability testing easy using GNOME.

1.1. Plan

• collect data on personas through http://www.surveymonkey.com/s/GCFPJ9L (closed Feb 18, 2010)

• (just starting now) Put together a set of mockups to show how we'd like this application to work
• (later) Get some developers on board to help implement the mockups.

1.1.1. Resources

• usability-lab module in GNOME git.

• Software Usability Research Lab

1.1.2. Discussion

• 'Cheap Open Source Morae Alternative' Thread on usability-list@gnome.org.

• Máirín's open source portable usability lab

• UX Exchange discussion on open source alternatives to Morae

1.1.3. People

• Allan Caeg

• Máirín Duffy

• Ivanka Majic
• Wouter Bron
• Frederic Crozat

• Kirk Bridger

• Jan-Christoph Borchardt

1.2. Related Projects

1.2.1. Free

• Pongo (C version) - "The Pongo application is a beginning. It records the users webcam stream and puts it together with the screencast. As far as I know no sound is recorded though." (From Wouter Bron)

• Pongo (Python script) – »It catches sound and video from your web cam, records your desktop and merges it together into a ogg file that’s ready to publish on the web.« (Andreas Nilsson)

• Userecord – Continuing Andreas Nilssons work on Pongo, help is appreciated. (Jan-Christoph Borchardt)

• Screenie - "Last summer at Guadec I got all excited chatting with David Siegel (who is on my team) and Andreas (who worked on Pongo) so I started this and then, to my shame, did absolutely nothing with it." (from Ivanka Majic)

• Mandriva's Usability Lab Setup

1.2.2. Web-based

• ScreenToaster (discontinued) - "It's a Flash webapp that records your screen, audio, and webcam. Does it from a VNC server too. This app is a good alternative, but an offline GNOME-native app that does bookmarking is a lot better. Just an idea, but since this is a flash app, it could use something like youtube's caption feature." (From Allan Caeg)

• ScreenCastle uses Skoffer – Java app, flexible capturing area, endless recording (?)

• Screencast-O-Matic – Java app, flexible capturing area, 15 minutes recording

• Screenr – Java app, flexible capturing area, 5 minutes recording (seems to use the same system as Screencast-O-Matic)

• Screenjelly – Java app, fullscreen capturing, 3 minutes recording

1.2.3. Proprietary

• Morae (Windows) – it records a video of the user while using the product via webcam along with the voice (for the think aloud protocol), and the screencast. I got the trial version of Morae Manager. Apparently, Morae is composed of three apps. Other than the Manager, it has a "Recorder" and an "Observer". The Manager is in charge of analyzing, graphing, and presenting the data. If I understand correctly, Morae is split in three apps only because they are sold separately. In other words, we do not really have have to split our app into three (From Allan Caeg)

• Silverback (OSX) – Sophisticated version of Pongo with UI. Watch the screencast.

1.3. Current Approaches

1.3.1. Recording on the machine

We have tried mixtures of GTK record my desktop, etc and Pongo and it does the job but is all a bit complicated to set up and can sometimes slow down a machine which renders it pointless.

1.3.2. Recording external to the machine

Máirín's usability lab records via DVR external to the system so there is no slow down of the performance of the machine.

1.3.3. Using Gstreamer to stich video together

Ray Strode put together scripts that stitch together streams of video for usability testing. Máirín's usability lab spits out multiple video streams and an audio stream which are later stitched together using gstreamer, which works quite well and reliably.

1.4. Cool Ideas

1.4.1. CMML Bookmarking

There is a format called CMML with which you can set timestamp-based markers in video. You can set these markers using gstreamer. But we don't have players to understand them. I was thinking it would be awesome to be able to set markers in the video file (eg. task #1 starts now, task #2 starts now, point of interest A, point of interest using the CMML format, and then opening up the video in totem and writing a totem plugin so that the bookmarks show up in the totem sidebar and you can click on them to jump through the video. Or when rendering the video you ould use them to render the video into chunks based on the bookmark. (from Máirín Duffy)

1.5. Mockup Ideas

• Allan Caeg

  • Welcome

  • Record

  • Observe

  • View

• James Moschou

  • No tabs

1.6. User Definitions

• See Personas sub-page.