Project

Running a usability test and developing  standardized processes for testing.

 areasMethodology

I took cues from the Nielsen Norman Group and Tomer Sharon, developing a process around the following areas:

  • Test design & planning
  • Piloting tests
  • Collecting test data
  • Collecting survey data
  • Evaluating data
  • Reporting findings

 

Skills used

  • Test design
  • Remote usability testing
  • In-person usability testing 
  • GoToMeeting
  • Excel 
  • Sketch

Problem

Big project

My team was tasked with improving an enterprise software that was visually and functionally outdated. The existing interface was difficult to use and left the software vulnerable to competitors seeking to develop similar products.

Users primarily complained about the time and effort required to complete specific tasks. 

I developed a usability test using a prototype designed to address and streamline the most tedious tasks such as data entry.

 

Challenges

The timeline was very tight and it was difficult to find representative users and get them to participate in testing. 

There was no existing standardized process for the collection and organization of data. I had to document the process and detail what worked and what didn't in order to begin developing a standard practice.

 Test Design

Test Design


Action

Planning  

I met with product managers in the early stages to discuss and prepare our approach.

I developed a shared test plan document that we could review together and update as needed.  This document helped us outline research goals, logistics, level of effort, participants, etc.

Collecting data

During research sessions, I collected data using the 'Rainbow Spreadsheet' method to record task scores and user comments. 

The sessions were recorded using GoToMeeting so I was able to review video and audio to fill out this score sheet. 

Synthesizing Data

I analyzed the data in the note taker form and began distilling it into numbers using an internal scoring system to determining things like; success, time-on-task, error rate and ease-of-use. 

 note taker sheet

note taker sheet

 Observation note taker

Observation note taker

 Rainbow spreadsheet

Rainbow spreadsheet


Results

Developing Standardization

I created and documented a system to keep track of the best practices for testing and outlined an engagement process for product managers to reference.

Reporting

An elaborate spreadsheet was created to house and score the top observations and insights.  This was ideal for communicating with other designers and those with research backgrounds but it was to complicated for high-level reporting so I developed more concise reporting formats that were easy to understand at a glance that included: 

  1. Simple visualizations
  2. A summery of the average users experience
  3. Top observations and insights

This became the collecting and reporting standard for future tests. 

 

 Engagment process map and timeline

Engagment process map and timeline

 Task evaluation summery 

Task evaluation summery 

 Usability study summery

Usability study summery