Participant 1
Think-Aloud Protocol Participant-1-TAP
Usability Testing Data Sheet Participant-1-UT-data-sheet
Usability Testing Data Sheet (SUS-Record) Participant-1-UT-scale-consent
Pre Usability Test Question
First, what’s your occupation? What do you do all day?
Estimator
Roughly how many hours a week altogether—just a rough estimate— would you say you spend using the Internet, including Web browsing and email, at work and at home?
60 hours
What kinds of sites (work and personal) are you looking at when you browse the Web?
Google, emails, Facebook, DoneDeal, Youtube, Netflix
Do you have any favourite Web sites?
Not Really
Key Observations
- Identified opening screen as the opening screen for a game that involves the player escaping to Mars.
- Notes that the two button layout is not clear and that they would not know which button to select.
- Tried to select the setting button before the home button to navigate back to the home screen.
- Repeatedly tried to drag the H2 onto the bottle before the O.
- Was confused by whether to drag Titanium or Aluminium to make the landing craft.
- Tried to drag the H element into the molecule before dragging both C elements.
Participant 2
Think-Aloud Protocol Participant-2-TAP
Usability Testing Data Sheet Participant-2-UT-data-sheet
Usability Testing Data Sheet (SUS-Record) Participant-2-UT-scale-consent
Pre Usability Test Question
First, what’s your occupation? What do you do all day?
Relationship Counsellor
Roughly how many hours a week altogether—just a rough estimate— would you say you spend using the Internet, including Web browsing and email, at work and at home?
16 hours
What kinds of sites (work and personal) are you looking at when you browse the Web?
Youtube, Facebook
Do you have any favourite Web sites?
Youtube
Key Observations
- Identified opening screen to be suggestive of a game for children.
- Identifies the home screen as being in the spaceship and thinks that they might have to use the control panel to navigate to Mars.
- The participant is unable to complete the find the interesting fact about Titanium task, moves straight to task two, complete level 1.
- The participant completes the oxygen level of the first task, they do not understand that they have to move the level down until the bar goes blue, cannot complete the second task.
- Clicks on the level bar at the top thinking that this is what is bringing them to the next screen.
- Tries to drag H2 element to bottle first on first three bottles, does work out on the fourth bottle that this is not working as the O has to go first.
- Is confused at dragging the element on top of the landing craft as the final portion of the illustration is a flame.
- Some difficulty using the drag and drop function.
Participant 3
I recorded the third Usability Test session using QuickTime. This was great as it allowed me to complete the whole session in one recording. However, due to the large file size, I have had to upload it to google drive rather than insert the clip directly in this blog. The session is viewable using the below link.
Participant 3 usability test recording
Think-Aloud Protocol Participant-3-TAP
Usability Testing Data Sheet Participant-3-UT-data-sheet
Usability Testing Data Sheet (SUS-Record) Participant-3-UT-scale-consent
Pre Usability Test Question
First, what’s your occupation? What do you do all day?
Steel detailer
Roughly how many hours a week altogether—just a rough estimate— would you say you spend using the Internet, including Web browsing and email, at work and at home?
40 hours
What kinds of sites (work and personal) are you looking at when you browse the Web?
Youtube, Steam (PC gaming site), google chrome
Do you have any favourite Web sites?
Youtube
Key Observations
- Identified opening screen to be an exploration mobile game based in space, within our solar system.
- Identifies home screen as a control centre setting for a spaceship.
- Is easily able to navigate to the elements page and find an interesting face about titanium.
- Is able to identify the fact that titanium has been coloured grey as it is a metal.
- When clicking through on the first level they hit the element button in the top nav rather than anywhere on the screen. The participant realises they can click anywhere on the second level.
- Has some confusion on what elements to drag first in levels but realises quickly that they have to drag a different element in first.
System Usability Scale Calculations
Participant 1 SU Score: 85%
Participant 2 SU Score: 42.5%
Participant 3 SU Score: 80%
Average SU Score: 69.17%
I got mixed feedback from participants though found a strong correlation between time spent online and usability scores. Participants who spent 40+ hours online each week scored the app 80% and over while the participant that spent 16 hours a week online scored the app 42.5%
All participants scored the statement various functions in this system were well integrated as 4 out of 5 and strongly disagreed that there was too much inconsistency in this system (all score the statement 1)
There was mixed feedback on whether the participants would like to use this system frequently (two 3s and one 1). None of the users strongly agreed with this statement however as the game is aimed at a much younger age group I believe this was to be expected. There was also a high level of agreement from all participants that most people would be able to learn the system quite quickly. As participant two had the most issues using the system I have placed the observations made in their usability testing as high priority items to fix to make the system more accessible to all users.
Actions to be taken
- Make the two selections on the home screen more clear so that participants can understand where each button will bring them.
- Give users hints so that clear instructions are available if they are unable to complete a level.
- Include signifiers so that the user knows what elements to select first.
- Swap the H2 and O placement in the water level and make H2 selectable first follow how the elements are ordered when spoken “H2O” not “OH2”.
- Include buttons on screens that take the user to the next page.
- Swap setting and home button placement.