TheDesignProcess
Ch. 1
1
2
3
4
5
6
7
8
Ch. 2
9
10
11
12
13
14
15
16
17
18
19
20
21

Lesson 18: Testing the App

Overview

Question of the Day: How does our digital prototype work when tested by real users?

By the end of the previous lesson each team should have a working prototype of their app. The primary purpose of this lesson is to have the team actually test the app with other people, preferably from the target audience the app is intended for, or from different teams in the class while observers from the team will record the results. As with testing the paper prototypes, teams will start by planning for the specific scenarios before running and observing tests.

Purpose

By now students should be fairly practiced at testing with users. This time around, the role of the computer is actually done by the computer! This means that the team running the test needs to focus their energy on getting carefully watching what the user does, and encourage the user to say aloud what they are experiencing. While it may seem at first that testing with an interactive prototype is easier (since no one has to manually change screens), it can actually be significantly more difficult to observe the user's experience.

Assessment Opportunities

  1. Design and run tests that use digital prototypes to gather feedback on a design

    Activity Guide: Several relevant tasks should be identified in the chart on the second page, along with notes describing the user testing around those tasks.

Agenda

Warm Up (5 min)

Activity (45 min)

Wrap Up (5 min)

View on Code Studio

Objectives

Students will be able to:

  • Design and run tests that use digital prototypes to gather feedback on a design

Preparation

  • Print one copy of the activity guide for each student
  • If possible, have outside users available to test the teams’ apps prior to this testing day

Links

Heads Up! Please make a copy of any documents you plan to share with students.

For the Teachers

For the Students

Support

Report a Bug

Provide Lesson Feedback

Teaching Guide

Warm Up (5 min)

Discussion Goal

Goal: Students should recall the user testing process, which they’ve already completed several times throughout this unit. Discussions will vary, but they should anticipate that some people will be observers taking notes on what the user does so they can incorporate it into feedback. Students may also realize that there is no more need for a “computer” because we’ve coded all of the navigation in App Lab!

Getting Prepared

Distribute: Make sure each team has their materials available and is able to open their digital prototypes on Code Studio.

Journal

Prompt: Today we will be testing our digital prototype with users. How do you think this will be similar to when we tested our paper prototypes? How do you think it will be different?

Discuss: Have students journal individually, then discuss in their groups before drawing out conversations as a full class.

Remarks

Today we will be testing our digital prototypes and getting one last round of feedback before we get our apps into a final state. After importing all of your screens yesterday, you should each have a copy of your team’s app. The first decision we need to make today is: who’s app are we going to use for testing?

Question of the Day: How does our digital prototype work when tested by real users?

Discuss: Have students discuss as a team which app they want to use for user testing. Teams may have already decided this at the end of class yesterday - if so, they can skip this discussion and jump right into the activity.

Activity (45 min)

Teaching Tip

Deja Vu The instructions and activity guide for this lesson will feel very familiar - it is basically a repeat of the Paper Prototype testing process. You may want to acknowledge this similarity with students, since it can help reinforce that they’re acting like real app designers who test their apps multiple times before releasing it - the fact that it feels so familiar means they’re getting more comfortable with the design process.

Preparing for Testing

Distribute: Give each team one copy of the Digital Prototype User Testing Activity Guide. Also give each team a copy of the User Test Cases Activity Guide - one copy for each user the team plans to test with.

Digital Prototype User Testing

Teaching Tip

Testing Logistics You may want to decide ahead of time how students will be able to physically test each other’s apps. If all students are using computers, consider having both teams gather around one set of computers for one test, then everyone moves and gathers around another set of computers for the other tests. Or, if students are able to use their personal devices and they are comfortable letting other students handle them, have students test the apps on each other’s devices.

Overview: Read through the (1) Overview section of the activity guide together. This gives an overview of today’s activity.

Assign Roles: Read through the (2) Assign Roles for Testing section of the activity guide. Have students discuss in their teams which roles they will take on in the first round of testing. There should be only one narrator and all other team members can be observers.

Create Test Cases: Read through the (3) Create Test Cases section of the activity guide. Students will work in the User Test Cases Activity Guide for this section, filling in the App Introduction section and completing the left-column of the table. They should not fill in the rest of the table - the user will do this instead.

Circulate: Monitor students as they fill in the User Test Cases Activity Guide. Make sure they are only filling in the left-most column with the tasks they want their user to test - the rest of the chart should be blank.

Test Your Prototype: Read through the (4) Test Your Prototype section to remind students the process for user testing. It is the same process they completed when testing their paper prototype, except this time they will test using their App Lab digital prototype.

Teaching Tip

Actionable Feedback: A key takeaway from this activity is that students leave with a plan for changes they can make to their app based on their testing. This usually requires setting your ego aside and seeing feedback as opportunities for improvement rather than evidence of failure. Remind students that these are just prototypes, nowhere near the final project. Also remind students not to take feedback personally - that all apps go through this process and they get better because of it.

Do This: Find another team to test your app with. Spend half your time for them to test your app, then take the other half of the time for you to test their app.

Circulate: Students will start their tests which should run for about 10 minutes each. The tests should last a little longer than the paper prototype tests, since the apps are more developed now that they’re in a digital form. Encourage students to keep on task, and encourage the observers to write as much as they can.

Summarize Findings: Read through the (5) Summarize Findings section of the activity guide. Have groups discuss what they observed and record their findings on the activity guide. In particular ask them what their observations mean in terms of changes they'll need to make for the user interface of their prototype.

Wrap Up (5 min)

Journal

Discussion Goal

Goal: Students may notice that, once again, users may have discovered issues that challenged their assumptions. However, students may also notice that the feedback was targeted more towards cosmetic and aesthetic choices, such as the size of fonts or the color of elements on the screen. Even though these might seem like small details that are easy to fix, they can have a large negative impact on the user experience.

Prompt: Think about the type of feedback from today’s testing compared to the feedback from testing your paper prototype. What was similar between the two types of feedback? What was different?

Discuss: Have students journal individually, then share in their groups. If there is time, ask groups to share their responses with the class.

Standards Alignment

View full course alignment

CSTA K-12 Computer Science Standards (2017)

AP - Algorithms & Programming
  • 2-AP-15 - Seek and incorporate feedback from team members and users to refine a solution that meets user needs.
  • 2-AP-17 - Systematically test and refine programs using a range of test cases.
  • 2-AP-18 - Distribute tasks and maintain a project timeline when collaboratively developing computational artifacts.
CS - Computing Systems
  • 2-CS-01 - Recommend improvements to the design of computing devices, based on an analysis of how users interact with the devices.
DA - Data & Analysis
  • 2-DA-08 - Collect data using computational tools and transform the data to make it more useful and reliable.
IC - Impacts of Computing
  • 2-IC-22 - Collaborate with many contributors through strategies such as crowdsourcing or surveys when creating a computational artifact.