Looking Chapter in Innovating for People: Handbook of Human-centered Design Methods. Pittsburgh, PA: LUMA Institute, LLC 2012. Print
- Think-Aloud Testing
- People narrate their experience while performing a task
- Purpose:
- Reveals what people are thinking
- Deepen empathy
- Uncover opportunities for improvement
- Lowers development costs through early discovery
- Preparation steps:
- Identify tasks to investigate
- Recruit 6-9 participants
- Schedule testing sessions for each participant
- Implementation steps:
- Introduce yourself and purpose
- Remind them – We are not testing you.
- Instruct them to perform each task while thinking aloud.
- Take good notes.
- Thank each participant.
- Helpful hints:
- Withhold question until end of test
- Avoid temptation to demonstrate task
- Mimic functionality if the design is still in process
- Sample process:
- Purpose: Rapid iteration (quick test, implement and improve solutions)
- Steps:
- Rough & Ready Prototyping (Making)
- Think Aloud Testing (Looking)
- Bull’s eye Diagramming (Understanding)
- Schematic Diagramming (Making)
- School applications:
- Can use this method to research how classroom procedures are working out for students
- Can use this method to gather data from students on how they experiences new activities for learning key processes
- Heuristic Review
- Auditing procedure based on 10 reals of thumb for good design
- Purposes:
- Leverage good design principles
- Identify problems quickly
- Yields data in the absence of test participants
- Shows improvement opportunities
- Preparation steps:
- Identify research topic
- Assemble a team with multiple perspectives
- Select a small number of key tasks.
- Implementation steps:
- Get everyone familiar with 10 heuristics
- Match mental model
- Minimize perceived complexity
- Use consistent form, words and actions
- Provide sense of place
- Account for use and environmental constraints
- Anticipate needs
- Use clear and concise language
- Geed feedback about actions and status
- Prevent errors and provide graceful recovery
- Strive for appropriate and minimal aesthetics
- Instruct participants to perform tasks with heuristics in mind
- Give each reviewer a pen and sticky pad.
- Have reviewers annotate issues they discover and cite each to a design heuristic.
- Get everyone familiar with 10 heuristics
- Helpful hints:
- Have reviewers initial their notes
- Encourage clear communication
- Discourage inclusion of solutions at this phase
- Sample process:
- Purpose: Evaluating and improving existing systems
- Steps:
- Heuristic review (Looking)
- Affinity clustering (Understanding)
- Importance / Difficulty Matrix (Understanding)
- School applications:
- Can use a simpler list of design principles and use this as an activity for students to evaluate proposed solutions to a project
- A school team can use this activity to analyze the activities and policies that are supporting a targeted initiative
- Critique
- Forum for people to give and receive constructive feedback
- Purposes:
- Facilitate constructive discussion
- Revel blind spots in design activities
- Show improvement opportunities
- Builds organizational alignment
- Preparation steps:
- Identify project and group of reviewers.
- Pick time and place for session.
- Implementation:
- Presenters:
- describe what has been done and why
- provide clarification
- invite suggestions from reviews
- thank everyone for participating
- Reviewers:
- ask questions
- start with warm (positive) feedback
- end with cool (negative) feedback
- Presenters:
- Helpful hints:
- Invite reviewers who didn’t do the project work
- don’t wait for completeness to invite critique
- ask for feedback often
- Sample process:
- Purpose: Developing, testing and advancing a concept
- Steps:
- Concept poster (Making)
- Rose, Thorn, Bud (Understanding)
- Critique (Looking)
- Storyboarding (Making)
- School applications:
- Can use this activity to gather teacher and peer feedback on project drafts
- Can use this activity gather feedback on project drafts prior to finalizing and implementing projects
- System Usability Scale
- Short survey for quantifying usability feedback
- Purposes:
- Standardizes evals
- Provides manageable numeric score
- Helps with quick assessments
- Preparation steps:
- Identify research topics and key tasks
- Invite group of participants
- Download System Usability Scale (SUS)
- Implementation steps:
- Conduct task-based usability test.
- Administed SUS questionnaire after test.
- Calculate total score for each questionnaire.
- Average all scores.
- Helpful hints:
- Give participants a printed or online form.
- Tell them to mark the center point if undecided.
- Don’t allow them to think about each item for too long.
- Sample process:
- Purpose: Benchmarking and improving usable features of a current design
- Steps:
- Think-Aloud Testing (Looking)
- System Usability Scale (Looking)
- Affinity Clustering (Understanding)
- Schematic Diagramming (Making)
- School applications:
- Can use this to gather data on school systems such as discipline system, T-TESS learning systems
- Can use this to gather data on classroom systems such as grading policies, classroom management systems
The Evaluative Research Methods can be used by teachers to gather feedback on project design. These methods can be scaffolded for students to use in design projects. Students can use these methods to gather feedback on their project designs before formalizing (building final versions) of their solutions.
Preparation Steps
- For teacher use (researching students):
- Decide research topics (examples: student feelings/preferences on project design, late work policies, classroom norms, classroom routines, etc.)
- Select method(s) that will help gather most useful information related to research topics
- For student use (researching stakeholders for project):
- Brainstorm research topics in projects that lend themselves to evaluative research methods
- Design resources that help students prepare and implement methods. See above.
Early Implementation Steps
- For teacher use (researching students):
- Implement evaluative research methods
- Follow-up with understanding steps. See hyperlinks above for ideas.
- For student use (researching stakeholders for project):
- Scaffold evaluative research activities (preparation and implementation)
- Follow-up with understanding activities. See hyperlinks above for ideas.
Advanced Implementation Steps
- For teacher use (researching students):
- Share findings with students and have them reflect on whether or not the findings have any validity.
- Develop interventions, calendars, strategies, activities, and routines that address verified findings.
- For student use (researching stakeholders for project):
- Have students reflect on evaluative research methods – how did it work? what assumptions were challenged? what new things were learned? what new ideas were inspired? how can this approach be used in other settings?
