CSE 270: Software Testing - Reading Material

5.2 Usability Testing

Introduction

Usability is a fundamental aspect of software design that focuses on how user-friendly and efficient a system is for its intended users. The five factors of usability, often referred to as the "Usability Heuristics" or "Nielsen's Heuristics," were introduced by usability expert Jakob Nielsen. These factors provide guidelines for evaluating and enhancing the usability of software. Here are the five factors with examples:

Five Factors of Usability

• Learnability:
  • Definition: The ease with which users can understand and learn how to use a system on their first interaction.
  • Example: A mobile app with an intuitive onboarding process that guides users through key features upon first use, allowing them to quickly grasp the app's functionality without the need for extensive tutorials.
• Efficiency:
  • Definition: The speed and accuracy with which users can accomplish tasks once they are familiar with the system.
  • Example: A web application with a streamlined navigation menu and shortcut keys that enable experienced users to perform complex actions quickly, minimizing the number of steps required to complete tasks.
• Memorability:
  • Definition: The ability of users to remember how to use a system after an initial learning period and during subsequent interactions.
  • Example: An e-commerce website that employs consistent and standardized icons, labels, and navigation patterns, making it easy for users to remember where to find specific features or information across multiple visits.
• Errors:
  • Definition: The extent to which users make errors while interacting with the system, the severity of those errors, and the system's ability to help users recover.
  • Example: A software application that provides clear error messages with actionable guidance when users input invalid data, helping them correct mistakes and continue with their tasks without frustration.
• Satisfaction:
  • Definition: The overall pleasantness and user satisfaction derived from interacting with the system, considering the users' subjective experiences and emotional responses.
  • Example: A mobile game that not only meets functional requirements but also incorporates engaging graphics, smooth animations, and positive feedback, creating an enjoyable and satisfying user experience.

Planning and Executing a Usability Test

Planning an effective usability study involves careful preparation, thoughtful consideration of objectives, and the selection of appropriate methods and participants. Here is a step-by-step guide on how to plan and execute an effective usability study.

• Test Planning:
  • Testers collaborate to define the goals, scope, and target audience for the usability testing. They identify key user scenarios, tasks, and metrics to measure usability.
• Test Environment Setup:
  • Testers prepare the necessary test environments including software build or version to be tested, test data and tools or equipment required for the test.
• Test Script Development:
  • Testers create test scripts or scenarios that outline the specific tasks and interactions users will perform during the test. These scripts provide instructions for the test participants to follow.
• Participant Recruitment:
  • Testers identify and recruit representative users who match the target audience profile. This ensures that the usability test covers a range of user perspectives and experiences.
• Test Execution:
  • Testers conduct the usability test sessions, either in-person or remotely. They guide participants through the test scenarios and observe their interactions with the software, noting any issues, challenges, or areas of confusion.
• Data Collection:
  • Testers collect qualitative and quantitative data during the usability test, including observations, user feedback, task completion rates, time taken to complete tasks, and subjective ratings from participants.

Qualitative and Quantitative Metrics in Usability Testing

In usability testing, qualitative and quantitative methods serve distinct purposes, each contributing unique insights into the user experience.

Qualitative methods focus on understanding the underlying reasons, behaviors, and attitudes of users. Through approaches like interviews, observations, and think-aloud protocols, qualitative methods delve into the subjective aspects of user interactions, providing rich, non-numeric data.

Quantitative methods aim to measure specific metrics and quantify aspects of the user experience. This involves numeric data, statistical analyses, and the measurement of variables such as task success rates, completion times, and error rates. The quantitative approach requires a larger sample size for statistical significance, allowing for objective comparisons and the identification of patterns or trends.

Below are some examples of each type of method.

Qualitative Metrics:

• Observations: Detailed notes on participant behavior and comments.
• Think-Aloud Protocol: Participants verbalize thoughts while interacting with the application.
• User Feedback: Gathering subjective opinions and insights through interviews or surveys.

Quantitative Metrics:

• Task Success Rate: Percentage of successfully completed tasks.
• Time on Task: Average time taken by participants to complete specific tasks.
• Error Rate: Frequency and severity of user errors.
• Satisfaction Scores: Numeric ratings or Likert scales measuring user satisfaction.

Usability Testing Outcomes

Usability testing generates valuable outcomes that provide insights into the effectiveness, efficiency, and satisfaction of users interacting with a software system. Here are key outcomes of usability testing and how software testers can benefit from and use them:

• Identification of Usability Issues:
  • Usability testing reveals specific issues users encounter while interacting with the software, such as navigation challenges, confusing interfaces, or unclear instructions. Testers can use these identified issues to create targeted test cases, ensuring that problematic areas are thoroughly tested and addressed in subsequent releases.
• User Feedback and Insights:
  • Usability testing captures qualitative feedback and insights from users, providing an understanding of their preferences, expectations, and challenges. Testers gain a deeper understanding of user perspectives, allowing them to advocate for user-centric design changes and improvements that align with user needs and preferences.
• Quantitative Usability Metrics:
  • Usability testing generates quantitative metrics, such as task completion rates, error rates, and time to complete tasks. Testers can use these metrics to objectively measure the success and efficiency of user interactions. Benchmarking against previous tests or industry standards provides a basis for quantitative comparisons and improvements.
• Validation of Design Changes:
  • Usability testing helps validate the effectiveness of design changes or new features by assessing how well users can interact with and understand them. Testers can use usability testing results to confirm that design modifications have the intended positive impact on user experience. This validation contributes to making informed decisions about design iterations.
• Prioritization of Usability Enhancements:
  • Usability testing reveals the severity and impact of identified usability issues, helping prioritize which aspects require immediate attention. Testers can collaborate with developers and product teams to prioritize usability enhancements based on the severity of issues uncovered during testing, ensuring that critical issues are addressed promptly.
• Enhanced Communication with Stakeholders:
  • Usability testing results provide tangible evidence of user experiences, making it easier for testers to communicate findings to stakeholders. Clear communication fosters a shared understanding among team members, stakeholders, and developers, facilitating collaborative efforts to address usability concerns and improve the software.
• User Persona Refinement:
  • Usability testing often reveals variations in user behavior and preferences, contributing to the refinement of user personas. Testers can use these refined personas to tailor testing scenarios and test cases more accurately, ensuring that testing reflects the diverse needs and expectations of the target user audience.

Useful Links:  ←Unit 5.1 |  Unit 5.3→  | Table of Contents | Canvas