Two principles of product evaluation methods
Product evaluation is very important and becomes an activity in every company that has a product application. Usually in a company, there are divisions and individuals who specialize in this field. The job specialization at the company is usually named UX (User Experience), for example like UX Researcher and UX Designer. Naming it depends on each company, but generally like that. There are two principle methods in evaluating a product that uses the User Centered Design approach. the first is Inspection Methods and the second is User Test.
Inspection Methods
Inspection Methods can be interpreted as an evaluation method that does not require the users in the evaluation process, but involves usability experts to find usability problems by following instructions, heuristics, or tracing. These aspects of usability include effectiveness, efficiency, and user satisfaction with the system as a whole. There are several methods that use Inspection Method principles, some of them are Heuristic evaluations and Walkthroughs.
Heuristic Evaluations
As we all know, Heuristic Evaluations is method for finding usability problems. Popularised by Jakob Nielsen.
Heuristic evaluation is a usability inspection method for computer software that helps to identify usability problems in the user interface (UI) design. It specifically involves evaluators examining the interface and judging its compliance with recognized usability principles (the “heuristics”). These evaluation methods are now widely taught and practiced in the new media sector, where UIs are often designed in a short space of time on a budget that may restrict the amount of money available to provide for other types of interface testing.
The main goal of heuristic evaluations is to identify any problems associated with the design of user interfaces. Usability consultants Rolf Molich and Jakob Nielsen developed this method on the basis of several years of experience in teaching and consulting about usability engineering. Heuristic evaluations are one of the most informal methods[1] of usability inspection in the field of human–computer interaction. There are many sets of usability design heuristics; they are not mutually exclusive and cover many of the same aspects of user interface design. Quite often, usability problems that are discovered are categorized — often on a numeric scale — according to their estimated impact on user performance or acceptance. Often the heuristic evaluation is conducted in the context of use cases (typical user tasks), to provide feedback to the developers on the extent to which the interface is likely to be compatible with the intended users’ needs and preferences.
Walkthroughs
A User Experience (UX) Walkthrough is a technique used to identify usability and accessibility issues in a website or application. It is a procedure for examining a user interface following a set protocol and making assessments based on predetermined criteria. It emphasizes paired or collaborative evaluation of user interfaces by designers and non-designers alike, and serves to bring a diversity of perspectives to bear on the design process.
The UX Walkthrough technique is a synthesis of conventional methods (heuristic evaluation, cognitive walkthrough and code review) that enables the reviewer to make assessments both from the user’s point of view and that of the designer. The multifaceted nature of the UX Walkthrough enables the reviewer to make assessments across several dimensions, including: general design quality, task-oriented usability, assistive technology usability, accessibility standards compliance, and code quality. A UX Walkthrough produces a result that reveals usability and accessibility issues effectively and efficiently.
A UX Walkthrough can be performed by novices as well as experienced evaluators. The result is a comprehensive and multidimensional report of usability and accessibility issues.
User Tests
User Test is an evaluation method that requires the user in the evaluation process and at the time of evaluation must focus on the user. This method is usually used most often by companies that apply the Data Driven Company principle. There are several methods that use User Tests Method principles, some of them are Observations/Ethnographys and Usability tests/ Controlled Experiments.
Observations / Ethnography
Ethnography can help investigate very complicated or critical design challenges. A good researcher is essential when observing and interacting with target audiences in their real-life environment. Ethnographic research is a qualitative method where researchers observe and/or interact with a study’s participants in their real-life environment. Ethnography was popularised by anthropology, but is used across a wide range of social sciences. Within the field of usability, user-centred design and service design, ethnography is used to support a designer’s deeper understanding of the design problem including the relevant domain, audiences, processes, goals and contexts of use.
The aim of an ethnographic study within a usability project is to get ‘under the skin’ of a design problem and all its associated issues. It is hoped that by achieving this, a designer will be able to truly understand the problem and therefore design a far better solution.
Usability tests / Controlled Experiments
Usability testing is the most common technique to check how usable your interface is from the human-center design framework. The power of this qualitative technique is that it focus on what the user do and not what the user say.
Usability testing reduces the risk of building the wrong thing. It saves money, time and other precious resources. It finds problems when they are still easy and cheap to fix. When a designer is working on some assignment, he’s so close to the solution that it that you may not realize that something could be improved by testing the proposed solution with real users. The usability testing is an iterative process, it’s not one time blessing, you need to repeat the process until the design is not confusing anymore and your users are able to achieve the scenarios you propose.
So, Which is the most valid method?
Triangulation is the key and some will be more valid in certain scenarios e.g. where you have some designs you want to test then experiments might be good but if you are at an early stage then inspection methods or observations may be better. Whether you want to be theoretical I.e. see the effect of interfaces on users (in which case the psychological methods of controlled experiments will give you sound scientific data) or want to design a product where causal inference may not be so important Dependent on constraints (time/budget).
I think the text above is only an introduction. If you want to find out more, maybe we can discuss or someday I will make a complete article about the above. Btw, thanks to dozens of papers and journals maker that I have read in their journals and papers. But I cannot mention one by one. Also, thanksto my college lecturers who have made me read so much papers and journals and thanks to School of Computer Science University Birmingham which has allowed me to read and study.
