Overview of Ultra-detailed Usability Testing Methods

I have summarized a version of the flow chart of interaction design before, mainly to sort out the process for my future design, but there are only branches and leaves in that process, so I will write a method summary or guidance for each method in order to prepare for future practice. What I expect is that if a person who has not been exposed to this method sees this summary, he can complete the experiment step by step according to this summary. This is my biggest purpose. The following is the first summary "Summary of Usability Test Methods".

(Warning: Be careful when writing long articles! However, after reading it patiently, you will certainly gain something. )

= = = = = = = dividing line = = = = = = = = = = = = = = = = =

The process of usability testing mainly includes seven steps: thinking before testing, making test prototype, writing test script, recruiting testers, setting test environment, predictor, formal testing and statistical analysis of test results. Some of these seven steps can be performed in parallel, while others need to be performed in strict order. The flow chart consisting of seven steps is as follows:

Let me talk about how to do these seven steps.

No matter which platform is tested for usability, such as PC, mobile and WEB, the most important thing is to clarify some basic issues first. The basic problem is the classic 5W problem:

Why do you want to do this test (why)? Testing can verify some doubts in the design, or find problems in the existing interface and process design, and analyze the specific problems.

When and where to do the test (when? Where is it? )? Time generally needs to be coordinated with testers; The general location can be chosen in a quiet conference room, and it would be best if the company had a special laboratory.

Who will be the tester (WHO)? We can discuss it in detail when we are in recruitment tests, but testers are generally close to our personas, or in other words, testers are generally our target users.

What are we going to test? Testing some function points, testing interface design, testing process design and testing design are controversial and questionable.

Of course, these questions are not too difficult, but they are all critical. Without this step of thinking, the whole usability test will be like a headless fly without a general guide.

After thinking about the above problems, we need to do some preparations for usability testing. The main work includes: ① recruiting testers; ? (2) write a test script; ? ③ Make a test prototype.

These three processes are in no particular order, and can be carried out at the same time when conditions permit (sufficient manpower and material resources).

Recruiting testers is one of the most important links in usability testing. Whether the tester is suitable or not is directly related to the test results, and the test results are directly related to whether the problems existing in the product can be found. Therefore, recruitment tests personnel are the top priority. The ideal tester is our target user, so usability testing should try to find the target user as a tester. The way to find it is as follows:

A) The easiest way is to choose colleagues or friends as testers, if they are also the target users.

B) Secondly, large companies will have their own user database, from which testers can be found.

C) In other words, it is allowed to entrust a third-party agency to help find testers, but the effect may not be as good as that you find.

D) Of course, applications now generally have their own Weibo, WeChat, official website or forums, which are very good channels for finding testers. We can push the announcement of recruiting testers, let users fill out a survey, and then we can filter out the dead people we want. The announcement should indicate the reward, usually a small gift, to ensure that it is attractive to testers, and at the same time, it will not let them falsify personal information for this gift. Secondly, for testers, we need to make a screening 3. First of all, users need to fill in the necessary personal information, such as name, telephone number (email address), free time, etc. Then select some other personal information according to the survey: gender, age, occupation, and finally leave a few questionnaire questions for screening.

The dimensions of filtering mainly include:

Platform. If the product under test is platform-related, such as Android or iOS, you need to make a screening here.

Familiarity with the product. For example, if we want to find some beginners and some advanced users, we can choose "use time" to measure users' familiarity with the products.

The quality of test scripts is directly related to the quality of results. Before writing the test script, we need to determine some dimensions of the result analysis. The general dimensions are: a) degree of task completion b) fatal errors c) non-fatal errors d) time to complete tasks e) subjective emotions f) preferences and suggestions. For the explanation of these dimensions, please refer to the last part of the first article, "Statistical Analysis of Test Results".

Because the dimension of analysis will involve the problem of script, after the dimension of analysis is determined, the task of function point can be analyzed. List all the function points that need to be tested and design tasks for each function point. For tasks, users have two subjective feelings: interface and process. So the test script can be subdivided from these two dimensions.

It should be noted that in usability testing, questioning is only a part, and observation is another important content, so the test script should not only have questions to ask, but also write some points for the staff to observe. At the same time, you can write the summary outline while writing the test script, which is convenient for the unified display of the results in the later summary.

In particular, the points that are in doubt or controversy during design can also be well verified in usability testing.

After writing the test script, you can discuss it with stakeholders (project manager, product manager, developer, etc.). ) and ask them to check the test script.

Interface:

A) What is the current interface?

B) What do users think of everything?

C) Is it operable?

D) What gestures are used?

E) What will happen after the operation?

F) Is the content displayed in the interface sufficient? Did you lose anything?

Process: Process testing is task-based. List the requirement documents of the product, and then match each requirement with the appropriate scenario. Of course, there will be a scenario that covers multiple requirements, which is also allowed. Then let the user perform tasks in the scene, observe the user, and then ask the user at any time, ready to answer the user's questions.

The above two points are applicable to all usability tests, but for the usability test of version update class, we still need to know how the update is accepted by users, so we need to add some comparative questions, such as the fluency of the old and new versions and the contrast of interface expression.

Finally, it should be noted that the coverage of usability testing is limited, so it is necessary to limit the number of script problems and prioritize them.

For example, I have done a crowdfunding platform on WeChat before. I can set the following tasks:

The prototype of usability testing is generally high-fidelity Demo, which can be made of proto, Flinto, Proto, ink knife and so on. To make the final effect of the application as real as possible. Making high-fidelity Demo is a time-consuming and labor-intensive work, so some dynamic effects, interfaces, etc. Can be appropriately ignored when making. However, Demo can also be used as a reference for development in the end, so the hard work is worth it. Even, you can ask developers to demonstrate native programs (for Android platform), and the program demonstration experience will be better.

Of course, paper model is another very good tool. The paper model needs to be only a paper model, and then all the pop-up windows, drop-down menus and other controls should be made. Then the designer acted as the Wizard of Oz, helping the user to complete the task. That is, the user operates on the paper model, and then the designer feeds back the user's operation in real time. This requires designers to be very familiar with the application of testing, and the testing time will be greatly increased. At the same time, the dynamic effect as a part of the design cannot be displayed here, so the effect may not be as good as the high-fidelity demonstration. In short, each has its own advantages and disadvantages, which should be considered according to the actual situation.

Test environment refers to the recording equipment that needs to be used in the test process. By recording the test process, we can better analyze the user's behavior, especially some things that users are not aware of.

First of all, the most important thing is recording. On the one hand, recording can help designers to recall the interview scenes when sorting out the interview records, and then make up some missing notes. On the other hand, audio recordings can also be used as archival materials. At the same time, the recording is simple, easy to operate and hidden. Recording can be done with a recording pen or a smart machine that can be seen everywhere now. Therefore, it is strongly recommended to record at least during usability testing.

Besides recording, there are also videos. If there is video recording, the recording step can be omitted. Video mainly records users' expressions and actions. Sometimes, users' expressions and actions can convey many things. By recording this information, designers can occasionally dig out some flashing design points.

In addition, the user's screen recording is also a way. Through the user's screen, plus the user's actions and expressions, the user's usage scene can be truly restored, which is convenient for later analysis.

Video recording and screen recording are difficult. The main equipment can be referred to as follows: 5. For details, please check the related links:

Camera: record actions and some expressions.

Eye tracker: it can track the focus trajectory of the eye, and is not suitable for mobile terminals.

Mouse track record: record mouse track, only applicable to PC.

QuickTime？ (iOS): Record only the screen.

Mobizen? (Android): Recording screens and gestures

Show? Tape recorder? (iOS): Gestures, sounds

SCR？ (Android): Record the screen, gestures, expressions and sounds.

Magitest？ (iOS): Record screens, gestures, expressions and sounds.

Mobizen? +AirDroid？ (Android): Observe and record gestures, expressions and sounds on the spot.

Pre-test is a simulation before the formal implementation of usability test. Simulation helps to find problems, just invite colleagues. Go through formal testing process, including equipment debugging, interview, questioning, recording, etc. And then put the recorded audio and video out to see how the effect is, and then make adjustments if the effect is not satisfactory.

In short, pre-test can help to find problems, including the following aspects:

Equipment problems. For example, the location of recording equipment will affect the recording effect.

Problems with test scripts. Test whether the question is clear enough.

Cut into the interview and ask questions.

The recorder's record.

Find the problem and then solve it, so that the formal test can achieve better results.

The reception before the test is the tester's first impression of the company, leaving a good impression and good mood for the tester, which is conducive to usability testing. So here are some precautions.

First of all, you can confirm the user's itinerary in advance. In case of windy, rainy, snowy and other bad weather, you can send greeting messages in advance.

Secondly, when users are late, we should also exercise restraint. Call the user five to ten minutes late to ask about the situation. If the user cancels the test for any reason, he should also keep a friendly attitude.

After receiving the user, send a cup of warm water or warm drink, and then ask the user to wait. Finally, a special person can chat with the user first and ask something.

There is a warm-up introduction before the official start. First, the host introduces himself, and then introduces the purpose and time of the test. It is necessary to emphasize to users that the test object is the system, and we hope users can speak freely. If there are audio or video recordings, users need to be informed of such behaviors, but the results are completely confidential. Finally, you need to sign a confidentiality agreement.

The formal problem is divided into two parts: the small problem of personal information and the problem of usability testing task.

The small problem is mainly to let users have an adaptation process and get into the state quickly. Generally, you can ask about product habits, product preferences, internet access, etc. Then the test questions are the main usability questions. Here we need to put the problem into the scene and let the user complete the task in the scene. Or you can ask the user's usage habits and then guide to the questions in the script. It should be noted that you don't have to ask questions in the order of the script, you can improvise, so the host should be very familiar with the content of the script. In addition to asking and listening, the host should also observe the user's expressions and actions. When you meet the user's puzzled expression, you can insert new questions appropriately, but try not to help, nor point out the user's mistakes or accuse the user of being too slow, but you can ask the user "why did you do this" and choose to stop the task if necessary.

There needs to be a recorder during the test. The recorder needs to record what actions and steps the user has done (key points), what the user has said, and write down his own questions (you can ask questions or ask the host to ask questions when appropriate).

After the test, the host can ask the user's thoughts and ask the recorder to add questions. After all the questions are over, you need to thank the users. Send gifts and accept some transportation expense reimbursement bills from users. Finally, the user should be sent to the door of the company.

After the exam, if you have time, you can sort it out immediately, because the shorter the time, the richer the content. If necessary, it can be supplemented by audio or video recording. There is also a summary outline when writing the test script, and the content is arranged according to the outline. The outline should be flexible and can record the new problems found in the test center.

Remember, it's just sorting, and there will be a sorted data at the end of each test. Finally, it is necessary to summarize several usability test summaries, and finally issue a usability test result, and make corresponding improvement work according to this result.

We can analyze our usability test 8 from the following dimensions (there may be cross between dimensions):

A) the degree of task completion. Each test task corresponds to a goal, and only when users reach the goal can they be considered to have completed the task. How well do users complete each task? How many users did not finish the task in the end? How many users do you need to complete the task at the prompt of the host? How many people can finish the task by themselves? These are very important indicators.

B) fatal mistake. Critical errors are those that prevent users from completing tasks. These mistakes are very important and should be paid enough attention to.

C) non-fatal error. Non-fatal errors refer to errors that hinder, pause or think in some places when users can complete tasks. These mistakes are relatively unimportant, but if they occur frequently, they need to be taken seriously.

D) time to complete the task. How long each task needs to be completed determines whether the interaction design process and interface design are friendly enough.

E) subjective emotions. Users' subjective feelings about the task, such as whether the task is simple enough and whether it is easy to find information, can be measured by users.

F) preferences and suggestions. Can users tell which parts of the product they like very much? What don't you like? Or ask them for advice.

1 Adjust your mobile phone usability testing practice/51194-79597.aspx.

4 summary of user interviews-Tencent CDC/? p=5690

5 Mobile usability testing (3): Field testing–Tencent isux–social user experience design/mobile–usability–testing–-three.html

6 user research experience-learning from copper mines-blog bus /c406 1443/

7 Simple and Quick Usability Test | Netease User Experience Design Center /4 15 1.html

8 Plan Usability test Usability? http://www . usability . gov/how-to-and-tools/methods/planning-usability-testing . html