In our May Content Improvement Club session, we focused on how to run a usability test. We ran through the basics of putting a script together, watched a clip from a test and had a go at prioritising some issues.

Two problems for people working with content

We started this session by outlining two problems for people working with content.

We don’t see people using our websites

The first problem is that we don’t typically see people using the things we create. We build a website based on our best assumptions of what our users need and how we think they will interact with it. Then at some point in the future, someone uses the site. They might find it easy to use. They might find it difficult. But we don’t know, because we don’t get to see.

It’s hard to self-assess our own website

The second problem is that it’s hard to self-assess a website that we’re already familiar with. When we look at the site, we bring our understanding of the structure and the context it sits in. We know what the acronyms mean. We know where to find the contact form. We know where the links go.

This isn’t necessarily the case for a user coming to the site for the first time.

Here’s Steve Krug, author of Rocket Surgery Made Easy:

“If you’re building something, you’re not going to be able to see where it’s going to confuse people. It’s not going to confuse you. You know too much about it.”​

Steve Krug interviewed on the Brave UX podcast

This is sometimes known as the ‘curse of knowledge’. Erika Hall, author of Just Enough Research, explains:

Whenever we learn things, we forget what it’s like not to know those things. […] The more you know, the more you expect other people to know. And if you become a real expert in a topic, forget it.

Erika Hall writing in the Mule Design Studio newsletter: Don’t chicken out about talking to people

Usability testing is a way of addressing these problems. It gives us a chance to see what it’s like for someone visiting our site for the first time and it counteracts the curse of knowledge. That gives us valuable insight into where a design is working and where it isn’t.

A quick definition of usability

Before we go any further, a quick word about what we mean by ‘usability’.

In short, usability refers to how easy, effective and satisfying something is to use.​

The cover of Don Norman’s book ‘The Design of Everyday Things’ features a coffeepot with a spout and handle on the same side. This would score poorly in a usability test as it wouldn’t be easy, effective or satisfying to use.

ISO definition

There’s also an international standard (ISO 9241-11) definition of usability:​

“The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”​

ISO definition of usability

This is a more technical definition, but it’s worth bearing in mind. It emphasises the fact that we need to think about specified users and their goals. That will come up when we get to writing our script.

A simple usability test

The simplest usability test you can do takes about 5 minutes:

1. Grab a colleague / friend / family member who doesn’t know your site​.
2. Sit them at a computer and open the homepage of your site.
3. Set them a task. For example: “Imagine you’re a student and you want to get a replacement student card”.​
4. Sit quietly and watch as they try to complete the task.

This doesn’t take much effort, and sometimes you uncover a usability issue or two.

In Content Improvement Club, we looked at how to run a more extensive usability test. But we wanted attendees to know from the outset that usability testing doesn’t have to be a big complicated process. It can still be beneficial even if you do it on a small scale.

There are three roles in every test

In a usability test, there are three roles:

• A moderator, who reads the script, sets tasks for the participant and asks questions​.
• A participant​, who completes tasks on a website​, thinking out loud​ as they do so.
• An observer​, who watches and takes notes​.

There can be any number of observers, who watch the test live or on a recording.

We watched an example of a usability test

The best way to understand how a usability test works is to watch a recording of one.

Here’s Steve Krug demonstrating how he runs a test:

Usability Test Demo by Steve Krug (YouTube video, 24 minutes)

In Content Improvement Club, we watched a short clip of someone completing a task on a library website at a UK university.

This was the task:

You’re working on a presentation with four other people from your course. You need to find a study room in the library for the four of you this weekend. Can you find out if a suitable study room is available at the library?

We noted down the issues that we saw, and then we shared them on a Microsoft Whiteboard. Even though it was a single task, there were a lot of issues, which is fairly typical. That’s why it’s often helpful to follow your observations with a prioritisation exercise.

We prioritised the issues

In the session, we demonstrated how you can prioritise issues using a matrix like this:

To place things on the matrix, you make a quick assessment of how significant the issue is. Then you assess how easy it would be to fix. The issues you typically want to focus on first are those in the top right quadrant: major issues that are easy to fix.

It can be helpful to have a set of questions to establish whether an issue is major or minor.

These are adapted from Dave Travis’s work on Red Route usability testing:

• Does the problem occur in a task that is highly important to you or your users?
• Is the problem difficult for users to overcome?​
• Did multiple participants experience the same problem?

Red route usability: The key user journeys with your web site (Dave Travis / archive.org)

Using questions like these give you a shared set of criteria when assessing the significance of usability issues in a group.

But we’re getting ahead of ourselves. How did we get to this point?

How to write a script

Before you can run a series of tests, you need a script.

Use a template

We shared our usability testing script template, which draws on the work of Steve Krug:

Research brief and usability testing script (DOCX, 51KB)

The template includes:

• a brief, where you articulate the goals of the testing
• a lead-in script, which you read to participants before testing begins. This covers what the session will involve and sets expectations. ​
• a set of tasks, presented in a table with two columns. One column contains the task, and another the expected path to complete the task.​

Come up with tasks

We start by identifying tasks and then develop them by adding a scenario.

We practised this in the session. Attendees suggested tasks that someone might need to carry out on a university library website. For example:

• Check opening times
• Find a book on your reading list

Develop tasks by adding scenarios

Next, attendees developed these tasks by adding a scenario.

For example, “Check opening times” becomes:

• You’re a student and you want to check the opening times of the library during the winter break. How would you go about doing this?

“Find a book on your reading list” becomes:

• You’re a new student and you want to find the book “Campbell’s Biology”, which is on the reading list for your course. Can you show me how you would find out if the library has a copy of this book?

Tips for writing good tasks and scenarios

We shared some tips for turning tasks into questions​:

• Aim for 5 to 10 scenarios per session.​
• Keep tasks focused: one goal per task.​
• Frame tasks as realistic scenarios, not instructions.​
• Add light context to make it realistic.​

We recommend running a pilot usability test before going ahead with a round of testing. This helps you see how the full usability test flows from start to finish. ​It also gives you a chance to refine the script and ​ensures the session runs smoothly for participants on the day.​

Logistics

Ahead of the Content Improvement Club session, we asked attendees if they wanted us to cover any topics in particular. Most of these came under the wider topic of logistics.

When should you test in a design process?​

Test earlier rather than later. Testing earlier in a design process is ideal because it’s easier to make changes based on what you learn. If you’re creating something new, this usually involves creating rough drafts or prototypes, and then later, a high-fidelity version.

Changing a prototype is easy because no one is very attached to it. But when a design is a later stage of development, it tends to be more painful to learn that something isn’t working. By this point, you and your colleagues have usually invested a significant amount of time in an idea. If the fix involves changing something fundamental to the product, you might have to undo work that’s already been done.​

How long does a testing session take?

It varies, but we usually run testing sessions of about 30 minutes.

How many tasks do you set?

In 30 minutes, we can usually fit in between 5 and 10 tasks.

How many participants do you need?

Three to five participants is a good target. Jakob Nielsen argued that five participants is enough for one round of tests. This is because when you run the same tests with multiple people, you tend to see the same usability issues reoccurring. It’s a case of diminishing returns: by test number six, you aren’t usually learning anything new.

Why you only need to test with five users (Nielsen Norman Group)​

In Rocket Surgery Made Easy, Steve Krug recommends three participants per round of testing. This way you can run more rounds of tests – for example, by testing an initial idea and then a later iteration of the same design.

How do you recruit participants?

Mailing lists, Teams channels and surveys are great places to put call outs. We advise that you avoid revealing the testing topic or content in advance.​

Aim for participants who reflect real users where possible. This can prove difficult, so don’t let it stop you if you can’t find participants who don’t match the profile of your users.

How do you take notes?

Obviously, you can scribble notes anywhere you like. When we’re running a series of tests, we often take notes in a spreadsheet. This allows us to quickly spot which tasks caused problems for multiple participants.

Usability test results spreadsheet (XSLX, 36KB)

How can we make testing more accessible and inclusive?

Accessibility and inclusivity should be considered from the very start of the testing process. One important step is asking participants early on whether they use any assistive technology, so you can understand their setup and make any necessary arrangements. Our own introduction questions and templates include this for that reason. In the template, this is an introductory question, but if you’re recruiting via a survey, you could mention this there.​

​Having a diverse participant group will usually lead to more useful and representative findings. For example, if you’re testing a student-facing service, including both undergraduate and postgraduate students can help capture a wider range of experiences and needs. Again, this can sometimes be a challenge, but if possible something to aim for.​

Ideally, usability testing should include participants who regularly use assistive technologies. This provides valuable insight into accessibility barriers that might otherwise be missed. However, this can sometimes be challenging due to recruitment costs, specialist panels, or limited budgets.

Some resources that can help:

• Deque: Considering accessibility when designing a usability test ​
• Scottish Government Service Manual: Usability testing with disabled and non-disabled people​

Acting on the findings​

So you’ve run a round of testing. What happens next?

Involve senior colleagues in playbacks

In 2015, Caroline Jarrett wrote about a phenomenon she and Steve Krug had noticed when working on websites. They would run a set of tests on a website and uncover some usability problems. But then six months later, the problems were still there: no one had gone in and fixed them. To investigate why this was happening, they ran a survey of UX professionals.

Caroline Jarrett and Steve Krug’s analysis of why usability problems go unfixed

Out of 131 responses, the most common reason was that findings from usability testing conflicted with a decision maker’s opinion.

One way to address this is to get decision makers in the room (or on the Teams call) when you watch a highlights reel of test recordings. There really is no substitute for seeing user behaviour first hand. Reading about it in a report doesn’t carry the same weight.

Use the momentum created by testing

Testing creates a shared motivation to fix things​. Use this to your advantage. If you can, take action to fix usability problems while people’s memories are fresh​.

Make small changes first

Sometimes usability testing highlights small problems that are easily fixed. A link that doesn’t go where someone expects it to. An item missing from an A-Z.

Sort these things out first. Small wins like this give you the motivation to sort out the knottier problems.

If you want to find out more

This was a quick introduction to running your own tests. If you want to learn more,  Steve Krug’s introductory guide is a great place to start:

Rocket Surgery Made Easy by Steve Krug (listing on DiscoverEd)

We post about case studies of usability testing at the University on this blog:

• How usability testing guided user-centred improvements of the Digital Collections Platform (Emma Horrell)
• How usability testing helped the Library to improve their website (Nick Daniels)

The Prospective Student Web Team do the same:

• Usability testing the display of multiple entry years in the undergraduate degree finder (Pete Watson)
• Usability testing of new programme search and page navigation features (Nicole Tweedie)

Acknowledgements

Various points made in this blog post are taken from this 2015 post by Neil Allison:

Making usability testing agile

How to hear about future sessions

We promote these sessions via our mailing list. If you’re interested, please sign up:

Join the UX and Content Design mailing list (University login required)

Suggest a topic for a future session

We picked usability testing following a suggestion from the community. We’re keen to continue covering topics that colleagues across the University would find useful. It would be really helpful if you could let us know any ideas you have using this form:

Suggest a topic for Content Improvement Club (University login required)

Other training that we offer

More training is listed on the User Experience Service website:

Training | User Experience Service

What if the most broken user experience you deal with daily… is your refrigerator? This UX teardown applies product thinking to the humble fridge — exposing its dark patterns, terrible navigation, and hope-based interaction design.

The User Experience Service worked with colleagues from the Library to assist with their content development work. In this post, we’ll summarise what this collaboration involved and how it helped the Library to improve their site. 

Aims for this work

In Autumn 2025, the Library team got in touch with the UX Service to get help with a project to improve their website.

Library website

Working with the Library, we established that some user research would help us to know what needed to be improved. Through this research, we wanted to:

• find out if users can complete important tasks using the Library website by locating content
• understand if the Library website is easy for users to navigate
• identify any usability issues with the current Library website, which can then be prioritised and resolved
• understand users’ perceptions of the Library website and its purpose
• understand what role the Library website plays in users’ information-seeking behaviour

We agreed that running a short series of usability tests would help to identify where the existing site design was working well and where it could be improved. The testing would also provide an opportunity for staff to talk to students and learn about both their impressions of the site, and the role it plays in helping students to answer queries about library-related topics.

How a usability test works

In a usability test, you ask people to complete tasks on your site while you sit on the sidelines and watch. Participants are asked to share their thought process as they navigate through the site. Sometimes you find that participants can complete the tasks easily and without too much fuss. With other tasks you notice that participants encounter obstacles and have difficulty getting to an answer.

By testing like this with a small number of people, you get an indication of where users might be struggling with your site. This then helps you to identify which elements of your site design you might want to change.

We created tasks for test participants

In this work, the Library wanted to focus on the experience of students using their site. We started by making a list of the tasks that students need to complete. These included tasks like:

• check if a book is overdue
• request a book for an assignment I need to write
• find the opening times for the Main Library

We then worked with the Library team to develop these tasks into scenarios that we could include in a usability test.

Developed into a scenario, the tasks looked like this:

• You need to find a study room in the Main Library for four people to work on a presentation at the weekend. Where would you go to find out if a suitable study room is available at the Main Library?
• There’s a book chapter you need to read for an assignment, but there’s no e-book available, and the only print copy is at the University Collections Facility. One of your friends says that the Library can provide a scan of a book chapter, but you’re not sure how to request one. How would you find out how to request a scan?

With the test script finalised, the Library recruited six student interns working within Information Services to take part in the testing.

Running the tests

The UX team ran a pilot test session with the Library to try out the script and to demonstrate what’s involved with running a usability test. For the rest of the testing, Library staff ran the tests. This upskilled their team in this method of user research, which supports any future testing that they want to carry out on their site.

When running a usability test, Library staff needed to:

• read through the script
• make participants feel at ease
• ask questions where appropriate
• make notes of any usability problems that occurred

It was also important to hold back from helping participants to complete a task. In usability tests, we often find that we learn the most when a task doesn’t go smoothly. So although it can be difficult to watch, it’s useful to see where people get tripped up and work out what might be causing the problem.

The tests took place over Microsoft Teams, with members of the UX team sitting in on the call to make observations of any common themes.

What we found out

When the testing was complete, we all met up to compare notes.

Some of our observations included the following:

• The horizontal navigation menu was an important way for participants to get to the content they wanted. Getting the language right in this section was therefore critical to participants being able to find key content.
• The meaning of some section names such as ‘Academic Support Librarian’ were not sufficiently clear to participants, and this proved to be an obstacle in some tasks.
• Information about laptop loans was difficult to find, with some participants looking for answers in the wrong part of the site.

As a result of the testing, the Library made changes to their site design. These included changes to the menu.

In the tests, the menu looked like this:

After the testing, the Library renamed the ‘Academic Support Librarians’ section to ‘Study Support’. This also addressed an issue where participants were looking in the Research Support section for a task that really called for study support.

In addition to this, the order of the menu items was changed to better align them with the order of cards on the homepage.

The new menu looks like this:

Finding out about how to borrow a laptop was identified as a challenging task to complete. As a result, the Library created a new ‘Laptop loans’ link in the menu to make this service easier to find:

The Library have also noted several other areas of the site that they plan to address at a future date. In particular, the testing uncovered a need to clarify the relationship between location information used in DiscoverEd and corresponding information on the website.

Looking back on the work

We were delighted to hear that the Library team benefited from this collaboration.

Working with your team was such a positive and useful experience. I really appreciated that you were able to steer us through the whole process from writing the scenarios, to doing a test run and providing independent feedback from observing the testing. Not only has it made us look at our web content with new eyes, but we now have a group of experienced UX testers.

– Angela Laurins, Library Learning Services Manager.

The Library is a high-profile site at the University and it was a privilege to help them review their content. We enjoyed collaborating with the team and it was interesting to learn about how students are interacting with EdWeb 2.

Find out more about usability testing

If you would like to learn more about how usability testing can help to inform decisions about your site, get in touch.

Contact the User Experience Service

When the team behind the health and wellbeing website contacted the UX Service for help improving their student-facing content ahead of the new academic year, we were happy to oblige. Adopting a coaching approach, we guided them through usability testing to identify and prioritise content changes, to make it easier for students to find out about support.

The University student health and wellbeing site contains critical support content to help students while they learn with us. From this website, students can access important detail about how to engage with the services available to support their physical and mental health. Ahead of the start of semester one of the Academic Year 2025-2026, Essi Kauranen asked for UX support to make some improvements to the site, with a view to making it easier for students to find key pieces of content.

Getting a site map and checking analytics provided insights on site set-up and usage

Before thinking about how we could help the health and wellbeing team with specific improvements, it was important for the UX team to obtain some background knowledge of the health and wellbeing site, its content and its commonly viewed pages and sections. To learn about how the site was composed, we generated an XML site map and converted this to an Excel spreadsheet to provide a list of all the site URLs and page titles. This served as a useful way to visualise all the pages in the site, and to see the different topics covered by the pages. To gauge which pages received most interactions, we used a Looker studio dashboard to cover the period of a year which showed that:

• The homepage received the most webpage views
• The wellbeing services page and the student counselling pages received high numbers of website clicks and webpage views compare to other pages
• Pages about period products, mental health and crisis support were also well-visited

As well as helping the UX team to familiarise with the site, both of these datasets were useful to share with the health and wellbeing team, to give them a fresh perspective on their site and its content.

We established students’ main reasons for visiting the site, and used this to shape usability tasks

In a short call with Essi, who looks after the pages on the health and wellbeing site, we discussed how the site is currently used by students to access the support services available. The analytics figures on the most-visited pages were an indicator of the most sought-after content, however, insights from staff involved with promoting and delivering the support services gave a clearer steer on the associated demand and use of the services available. Logs of enquiries were another useful source of data to highlight which services students wanted and needed to access, and those they had difficulty accessing through the website (which in some cases had prompted them to send direct emails).

These discussions helped tease out typical circumstances which prompt students to access the health and wellbeing site, and the associated tasks they are looking to perform. Furthermore, it helped establish the site’s priority content – in other words the information within the site that was most important for students to be able to access easily.

The scenario-led tasks were as follows:

1. You’re a current student at the University. It’s a Friday and you want to find out if the Health & Wellbeing Centre is open tomorrow. Where would you go to do this?
2. You need to find a counsellor to speak to and you want to refer yourself. Can you find where/how you would do this?
3. Imagine that you are just starting at the University and have just moved to Edinburgh. You want to find out how to register with a GP practice in case you fall ill. Where would you go?
4. Can you identify where in the King’s Buildings you can find period products?
5. You are struggling with your finances and you want to see how you can access support. Can you use this website to help you find this sort of information?
6. You’re concerned about your flatmate, who has appeared sad and withdrawn lately. You want to find out how you can support them. Can you show me where’d you’d go on the site?
7. You think someone in your course is being bullied and you want to let someone at the University to know about it. Ideally you would like to do this anonymously if possible. Can you identify where on the website you would go to do this?
8. You feel your mental health is declining and you need to speak to someone urgently about your wellbeing. It’s late Tuesday evening. Can you identify where you could find someone to talk to?

Using a template, we worked together to prepare a usability script to use in tests

With tasks and associated scenarios confirmed, this information could be used to draft a usability script. A usability script is an important tool for conducting usability tests to help ensure that test facilitators follow a protocol that both keeps participants informed and gathers useful data. When watching people interact with technology (as the role of a usability test facilitator requires) it can be easy to forget to explain the test set-up, react to participants’ actions, become distracted or veer in various directions, therefore having a script helps to keep on track, and also helps to provide the facilitator with a level of control to counter feelings of nervousness about conducting the tests.

The UX Service supplied a template script which was adapted to fit the scenario and the tasks. Considering the sensitive nature of some of the tasks, we also added in scripted sentences to advise participants about these tasks in advance, to give them the option to skip them if they wished.

We coached the health and wellbeing team in usability testing

In preparation for running the tests, Essi  the communications manager behind the health and wellbeing website, set about recruiting participants to represent the student audience of the site. Essi was also keen to run the tests herself with support from the UX team, so we arranged a practice run of the test, using the prepared script and with a willing participant. This enabled final tweaks to be made to the script and the tasks before the usability tests with real participants took place.

The test results prompted content improvements to the site

Responding to what was observed in the tests, several changes were made to the health and wellbeing site, focused on making it easier for students to find support content.

The opening hours of the Health & Wellbeing Centre were made clearer

In response to the results from task 1, some small tweaks were made to the layout of the content on the page about the Health & Wellbeing Centre. In particular it was changed to include three lines outlining the opening times of the wellbeing area of the centre and the Bristo Square Pharmacy.

The number of steps to reach the student counselling service was reduced

The results of task 2 highlighted that there was a need to click through an interim page about the Student Counselling Service before reaching the self-referral form (on a card on the Student Counselling site homepage). Following the tests, this page was removed, making the journey from the health and wellbeing homepage to the Student Counselling site shorter.

 

Surplus content was removed from the period products pages

Finding out about the locations of period products in task 4 was impacted by the content on the Health services and Access to free period products pages. As a result of the testing, changes were made to make the key information about where to find the products more prominent by removing content which distracted from the main purposes of these pages.

Content about wellbeing services was streamlined

Following the results of task 5, the content on the wellbeing services page was reviewed and content that did not support students finding information about these services was removed, and several design changes were made to the page.

A small amount of usability testing goes a long way to improve website content

Often when teams across the University contact the UX Service for help, they have reservations about the time commitment and the effort required to guide user-centred content changes. In this piece of work with Essi from the health and wellbeing team we demonstrated a high return-on-investment for the UX work. In other words, for a relatively low outlay of effort and time spent planning, running and analysing the results of usability tests, content was markedly improved to suit the expectations and mental models of students using the site to access important support content.

If you would like help improving your website content for your audiences, feel free to contact the University UX Service.