Normal view
Some of the World’s “Best” Websites Are Actually Terrible
-
Website and Communications Blog
- Just do it for me? Balancing human and AI input in content improvement
Just do it for me? Balancing human and AI input in content improvement
Working with student interns and contributing to the Drupal community, I’ve experimented with AI in various tools designed to support content improvement. I reflect on what I’ve learned about peoples’ preferences for engaging with AI tools and consider wider design challenges for web content to support co-browsing.
Writing good quality, user-centred content has always been a challenge, and accordingly, a suite of tools evolved to address this; Hemingway, Grammarly and many more. With their ability to learn patterns and predict word sequences, the advent of LLMs provided a new direction for content improvement. Where previous tools had used fixed rules, algorithms and formulas to flag content problems and score sentences, LLMs changed the playing field as they were able to be prompted to rewrite and create text and to adapt words and sentences not only in line with fixed rules, but also aligned to different interpretations of style.
The UX Service’s content design course, Effective Digital Content, and our ongoing Content Improvement Club meet-ups are both designed to help staff publishing digital content for the University. It’s been encouraging to see a positive uptake for both training formats, however, we realise we can do more to help staff publish the best quality content they can, and to this end, for the past few years it’s been a privilege to support a series of interns in the UX Service to look at how we can innovate and apply AI to the persistent challenge of writing good quality, user-centred content.
Thinking beyond – embracing the spirit (or grasping the nettle, depending on how you look at it) of AI innovation, it’s sensible to note that, more broadly, the browsing landscape of the web is increasingly shifting to include AI agents as well as humans, giving cause to consider what using AI to design content to be consumed by AI might look like. That’s the subject of another blog post (or several).
For now, as I begin another phase of work with a new student in the role of AI and UX Innovation intern (Viktor Sharha), it felt timely to pull together what I’ve learned so far.
Designing, creating and editing content happens in various ways
When designing any user-centred assistive features, a good place to start is the current workflow or user journey people adopt to achieve the task in question. In other words, to look at the stages content editors currently go through to make sure their content is as good as it possibly can be.
One of my earliest research tasks when I joined the University was to usability test the Editorial Style Guide, to find out how publishers used it to keep their content consistent in EdWeb. This work gave me insight into the type of workflows publishers followed to get compliant content published. Some publishers worked in Microsoft Word, using the in-built spelling checker for typos, and cross-checking against a printed-out copy of the style guide for University-specific errors before pasting content into the EdWeb body text. Others went straight to EdWeb, checking their content line-by-line with a PDF version of the guide. Others relied on colleagues giving their work the once-over before pressing publish. Some used Grammarly and Hemingway, some didn’t.
Overall, there wasn’t a dominant content-publishing workflow, with unified friction points waiting to be alleviated through a standard user-centred design process. Instead, there were varied approaches, habits and ways of working which people had adapted over time to suit their specific circumstances and to address the areas they found difficult as individuals. Tools were used at different stages of the process, and to a greater or lesser extent depending on personal preferences.
Therefore, designing content improvement tools isn’t one-size-fits-all
It follows that, since people follow different pathways to create, design and edit content, tools to assist this process need to be developed in different ways to suit varied preferences. Since 2025, interns in the UX team have prototyped our own different content improvement tools to test with publishers, and we’ve also tested open-source tools that exist already, developed within the Drupal community. From our testing and research, we found pros, cons and nuances associated with different tools. The summary below pulls together detail of the tools and how they compared.
Tool 1: Drupal Editoria11y – no AI, flags errors in the editing interface
Editoria11y is a Drupal module built by John Jameson from Princeton University that automatically checks content inside the editorial interface. It was originally designed to catch accessibility issues, but it can be configured to check content against other rules too, such as institutional style guide conventions.
What it does and how it works: Editoria11y checks content as authors type, and also on previews and published pages. It includes an indicator bar to show how many issues are on the page and has inline markers to show where each issue is, as well as a dashboard summarising site-wide errors. When an issue is opened, a pop-up modal explains the problem and suggests a fix.
What worked well: In tests, publishers were able to make sense of the inline markers and understand how these related to the indicator bar for the whole page. They liked the tracked change visualisation in the pop-up modals, and were able to work through successive modals, accepting or ignoring the suggested changes.
What needed improvement: Some publishers found the number of markers overwhelming, and since the pop-ups covered the text, it was difficult to judge each recommendation in context. In the pop-ups, some of the rule descriptions used formal grammatical terms (such as ‘ordinal suffixes’) which were unfamiliar to some publishers meaning they were confused about the problem to be fixed. Since rules were applied mechanically, there were instances where errors were missed, or where awkward fixes were suggested (such as swapping ‘1st floor’ for ‘1 floor’ due to straight removal of the ordinal suffix) – these degraded publishers’ trust in the tool.
Read more about what we found testing Drupal Editoria11y
Tool 2: ELM-powered content design chatbot – with conversational advice
This tool was developed by student Mostafa Ebid during a 2025 summer internship. It connected ELM, the University’s AI platform, to EdWeb2, so publishers could get help with their content by conversing with the University’s suite of LLMs without leaving the Drupal editing interface.
What it does and how it works: This tool sits in a chat sidebar next to the main Drupal editorial interface and offers three modes: Write Content, Design Content and Proofread. In response to a user prompt to improve content, it activates an orchestration of agents (each with specific tasks such as checking the style guide, structuring pages, profiling audiences) which read the page content and act on it accordingly, publishing the revised content in the chatbot window.
What worked well: In tests, publishers understood how the tool could be used as a writing partner, and were impressed with the way it restructured content with headings, bullet points and calls to action and hyperlinks.
What needed improvement: When faced with the empty chatbot interface, some publishers weren’t sure where to start, and they didn’t notice the three modes to work on content. Some were also unsure of the sources the chatbot was using for its revised content and skeptical that the revised content contained hallucinogenic content. They would have appreciated tracked changes to be able to pinpoint what had changed and found the long thin sidebar chat interface awkward to review the updated text. The chatbot was also limited to reviewing body text content, whereas publishers needed it to review content in other page components used in Drupal.
Read more about what we found developing and testing the ELM-powered content design chatbot:
Initial insights from UX testing our Drupal AI content assistant tool
Tool 3: QuillMark v1 and v2 – some AI, clusters errors with correction suggestions
QuillMark was an AI-enhanced tool which worked in Drupal to check content against the University’s Editorial Style Guide. Shlok Gupta built and tested two iterations of Quillmark during his 2026 summer placement as AI and UX Innovation Intern.
What it does and how it works: QuillMark sits in a side panel of the Drupal editorial interface and scans content on the page in two stages. The first stage uses code-based pattern matching (without AI) to catch straightforward errors. The second stage brings in AI to find deeper issues (such as passive instead of active voice) and to suggest fixes. Findings from both stages are displayed in the side panel – each finding describes the problem, links to the relevant rule in the style guide and offers an ‘Apply fix’ button (which can be reverted). Findings are grouped by category, and this panel also includes an indicator of the environmental impact and cost of the scan.
What worked well: In tests, publishers understood that the tool was there to help them correct content on the page, they liked the environmental and cost indicator, the suggested fixes and the links to the style guide reassured them that the tool was applying University-specific rules.
What needed improvement: Publishers reported feeling overwhelmed by the number of findings on a single page and they wanted a way to deal with related findings and fixes in bulk, to avoid going through them one-by-one. Related to this, they wanted
Read more about what we found developing and testing the QuillMark tool:
QuillMark, iterated. How user research helped us evolve an AI-enhanced Drupal style guide assistant
Tool 4: Drupal AI Content Review – full AI, with a plan provided to improve content
As part of the Drupal AI Initiative, I’ve contributed to shaping the self-explanatory Drupal AI Content Review module. This module plugs into a Drupal site, references content rules and good practice from context items in Drupal Context Control Center (within the same site) and uses the configured AI provider to review site content
What it does and how it works: The feature reads the page content and checks it against criteria set by the site owner, such as tone and voice guidelines, inclusive language rules and target readability levels. These rules are stored in the Drupal Context Control Center, along with examples of good and bad content. The review gives the page a score for each criterion and recommends fixes to improve it. Accepted recommendations become an improvement plan, which is applied in the editorial interface. Re-reviewing the page shows the updated scores, with the option to keep making improvements. Over time, the rules can be refined by adding more examples of good and bad content to the Context Control Center.
Read more about Drupal AI Content Review on Drupal.org
Read more about the Drupal Context Control Center on Drupal.org
What worked well: In tests of this module within the AI demo, people liked seeing the red, amber and green indicators against page content for an easy visual highlighting areas to be worked on, and they liked being able to select a checklist of recommended improvements to apply to improve their content.
What needs improvement: The screens explaining the recommended improvements were difficult to scan read, and it was also difficult to compare the original text and the text with the recommended improvements side-by-side (despite colour shading to denote which was which). The stage where the recommendations were turned into an improvement plan felt like an unnecessary extra step, it would have been better to have implemented the recommendations directly. Also, it was presumed that implementing the plan would have corrected all errors resulting in a score of 100%, without the need for further work and re-review. Finally, it wasn’t immediately clear how to use the Drupal Context Control Center and the AI Content Review together.
When designing AI-enhanced tools, consider three AI design maxims
Taking on board variations in blending human effort and automated operations, there are three approaches to designing AI features and AI-assisted workflows worth considering. These are defined below, with ideas for how they may apply to content improvement.
1. Human-in-the-loop
In this set-up, humans typically share the task-load with AI – actively working alongside an automated operator to supervise or influence actions, and to approve AI decisions before they happen (ensuring AI outputs are not released without the green-light from a person).
Human-in-the-loop tends to be adopted when:
- there is a need for human judgement to get something correct
- the consequences of an error could be high-profile or damaging
- trust in an automated tool is still being built, and the boundaries of its capabilities are still being realised
- the goal is for the system to facilitate shared learning between the person and the automation
For content improvement, human-in-the-loop is appropriate in these scenarios:
- ensuring correctness of important factual data – for example, prices, dates, contact details, terms and conditions
- checking the tone of sentences in line with inclusive language guidance
- verifying accuracy of high-stakes published content such as legal information, policy wording or written instructional content
- rewriting complex topics in plainer language
- writing meta descriptions or page summaries to be accessed by search mechanisms
- translating content into other languages
2. Human-on-the-loop
These systems involve the AI or automator acting on its own, with the human reviewing and spot-checking at intervals, monitoring output quality over time, and intervening to correct failures periodically, where required.
Human-on-the-loop tends to be adopted when:
- there is a high volume of low-risk tasks to be completed, which would be tedious for humans to do themselves
- consequences of errors are minimal
- there’s a need for tasks to be completed quickly, and including human-input at each task would be too time-consuming
- consistency is required across tasks, and there’s a risk of human error introducing variability
For content improvement, human-on-the-loop is appropriate in these scenarios:
- fixing straightforward style rule violations such as correcting spelling errors
- categorising and tagging text content (for example, to facilitate its discovery in search)
- scanning content for readability and accessibility, to flag or report areas to be addressed
3. Human-out-of-the-loop
In this arrangement, the human sets up the AI to act autonomously, and leaves it to proceed, with no routine review.
Human-out-of-the-loop tends to be adopted when:
- there are very low-risk, behind-the-scenes tasks to be completed where errors have little or no consequence and can be fully reversed
For content improvement, human-out-of-the-loop is appropriate in these scenarios:
- autocorrecting typing errors
- detecting content duplication
- auto-optimising media content to defined parameters
Choice, pacing and transparency are constant design requirements
Testing four different tools has showed that options always need to be available, whatever balance of human and AI the tool includes. People approach AI differently based on preferences and past experiences. Some love nothing more than a good grappling of a sticky sentence, some just want an agent to do the work for them, and many sit somewhere in between, happy to have some AI help as long as they have the final say.
Alongside humans in, on and out of the loop, it’s important to remember the fourth option: human only, where people can choose to do their content unaided, using the methods they relied on before AI came along. AI shouldn’t be forced on anyone, so all tools need to come with a ‘no thanks to AI’ option.
AI is a fast-evolving and non-deterministic technology, so at any point in time it may work differently to the way it did before. Responding to challenge from a design perspective, it’s important to factor in pacing. People need to be taken through how a tool works, and the options available. They need time and space to experiment and try it out, so they can learn for themselves how it can work for them. Pacing not only matters in the way tools are introduced, but also in the way their outputs are presented, to avoid the overwhelm that comes when AI does the heavy lifting all at once, and so people know what changed and how, and they have a way to accept, decline or edit what it produced.
If people don’t trust an AI tool, they won’t use it. Earning trust requires transparency, explainability, relatability and reliability – so AI tools need to be upfront about what they can and can’t do, with clear, intuitive interfaces and labels. Reliability will always be hard to promise given AI is probabilistic, so instead of accuracy guarantees which may fall short, a better approach is to design for experimentation, so people can find their feet and make their own judgements.
Designing for humans and AI together – metadata, markdown and MCPs
Looking at the broader digital landscape, the challenge widens from designing tools that help humans make content to designing websites with content ready for both humans and agents to use.
In co-browsing, people are increasingly arriving at a website with an agent alongside them, handing off some tasks while keeping control of others and wanting to see what their agent is doing. That means firstly deciding which tasks belong to humans and which one to AI, and secondly designing for each. Humans need intuitive interfaces. Agents need data they can read and act on without ambiguity:
- metadata, such as schema markup that tells an agent what a page is about, who it’s for and how it connects to other content
- markdown versions of pages and llms.txt files, which give agents clean, structured text without layout and navigation getting in the way
- MCPs (Model Context Protocol servers), which let sites expose what they can do as well as what they say, from simple actions declared directly in HTML to more complex ones written in JavaScript – in preparation for use by the emerging WebMCP browser standard.
Labels matter for humans and agents alike, because words are becoming a currency that machines read as well as people. Alignment on terminology facilitates the creation of a single source of truth, to prevent the human experience from drifting from the agentic experience. On another level, humans need a way to consistently communicate with AI – to give AI the rules and instruct it and to train it with new insight. ‘Rosetta stone’ mechanisms like the Drupal Context Control Center will play an important part in designing for human and AI interaction and collaboration and I am excited to keep working and researching in this area to address these evolving design challenges.
The “Rage Click” Manifesto: Is Your Design Making People Angry?
-
Website and Communications Blog
- Testing with users informed the Institute for Academic Development’s content migration
Testing with users informed the Institute for Academic Development’s content migration
The User Experience Service worked with colleagues from the Institute for Academic Development to test their content with users. Results from the testing supported content decisions in migration work that these colleagues carried out later in the summer.
This is part two of a two-part series. Part one is available here:
How we helped the Institute for Academic Development to establish user needs for their website
We had established some user needs but wanted to learn more
In two online sessions, we had worked with Institute for Academic Development (IAD) staff to establish a shared understanding of what users needed from their service.
We now wanted to find out about the experience of students who use the IAD website. Specifically, we were interested in two audience groups:
- Postgraduate research (PGR) students
- Students from all stages who might need academic skills support
Along with tests on the existing site, we also wanted to try out some new layout ideas and learn about what types of content these audiences would find most useful.
An in-person event gave us an opportunity to hear from PGR students
We knew that presenting information about workshops would be a priority for the new sites and we wanted to know the best way to present this. To inform the design, we needed to speak to people who might use the website to look for these workshops.
Fortunately, the perfect opportunity was just coming up. On 17 June, a large group of research students would be attending PhD Horizons, an all-day careers event at McEwan Hall. We planned two simple research tasks that we could carry out on the day. Emma Horrell and I attended the event with UX team interns Hannah Watson and Shlok Gupta. This was a great opportunity for Hannah and Shlok to gain hands-on experience of in-person UX research.
Usability testing helped us to learn about the existing site
The first task was a quickfire usability test asking PhD students to use the IAD website. For the testing, we handed PhD students an iPad and presented them with two scenarios:
- Imagine your viva is coming up. How would you find a workshop to help you prepare for this?
- You’re going to a conference and want to get better at presenting. What can you find on this site that could help you to do this?
Paper prototyping gave us insight into alternative designs
For the second task, we presented four paper prototypes of potential homepages for postgraduate researchers. PhD students compared the prototypes and talked about which areas of the site they would find most useful. This gave us an indication of how closely these homepages aligned with their needs, and it also provided insight into how the language used in the prototypes might be understood.
We used paper prototypes of homepages to test ideas with students.
Students used sticky dots to indicate which areas of the paper prototype would be most useful to them.
What we learned from the usability testing and paper prototyping
In the end, we heard from about 20 PhD students who told us how they might search for workshops on the IAD website. From these conversations, we learned how these students responded to items presented on the live site and what they thought of some potential alternative ideas.
In the usability testing activity, we learned which ways of presenting workshops were perceived as being more useful than others. For example, most participants did not use the embedded PDF brochure, despite it being prominently located on the PGR landing page. Instead, we saw users gravitate to webpages showing workshops grouped by theme, or as an A to Z list.
Users did not interact with the embedded PDF brochure.
Observations like this gave us a steer on which categories and methods of presenting workshops would be most useful to PGR students in future designs of this content.
In the homepage paper prototype activity, one of our mock-ups included an option to complete a Development Needs Analysis. This is a form that new PGR students can complete to assess their learning needs. But when we presented the text ‘Development Needs Analysis’ in our prototype homepages, most students could not guess what this was. That seemed like something that could be addressed in the new website.
We shared the findings with colleagues tasked with developing the new site for postgraduate researchers.
We ran usability tests on the academic skills content
Alongside this work, we were helping IAD staff to develop content for a new academic skills website. We agreed that a short round of usability testing on the existing IAD site would help us to get a better understanding of what it’s like to use it. We wrote a script for a 15-minute test and recruited four students working on summer internships at the University. A week later, we were on a Teams call with those students, watching them use the site.
The tasks we set in the usability tests were as follows:
- You’re an undergraduate student at the University of Edinburgh. You’ve just started a new course. You’ve got your reading list, and you’re feeling overwhelmed by it. Your student advisor has said to book a one-to-one meeting with the Institute for Academic Development (IAD). You land on the IAD website homepage. Can you find out how to book a one-to-one?
- You’re a postgraduate taught student. You’ve seen a poster advertising dissertation workshops. You arrive at the IAD homepage. Can you find a workshop that will help you with writing your dissertation?
- You’ve just completed your first year of your History degree. Your exams didn’t go so well and you have a resit coming up in August. Your friend said there’s some advice on preparing for exams on the IAD site. Starting from the homepage, can you show me what advice you can find?
What we learned from testing the academic skills content
None of the participants clicked on the Study Hub resources. The Study Hub contains valuable content that could help students with the problems we presented in the test. But none of the participants followed links to the Study Hub, despite these links being present on several of the pages that participants used.
The Exam Bootcamp page did not lead participants to other exam content. When looking for support for exams, participants scanned the homepage for any mention of exams. This led them a page about Exam Bootcamp, a resource available on the Learn VLE. Participants seemed pleased to find this but didn’t find any of the other available resources or workshops about exams.
Participants did not seem phased by the amount of dissertation workshops on offer. When tasked with finding a workshop to help with dissertations, participants reached a page that listed 17 different workshops on the topic of dissertations. We had anticipated that that might be a problem. But none of the participants seemed worried about how they would choose a workshop to attend. They scanned through the list and picked those that looked like they might be relevant.
The new sites are now live
The teams working on the new sites took the findings from the strategy sessions and the user research and fed this into the development of four new sites. The new sites went live in July, using layout, wording and information architecture choices that reflect some of the insights gained from the content review work.
For example, the Postgraduate Researcher and Career Development site puts PGR workshops in a prominent location on the homepage. Users can browse all workshops, or view them grouped by theme. Next to that, a card shows development support options for PGR students at different stages of their studies (‘Starting out’, ‘Keeping on track’ and ‘Final stage’), echoing the variation between needs at different stages that we identified in the content review.
Postgraduate Researcher and Career Development
![]()
In a similar way, the new Academic Skills homepage leads with the priority content we identified in the review. Study Hub is given the most prominent location at the top left of the screen on desktop, and at the top of the screen on mobile. Cards for workshops and one-to-one meetings show the support services that are available to students. Elsewhere on the page, students can find support for topics such as writing, making notes, literature reviews and exam preparation.
![]()
Reflections on this work
This work was a great example of how you can take an evidence-informed approach to content strategy when you’re working to a tight deadline. It also showed how you can use a website migration as an opportunity to review the content you have and think about how it supports your service. By drawing on existing knowledge about users, and testing ideas and prototypes with them, the teams were able to make decisions about their new websites with user needs in mind.
If you’re reviewing your content
If you’re thinking about reworking what you have on your website, get in touch. The User Experience Service can help you to evaluate what you have, develop your content strategy and bring user research into the process.
-
Website and Communications Blog
- How we helped the Institute for Academic Development to establish user needs for their website
How we helped the Institute for Academic Development to establish user needs for their website
This summer, the User Experience Service worked with colleagues from the Institute for Academic Development to review their content ahead of a website restructure. In this post, we look at how we developed a shared understanding of user needs to help guide this work.
Supporting the migration of IAD content
At the end of the academic year 2025 to 2026, the services performed by the Institute for Academic Development (IAD) moved to different areas of the University. As part of this change, the IAD website was being shut down.
Content from the IAD website was moving to four new sites:
- Academic Skills
- Postgraduate Researcher and Career Development
- Academic Development
- Researcher Development
In June 2026, there were 900 published pages on the IAD website. Time was tight, with the content needing to have moved to its new location by August.
Members of the User Experience Service (Emma Horrell, myself, and interns Hannah Watson and Shlok Gupta) worked with colleagues from the IAD to prepare for this move. In this post, I’ll outline how we worked together to establish who uses the content and what user needs it supports.
We ran two online sessions to identify service users and their tasks
In a content review, we typically start by working out who’s using the site and what they’re trying to do when they arrive at it. It’s also useful to know what kind of events trigger people to visit the site and what information is likely to be most valuable to them at that point.
To answer these questions, Emma Horrell and I ran two online sessions with colleagues who were familiar with the existing content and who would be involved in creating the new sites. We wanted to draw together existing knowledge about users of the IAD’s services so we could establish what these users need from the IAD website. Each online session brought together a small group of specialist staff. We met over Teams and used Miro to gather the information.
Session 1: Postgraduate research students
In the first session, we focused on content for postgraduate research (PGR) students. Through discussion, it became clear that this audience group has shifting needs as they complete their programme of study.
We grouped the student needs on Miro and three stages became evident:
- They become a PGR student
- They get started with their PGR programme
- Later in the programme
We collected user needs and grouped them into three stages.
New PGR students have distinct needs
We learned that new PGR students might hear about the University’s academic development services in Welcome Week or in a conversation with their supervisor.
At this early stage in their programme, their tasks might include:
- find workshops that are relevant to me
- sign up for newsletters that tell me about future workshops
- learn what I can expect from my upcoming programme of study
- connect with other PGR students at the University of Edinburgh
PGR student needs change as they progress in their studies
Later in their PhD, these same students will likely have different needs. For example, they might be looking for a particular workshop that will help them to:
- write a dissertation
- put together a literature review
- prepare for a viva
They might also be thinking about what comes after their PhD and what they need to do when they take the next steps in their career.
Some tasks apply at all stages of a PGR student journey
At all stages, PGR students could have these needs:
- find out which services support academic skills development
- find out what these services do
- engage with these services in the event of questions
- find out what’s on (workshops, support sessions)
- get updates about upcoming workshops (for example through a newsletter)
- access policy and standards information
At times, PGR students may also need to know about other University services that can support them. For example, Edinburgh Research Office and Edinburgh Innovations can provide support with funding applications. The Student Counselling Service can support with health and wellbeing issues.
Presenting available workshops was the priority
Our discussions made it clear that presenting information about workshops was going to be the top priority on the new site for PGR students. On the IAD website, workshops were presented in a variety of ways:
- in an A to Z list
- listed by theme
- on a page called “At a glance – workshops this month”
- in an embedded PDF brochure
Session 2: Users of academic skills content
Alongside this work, other IAD staff were preparing content for the new academic skills site. This would have a different purpose to the site for PGR students, and it would be pitched at a wider audience of students at all stages of study.
An online session helped us to understand user needs
We ran a second session with a small group of IAD colleagues to learn about the existing academic skills content. Specifically, we wanted to know:
- who uses academic skills content
- how they arrive at it
- what they need from it
As in the first session on postgraduate research students, we gathered together known user needs and looked for common themes.
We used Miro to collect the needs of students looking for academic skills support.
Through these discussions, we developed some scenarios to illustrate how a student might arrive at the site. For example:
An undergraduate student gets a poor grade for an assignment. They speak to their student adviser about how to improve their essay-writing skills. The adviser directs them to the IAD. The student goes on the IAD website to look for resources or workshops about essay-writing.
By thinking about user tasks in this way, we developed our understanding of what it might be like to be a student scanning the homepage, looking for content that will help them to complete the task they have in mind.
We also established that the priority for the new academic skills website would be content that supports the following:
- Booking on a one-to-one study skills consultation session.
- Finding a study skills workshop.
- Finding online resources to support study skills such as writing and presenting.
Understanding student users and their priority content like this gave us a foundation for what we needed to do to assess the site and consider alternative designs.
The sessions uncovered a need for testing
Through these two sessions, we collected existing understanding of who was using the IAD website and what they needed from it.
This led us to the next stage of this work. We wanted to know what it was like for these users when they interacted with the site. Could they find what they were looking for? What might be the best way of organising this content on the new sites?
To answer these questions, we carried out two sets of user research:
- To support the work developing content for PGR students, we joined a Careers Service event at McEwan Hall to hear directly from those students, seeing how they responded to existing pages and some prototyped alternatives.
- To support the work developing academic skills content, we ran a short series of usability tests to observe students using the existing IAD website.
Read more about this work
This is part one of a two-part series. In the next post, I’ll explain how we used paper prototypes and usability testing to learn how users interacted with various site layouts.
Testing with users informed the Institute for Academic Development’s content migration
-
Website and Communications Blog
- Architecting a new website to support dementia research at the University
Architecting a new website to support dementia research at the University
Alzheimer’s and dementia research is a flagship strength of the University. Websites of different research units play a key role in supporting this work. The UX Service worked with the Alzheimer Scotland Dementia Research Centre to design and launch Dementia.ed, a new home for dementia research at the University.
The University of Edinburgh is internationally recognised for research on dementia and Alzheimer’s, with different research activities occurring across groups, labs, centres in and across the various Colleges. The team at the Alzheimer Scotland Dementia Research Centre had identified a need to join up different research activities, to ensure connections could be made across different research initiatives by facilitating an interdisciplinary University network.
A website was the proposed way to meet this need, with the intention that this site would provide visibility of the different research areas, to enable academic researchers to exchange knowledge and expertise within the wider dementia and ageing research community. The site was also anticipated to have a secondary, non-academic audience: people with a general interest in learning about and supporting dementia research.
With the go-ahead to build an EdWeb2 site confirmed, Angel Garmpi from the ASDRC contacted the UX Service for help to shape the new site.
Before creating a site, we thought through a content strategy
In the seminal textbook on content strategy, ‘Content strategy for the web’ by Kristina Halvorsen and Melissa Rach, the authors introduce content strategy as a way to avoid being tormented by a burden of excess content to be managed. Starting a site from scratch is a golden opportunity to make decisions upfront to prevent content spiralling out of control, leading to website bloat and content debt further down the line.
The framework in the book outlines four elements of content strategy:
- Substance – what content does the site need and why?
- Workflow – how does the content get created and maintained?
- Governance – who owns the site and is responsible for its content?
- Structure – how should the content be organised so it can be found?
These elements served as a useful way to tease out key considerations from the start, setting a path for good decision-making and planning for the longer-term, past the website launch.
Substance: Content for dementia researchers and for people interested in dementia research
By considering questions about what the site was for, and who would visit the site and why, we were able to establish the site substance.
The ASDRC team had identified five main purposes for the site:
- To facilitate a network where researchers could come to collaborate with others on research work
- To showcase and bring together the different groups, centres and parties involved with and supporting dementia research
- To be a source of information where people interested in dementia research could come to read about latest insights
- To serve as a volunteer hub to capture details of people wishing to participate in dementia research (or make tissue donations)
- To act as a payment portal where supporters of dementia research could make financial donations.
Using these purposes as a starting point, we defined primary and secondary site audiences and their associated tasks to flesh out detail of the site substance.
Site audience 1 (primary): Academic researchers working on dementia and ageing
Needs and tasks:
- Meeting other researchers and groups working in related fields
- Establishing peers to collaborate with and exchange knowledge
- Getting support and help with grant and funding applications
Content to support:
- Contact details of the different groups
- An entry point to join the network
- Ways/touchpoints to stay informed such as events
Site audience 2 (secondary): People interested in tracking dementia research developments
Needs and tasks:
- Finding out about dementia research in relation to a specific condition of interest
- Staying up-to-date with advances in specific areas of dementia and/or ageing research
- Identifying relevant academic papers and publications to form the basis of journalistic content
Content to support:
- Publications and updates about dementia research
- News articles about topical research areas
Site audience 3 (secondary): People wishing to take part in dementia research
Needs and tasks:
- Registering to donate brain tissue
- Volunteering to participate in academic research studies
Content to support:
- A place to register interest in becoming a research participant
Site audience 4 (secondary): People wishing to make monetary donations
Needs and tasks:
- Donating money to contribute to the progress of dementia or ageing research
Content to support:
- A portal where financial donations can be made
Workflow: content is curated and published by the ASDRC
Considering the site substance led us to think about where this content would come from – the sources and the subject matter experts behind it. All too often, websites are built and filled with content for a launch, but without mapping out the content sources and understanding the underlying creation processes, launch-day content is at risk of quickly going stale because there is no plan in place to maintain it.
To fulfil its primary purpose of facilitating dementia research networks, the bulk of the content for the Dementia.ed site was planned to come from the different University-wide dementia research groups and initiatives. This content would be captured as links to the different sources, to ensure it was current. Events from the different groups needed to be highlighted as network-forming opportunities, so these were planned to be pulled together in a single location on the Dementia.ed site by the ASDRC team. It was also felt that the site should provide a round-up of news from different areas, which again would be curated by the ASDRC team. Finally, the site would include static content such as: practical information about the ASDRC and the dementia research network, contact details, information about how to join the network and details for how to contribute, either donations or by participating in a study.
Taken together, this meant the initial content-related work requirement from the ASDRC team would be: to create the static content for the site in the different sections and to establish the correct links for the different research groups. Going forward, there would be a maintenance required to check the static content and the links remained correct, and to produce events and news content, which all seemed doable given resource in the ASDRC team.
Governance: ASDRC team owns the site and manages the content
Considerations of how the new site would be governed were related to the workflow processes. A prerequisite for obtaining an EdWeb2 site is the need for a named site owner to take overall responsibility for the site, and the same ASDRC team member responsible for creating, curating and maintaining the site content had been assigned the role of site owner. This kept things straightforward in terms of decision-making about the site’s content and overall direction, although there was a need for the site owner to forward-plan a content calendar for maintenance, curation and creation tasks, and to assign responsibility to other ASDRC team members as a back-up to avoid a single point of dependency.
Structure: content arranged under common user tasks
Starting the design of a new website from a blank canvas can be difficult as there are lots of questions to consider at once. What should the menu contain? What should be on the top of the homepage? How should the content link together?
Another core textbook useful in site building is ‘Information Architecture’ by Peter Morville and Jose Arango. In this book, the authors encourage thinking about how to group or categorise content (in an organisation scheme) as well as how to establish relationships between those groups or categories (in the organisation structure).
Typically, websites adopt ambiguous organisation schemes – with content arranged by topic, subject, audiences and/or tasks to encourage browsing and discovery. This approach was appropriate for the Dementia.ed site to enable it to support the different audience groups and their associated needs and areas of interest. We considered a task
For the organisation structure – a non-hierarchical flat arrangement made most sense. Given the site was anticipated to have a relatively small number of pages, and would have clear calls-to-action for the different audiences, it didn’t seem necessary to plan a heavily hierarchical structure. For the content in the different research group sites, a hub-and-spoke model could be adopted, with Dementia.ed acting as the central hub, linking off to the different sources.
Before building, we needed to sense check the proposed site map with users
With ideas for the site mapped out, it was tempting to just get building in EdWeb2. Before creating the new site, it was wise to test assumptions with some testing.
Usability testing – where audiences are given a task to perform online and are observed as they do it – always provides an excellent source of insight into how audiences naturally interact with a digital artefact like a website, how they make sense of the menu labels and use on-page content elements to find their way to complete their tasks.
When it comes to new sites, however, it can be difficult to know what to test with users. If you mock-up a prototype, which parts should you include and which can you omit? Asking participants to make sense of menu labels that lead to empty pages can feel unnatural, and setting participants tasks can fall flat if there’s no real-life site content where they can find the information they need to complete tasks authentically. All that said, it’s impractical to build out an entire site just for testing purposes, as it’s likely the testing will reveal ways it needs to change which can end up adding more work.
We used another research network site as a proxy to test with
Thinking through the options with the ASDRC team, we came up with a plan. The ASDRC team had started looking at other research network sites for inspiration and ideas about how to build Dementia.ed, so we decided to adopt one of these sites for the basis of a test.
We chose the Edinburgh Mental Health Network (EMHN) as a testing site. This site included top level menu items: People, Research, News, Events, Blog and Contact. It had a total of 13 main pages with additional pages within the News, Events and Blog sections.
Access the Edinburgh Mental Health Network website
Since the site supported researchers in mental health as well as audiences with a more general interest in the topic, we were able to shape a usability testing protocol to use the site to assess how two audience types sought to complete representative tasks:
Audience 1: Academic researcher wanting to join the research network
Representative tasks:
- Find out what the network is, and its goals and values
- Find out how to become part of the network and what this would entail
- Find out about upcoming activities
- Find research opportunities in the network
- Find out how to develop your research project and involve people with lived experience
- Find content you can use in support of a funding application
Audience 2: Non-academic with a personal interest in the research area
Representative tasks:
- Find out what research is being done across the network
- Find out how the network is managed
- Find out how to keep up with what’s happening in the network
- Find out how you can contribute as a research participant for studies
- Find out how to donate money to support the research
Some of the tasks mapped clearly to specific sections of the EMHN site, whereas others were more ambiguous – they could have been completed in multiple sections. In all cases, it was useful to see how participants approached the tasks, so we could use their expectations to design Dementia.ed in the most audience-focused way possible.
We learned which menu labels worked, and which didn’t
The results of the testing gave us a clear steer on which elements of the Edinburgh Mental Health Network site supported the different user groups completing their tasks and therefore would be a good fit to adopt in the new Dementia.ed site. The tests also showed areas where the approach of the EMHN site was lacking – and therefore where different structuring and/or labelling was more appropriate.
The Events and News sections worked well
To complete tasks about finding out about activities and keeping up with what was happening, participants naturally gravitated towards the Events and News sections of the EMHN site, where their expectations of finding events to sign up to and articles to read were met.
The EMHN site was lacking a specific place for information about the network
When working on tasks geared at finding out what the network was, what its goals and values were, and detail about how the network was run, participants looked for an ‘About’ section when they used the EHMN site. When they realised this didn’t exist, they tried multiple other locations including the homepage and the Research section. This suggested an ‘About us’ section would be useful to include in the Dementia.ed site.
The People section didn’t resonate as the way to become part of the network
The EMHN site presented information for academic researchers to join the network in a ‘People’ section, which also contained information about the management of the network as well as researcher support resources. In the tests, participants struggled to find this content when tasked with finding out how to join the network, suggesting that a more explicit ‘How to join’ section would be more helpful to support this key task.
The Research section was too broad to usefully support specific tasks
Test participants gravitated towards the ‘Research’ section of the EHMN site for various tasks: to find research opportunities, to find help with research project development, to find content to support funding applications, to find out how to be a research participant and to find out how to financially support research activities.
On arrival in this section, when they realised it only contained content detailing different research activities, they went elsewhere. This suggested the need to separate out content about research activities from content supporting specific research-related tasks and to label it accordingly. In other words, instead of having a large, general ‘Research’ section, opt for smaller, more specific sections such as: ‘Research areas’, ‘Participate in research’ and ‘Donate’.
The new site has a flat structure, with content arranged by topic and tasks
Keeping site of the content strategy work, and using the results of testing the proxy site to make key decisions about structure, the new Dementia.ed site was built and launched.
The site contains six main sections, adopting a mix of topics and tasks to signpost the different sorts of content. The ‘About’ section contains background information about the dementia research network, with a dedicated page bringing together links to the different research centres and groups in a hub-and-spoke approach. A ‘What’s On’ section combines pages to Events and News content. The other four sections are labelled in task-focused ways, to ensure relevant audiences can quickly get to the content they need to complete their tasks: Join the network, Get involved, Contact us and Donate.
To ensure audiences reach content in the most direct way possible, the content in each section is either available as a top-level page, or a sub-page, avoiding long pages and deep hierarchies which may confuse those visiting the site.
Access the Dementia.ed website
If you need help architecting your website, get in touch
From start to finish, the process to design and launch the Dementia.ed website took a few months, including a few meetings between the UX Service and the ASDRC team, some short usability tests and an overall site review.
The ASDRC team provided the vision of what they wanted to achieve, and subject matter knowledge of their audience base and their associated tasks. With this information, the UX team could apply expertise to translate goals into a user-centred website, which is manageable for the ASDRC team to maintain.
We’re happy to offer help, advice and support to other University teams. If we can help, please contact us through the UX Service website.
Web Designers Love Dark Mode Toggles. Users Might Not Care.
How the Prompt Box is Swallowing Website Interfaces Whole
Nobody Wants Your Damn App
Chrome Can Now Measure Exactly How Obnoxious Your Website’s Ads Are
Apple’s Foldable iPhone Has a Left-Handed Problem
-
Website and Communications Blog
- What I learned in my summer internship researching digital content accessibility
What I learned in my summer internship researching digital content accessibility
In this post, User Experience team intern Hannah Watson shares her work over the summer researching digital content accessibility with the EdWeb 2 publishing community.
Introduction
As part of my internship with the User Experience Service, I have been investigating digital accessibility at the content level across University web pages, particularly on EdWeb 2 sites. Digital accessibility at the content level is about applying principles of accessibility to how web content is written and formatted. This includes, but is not limited to, content features such as heading levels, links, and alt text. It does not include anything that is not involved with content design or that is controlled at a higher level by the Content Management System (CMS), such as font, font size, or colour contrast.
Research aims and scope
The specific research questions for this project were:
- How do web publishers learn about digital content accessibility?
- What do web publishers know about digital content accessibility?
- How do web publishers implement digital accessibility requirements and principles in their content?
- What challenges do web publishers face in creating digitally accessible content?
More comprehensively, I was investigating the accessibility of:
- Heading levels
- Ensuring that heading levels are used correctly
- Not using heading levels for emphasis
- Links
- Clear and concise link text which describes the linked destination
- Link text which makes sense on its own
- Avoiding URLs on web pages
- Lists
- Alt text
- Clear link text that describes the image
- Images
- Ensuring that all images have appropriate alt text
- Avoiding images of text
- Videos
- Including human-corrected captions with any videos uploaded to web pages
- Making sure that transcripts are available for all videos uploaded to web pages
- Italic, bold, and underlined text
For more detailed guidance on these topics, please refer to the University of Edinburgh editorial style guide.
Editorial style guide | Information Services
This research has been necessary for the User Experience team as it allows us to identify which areas of content accessibility are challenging for web publishers. From this, the team can adapt guidance and training to provide extra support on these more challenging areas where possible.
Research methods
Interviews with web publishers
I started my research by setting up short, informal interviews with six University web publishers. In these interviews, I asked the publishers about their experiences of creating digitally accessible content. The aim of these conversations was to understand how publishers learned about digital accessibility and what challenges they face when making content as accessible as it can be. During the interviews, we referred to web pages that these publishers work on to get concrete examples that illustrate the topics we discussed. Through these interviews, I was able to identify a number of trends, particularly in the challenges that the interviewees and their colleagues face.
Survey of EdWeb 2 publishers and the Web Accessibility Special Interest Group
Following these interviews, I created a survey to further investigate the findings and collate more supporting evidence for these findings. The survey consisted of 10 questions, three of which were demographic based as a filter, with a final question asking permission to follow up with those who responded.
The other six questions asked about:
- which actions the participants took to make their content digitally accessible
- what resources they used to do so
- what challenges they face in doing so
The findings from the survey were effective in making the information gathered in interviews more robust and provided further evidence for some trends that were identified previously.
To publicise this survey, I sent a brief statement explaining my research into two Teams channels, the Web Accessibility Special Interest Group and the EdWeb 2 Community. Overall, the survey received five responses, and while this is a limited number, I found that it helped to support the findings from the interviews.
Analysis of Effective Digital Content workbooks
The final method of research that I used to learn about the digital accessibility of content on University web pages was by looking at pages that were submitted within Effective Digital Content workbooks as part of the course. The course requires learners to choose pages from a University website and assess the effectiveness of the content. By looking at the pages that learners selected, I was able to use active examples and make note of content accessibility issues that were present on live pages. This process also highlighted a number of trends.
This method of research was also useful in that it provided two separate sources of data. Firstly, the answers in the workbook helped me to gauge learners’ understanding of the principles covered in the course. Secondly, the web pages linked by those taking the course allowed me to have live examples of content to assess against content accessibility principles. While there was not necessarily overlap between the answers in the workbook and the pages submitted as part of the workbook (as some people may not have been fully or at all responsible for the content on those pages, or the content could have been updated since the workbook was submitted), it was useful to see these things separately.
Findings about accessibility
Combining findings from all areas of research for this project, I have identified a number of trends in the accessibility of digital content.
Headings, alt text, and links are the principles most often put into practice by participants
In the interviews and the survey, participants were asked what principles of accessible digital content they actively used when designing content. More than half of participants mentioned three principles in particular, with all participants mentioning at least two, which were:
- using the correct heading levels
- adding meaningful alt text to an image
- writing clear and descriptive link text
Interestingly, while these principles were mentioned frequently by participants, and evidenced by the websites that we discussed in interviews, they are also principles that are often missed on University web pages. This is discussed in more detail in the corresponding sections further on in this post.
PDFs are hard to avoid
One standout finding from the interviews was that web publishers sometimes struggle to find an effective alternative to PDFs. While PDFs are not necessarily accessible, they do have benefits which make them useful for publishers. For example, they are downloadable, searchable, and cannot be easily edited without permission from the owner. There are ways to make PDFs more accessible, such as avoiding decorative images, adhering to accessible content design principles within the PDF, and checking colour contrast. However, the interviewees were more in favour of finding a way to turn their content into a webpage as this is more likely to result in accessible content.
Out of the survey responses, three also mentioned that they had recently chosen to publish content as a web page rather than as a PDF, highlighting their knowledge that a web page is preferable in terms of accessibility. However, their personal preferences or how difficult they found this is unknown as I was unable to follow up with these participants.
Heading levels are often skipped and headings vague
Across web pages that I assessed for correct heading levels, there were several which skipped heading levels throughout the page content, such as going straight to a heading 3 without that heading being nested within a heading 2 section. Additionally, headings on the University web pages that I investigated were often generic, instead of being specific about what a page or section will contain, which is the recommended approach.
The fact that the web publishers who were interviewed and surveyed were aware of the importance of correct headings levels and specific headings, and that the majority of these publishers have attended a staff training or used the editorial style guide, suggests that the guidance provided is accurate and useful for publishers.
To increase the use of correct heading levels, as well as clear and descriptive headings on web pages, participant responses suggest that increasing the reach and engagement of existing training and resources involving headings would be effective. This includes training provided by the User Experience Service, such as Effective Digital Content or Content Improvement Club, and the University of Edinburgh editorial style guide.
Alt text is well written but sometimes missed
In three of the interviews, and in two survey responses, participants mentioned that they struggled to find the time to add alt text to images on their web pages. However, participants in the interviews also stated that they understood the importance of alt text, and what writing meaningful and clear alt text involves. This is reflected in the answers in Effective Digital Content workbooks. The course contains a question asking learners to write meaningful alt text for two images, and this question is often answered well. This suggests that web publishers understand how to write alt text, and that the obstacle in doing so is more likely to be related to time and resource.
One way that time constraints on adding alt text could be improved is to reduce the number of images on a web page, which will not only make this task more manageable but also is also more sustainable.
Explaining acronyms and abbreviations is common
All five responses to the survey stated that they had explained an acronym or abbreviation recently. Although this did not come up in the interviews often – only once – the survey responses suggest that this is common practice for web publishers who are familiar with digital accessibility principles.
Link text is frequently inline or not descriptive
Participants stated that they understand the importance of clear link text as a principle and make effort to implement this into their content. However, similar to headings, this sentiment is not reflected across numerous University web pages. In some of the pages submitted as part of the Effective Digital Content workbooks that I investigated, there were frequent occurrences of inline link text, or link text that does not clearly describe the linked destination.
To increase the writing of link text on a separate line, as well as clear and descriptive link text, participant responses suggest that increasing the reach and engagement of existing training and resources involving links would be effective. This includes training provided by the User Experience Service, such as Effective Digital Content or Content Improvement Club, and the University of Edinburgh editorial style guide.
Findings about staff experience and engagement
A trend from both the interviews and survey responses is that for many of the people involved with this research, their knowledge of digital accessibility started with a personal interest. Specific examples of this that participants mentioned include learning about accessibility as a student (and then going on to be an accessibility advocate for a student society) and working with disabled students and learning through experience.
During the interviews, multiple staff members mentioned that they believe, through their experiences, that a large part of issues with creating accessible digital content at the University surrounds communication. A combination of factors was discussed that had communication at the centre, including:
- the importance of digital accessibility not being widespread enough.
- consistency about expectations between schools or areas of the University, such as one page being edited by multiple schools or areas and having different standards.
Job-based constraints were also mentioned frequently, such as limited time to add alt text to all images on a web page and working on a page where the lead publisher takes a design forward approach which can sometimes clash with accessible content principles. These answers were also reflected in the survey responses, with all of the responses mentioning either one or both of these problems.
What resources do web publishers use for learning about and developing their digital accessibility skills?
The primary resource that participants mentioned using to learn about digital accessibility and develop their skills was University-provided staff training, with 10 of 11 participants mentioning this. Effective Digital Content was specifically mentioned twice, and Content Improvement Club three times. In the interviews, two participants mentioned more general staff training, with one survey response saying the same. In the survey responses, three participants also said that they used training provided by the Disability Information Team.
The Web Content Accessibility Guidelines 2.2 (WCAG) is another resource that was frequently mentioned by participants, with six in total saying that this is a resource that they use as guidance on accessibility.
The University of Edinburgh editorial style guide was mentioned by five participants as a resource that they used to provide guidance on digital accessibility, in which the guidance reflects what publishers learn in training courses, meaning that information they take from the style guide is in line with accessibility and content design training.
What I learned from researching content accessibility approaches in EdWeb
I learned a lot during my time researching how web publishers at the University of Edinburgh approach creating accessible digital content. I thoroughly enjoyed the opportunity to work with staff from a variety of different areas of the University. This helped me to learn how to identify commonalities in interview and survey responses despite the areas of work being distinct. I also enjoyed this because I was able to learn much more about the work that goes on across the University and what the work of other teams involved.
I also particularly enjoyed the format of my internship being a combination of individual work and working with other members of my team and the Disability Information team. This balance allowed me to set my own goals while still getting the opportunity to work and learn collaboratively as part of a team, prioritising my tasks between my own and those that I was working on with others.
A limitation of my work was that it was much easier to contact and work with staff who have a genuine interest in the subject area of accessibility, which does not lead to research that is representative of how University staff approach digital accessibility as a whole. Having done the research that I have so far, a continuation of the research would be most beneficial if it focused on the experiences and approaches of staff who are less familiar with digital accessibility requirements and principles, as this would create a more well-rounded understanding of web publisher accessibility approaches at the content level.
Furthermore, another limitation of the work was the time constraints which have restricted my ability to develop solutions to issues identified during the research period. This was expected to a degree, as the solutions depended on the outcomes of the research, which has taken the majority of the 12-week period. The positive outcome of this is that the research and findings will provide the User Experience service with information that allows for both further research and for adaptations to guidance and training if it is deemed necessary.
Conclusion
This research has helped the User Experience Service to assess their training and the resources that are available for publishers to learn more about digital content accessibility. Keeping in touch with the publishing community through projects like this helps the team to direct their effort to real challenges that publishers face on a day to day basis.
The research completed throughout the duration of this project will be supported and advanced by further investigation, particularly by communicating with web publishers who are less involved with digital accessibility as a whole. This will likely help in providing more concrete solutions to digital content accessibility issues across EdWeb 2 pages.
Why Tech Companies Hid the “Log Out” Button (And How to Win It Back)
The Infinite Scroll Was a Design Crime. Now Meta is Paying for It.
-
Website and Communications Blog
- Hearing about approaches to student voice at the Learning and Teaching Conference 2026
Hearing about approaches to student voice at the Learning and Teaching Conference 2026
In June, I headed over to the Nucleus Building to attend the University’s Learning and Teaching Conference. The theme of the conference was student voice, and I was there to present about how the Careers Service and the UX Service had involved students in the design process for a recent website development project.
Emily McIntosh’s keynote “Nothing about us without us”
The day kicked off with a keynote from Emily McIntosh, Chief Student Officer at Harper Adams University. Emily set up the theme of student voice and covered some important distinctions for us to keep in mind throughout the day. Emily compared two ways that students are involved in decision-making processes: representation and partnership.
Representation is where staff within a university create something – a new policy, for example – then show student reps and ask for their opinion. While this is better than not involving students at all, it can lead to student reps feeling that they are responding to decisions that have already been made. The input from student reps sometimes comes at late stage, and can lead to tweaks but not necessarily substantive change.
Emily suggested that “you said, we did” posters can be indicative of this sort of approach. She contrasted this with examples of universities working in partnership with students from the beginning. In this way, students are actively involved in helping to shape services and policy. Emily described this as a relational, dialogic culture of ongoing collaboration, where students and staff work on problems together. This approach is sometimes described as co-design.
For me, there were some clear analogies in what Emily was describing to processes we advocate for in user-centred design. For example, bringing user input into the early stages of a design process is generally more effective than getting the same input later in the day. In addition to this, user-centred design often emphasises the benefit of ongoing collaboration and continuous iterative improvement, keeping users involved before and after the launch of a product.
Working in partnership like this is not necessarily a simple thing to achieve. But I appreciated the message: incorporating students into how we decide to do things as a university brings some significant advantages. Whatever the practical challenges of implementing this approach, moving from designing-for to designing-with offers us opportunities to better meet service users’ needs.
Highlights from the rest of the day
The rest of the day made me think about issues around student voice.
Neil Speirs (Widening Participation Manager) raised the issue of working class students often being absent from student voice projects. Neil’s talk made me think about how good quality user research often depends on recruiting a diverse group of participants. Recruiting the right research participants can be difficult. We see that all the time in our work. But without a diverse range of voices, user research can miss important perspectives and draw an incomplete picture of what matters to users of a product or service.
Neil Speirs presenting at the Learning and Teaching Conference. Slide reads: “A neutral proposition? We must be aware that student voice does not stand as a neutral proposition. Without democratic and participatory structures and cultures, student voice projects in learning and teaching will not provide democratic and participatory outcomes. We might reflect on the class based inequalities that are ‘baked into’ (Major et al, 2023) our structures, cultures, policy, curriculum and pedagogy. Which students are readily able to participate and benefit from student voice projects? Which students are invited to take part? Do we even notice? Is it the usual suspects? Do all students know the rules of the game and therefore why and how it might be advantageous to partake in student voice projects? Is the working class student voice heard in these spaces?”
Clare McKay (Student Recruitment and Admissions) and Christine Emmerson (Communications and Marketing) talked about how student recruitment markets can help us to anticipate student needs. One important point they made was that marketing is not just about promoting courses and programmes. It’s also about understanding our audience and whether there is a need. This resonated with aspects of UX work, where an understanding of user needs can tell us not just how to communicate information, but also whether that information is needed in the first place.
Jenna Fyfe and Carine Abraham (Edinburgh Medical School) talked about research they had done with alumni networks. Their project focused on alumni from three online postgraduate programmes at the Medical School. Alumni described how they create networks with each other following graduation, and how they often retain an interest in maintaining contact with programme teams from the University. This contrasts with a picture where alumni wish to maintain contact with university-wide structures. I found this interesting, because we sometimes think of people outside the University seeing us as a single entity unified by an overarching brand. In contrast, the alumni in this research weren’t thinking about a monolithic institution. Instead, they retained an interest in personal connections with a small, specific part of the University.
My talk was about our work with the Careers Service
In my talk, I gave an overview of how the Careers Service developed their website in 2025. In particular, I highlighted how the User Experience Service assisted with the design of collaborative activities that brought student voice into the work. I wanted to show how bringing student input into this sort of design activity makes the work better, and I talked about the benefits that the Careers Service achieved by approaching the project in this way.
You can read more about our collaboration with the Careers Service elsewhere on this blog:
Blog posts about work by the UX Service and the Careers Service
Reflections on the day
I thought there was a lot of value in holding a conference like this, with most of the speakers being staff and students from the University of Edinburgh. Presenters were speaking from a broadly similar context, which meant the talks had a high level of relevance to our work. There were a lot of familiar faces and it was good to catch up in the breaks with colleagues I hadn’t seen for a while.
My main takeaway was that student voice projects often have a lot of crossover with user-centred design approaches. In both cases, we’re interested in making things better by involving end users. I felt it was a great opportunity to hear about how student voice projects have been informing colleagues’ work at the University.
About the conference
The 2-Second Rule: How to make your website feel like magic in 2026
-
Website and Communications Blog
- Can AI do a content audit of my website? A review of different ELM models and AI products
Can AI do a content audit of my website? A review of different ELM models and AI products
Content audits are a long-time staple of the user-centred website toolkit, but they take time and effort to complete, which many website owners struggle to find. In my ongoing AI experimentation, I tried using AI tools to help me with auditing web content.
Every day brings a raft of new AI developments. When choosing how to use AI, I’m less sold on getting it to do creative tasks for me because I like doing those myself. Instead, in pursuit of freeing up my time, I prefer to use AI to help me with the tedious tasks I never get around to, the ones I know will take me longer than I think, the ones I put off again and again. Here’s looking at you, content auditing.
Websites are easy to grow but difficult to keep in check
When people come to the UX Service for help with their websites, they tend to use several phrases to describe their site. These have included: ‘It’s a mess!’ ‘It’s a bit out of control’. ‘It’s grown arms and legs’. None of this is unusual when you consider the lifecycle of websites. Over time, websites change hands. Pages once carefully crafted are easily forgotten as new editors take over. New content is created without knowing what’s already there. Very often, content needs to be published to a deadline and it’s quicker to publish a new page than seek out and amend an existing one. Seldomly do publishers want to get rid content that’s taken time and effort to produce.
Read some thoughts on content housekeeping by UX Service team members (current and past):
Digital housekeeping – applying content management practices to improve digital sustainability by me
Be a gardener by Ari Cass-Maran
Content Audit Findings and the 100k Challenge by Milo McLaughlin
How to get a grip of your website (and then keep hold) by Neil Allison
A content audit is the best place to start improving a website (not the homepage)
When people seek to bring order to disarray on a website it’s difficult know where to start, so typically, they start on the homepage with ideas to change images or add new content. While this gives sites a superficial makeover, it doesn’t really help site audiences in search of content. Fewer and fewer website audiences are starting out on homepages, instead they’re parachuting directly into site content from Google search and increasingly, from AI summaries. What does this mean? It’s more important than ever to make sure your web content is up-to-date, accurate, useful and relevant for your website users. Here’s looking at you, content auditing.
Read more about getting your content ready for AI from a recent Content Improvement Club session:
Auditing a website needs a methodical approach (which is where AI can help)
If audits are so important, why are they so easy to put off? Short answer, they can take prolonged amounts of time and concentrated effort. The larger and more complex your site is, it’s likely that there will be more content to review, and the more difficult it may be to maintain a consistent auditing approach. Going through multiple, long pages of web content, it can be easy to run out of the time you’ve allocated for your audit. Breaking the audit into chunks may lead to different results from different sessions as biases creep in. Bringing colleagues in to help can share the load, but can also introduce new perspectives and contradictory auditing decisions. There’s a need to maintain ruthless objectivity, to counter human subjectivity and avoid errors infiltrating. Here’s looking at you, AI.
If you can plan and describe a website content audit, you can ask AI to help you do it
Website content audits can be done for many purposes, but commonly, doing a content audit involves making decisions about what content to keep, what needs to go and what needs to be changed – all in pursuit of reaching a future state of your website. A good way to decide what content to keep is to consider who it’s for and the purposes it serves. If a piece of content doesn’t meet audience needs and doesn’t align with the site purpose, then it’s probably a candidate for deletion. A good content audit of a website therefore typically requires several things:
- An inventory of every piece of content in the site. Typically arranged in a spreadsheet, with one row per webpage. Can be organised into sections or content types, or grouped by root URL
- A note of the site audiences, ideally in order of priority (answering the question ‘Who is this site primarily for?)
- A list of the main reasons those audiences visit the site (answering the question ‘What tasks do people complete on this site?’) together with a list of the main goals of your site (answering the question ‘What do we want people to do on this site?’)
Considering each of these things as different pieces of data, where item 1 is data to be audited, items 2 and 3 are factors or auditing criteria used to decide what happens to those data (typically one of three outcomes: ‘Keep’, ‘Delete’ or ‘Modify’), I reasoned that I could give AI these data, tell it what a content audit was and and ask it to complete this process for a given website.
I experimented with ELM to help me do a content audit of a website
The University’s AI platform, ELM was the natural place to start experimenting with applying AI to aid content auditing. The ELM interface allows you to input a prompt, as well as add additional files and data sources. It also allows you to select different models to handle queries. I was curious to see the differences between the different models – both non-reasoning ones and reasoning ones so I needed to design a prompt that would work for both.
Read more about ELM and its models
ELM new model guides (University log in required)
Being mindful of the environmental cost, I wanted to start with a small AI model (Llama 3.3)
AI comes with a significant environmental cost, which is highly dependent on the LLMs you use. Powerful models with more reasoning power are especially resource-intensive to run, so to avoid unnecessary wastage of tokens and other resources I’ve found it’s sometimes best to start with a smaller model and size up accordingly, as the need arises. The trade-off is that smaller models have limited reasoning power, however, so the responses they provide and the tasks they are able to complete can be limited.
Based on my previous experience of using AI and thinking about its potential application to a typical content auditing process, I had a hunch that providing a small model with something like a site inventory (such as a XML sitemap) and detailed data about website audiences and purposes could overload its context window and produce inferior results.
I therefore decided to begin with a lightweight prompt ‘Can you help me do a content audit of this website? (with a link to the website)’ to give a small model something manageable and learn how it approached handling the query. I chose a website that the UX team had recently worked on, so I was familiar with its content, and I entered the prompt, starting with the Llama 3.3 model – an open weights model (costed on computing power rather than per-token usage) which is hosted in University data centres (instead of being cloud-based). I toggled on the option to include web search so ELM was able to access the website.
The same prompt to different ELM models produced varied results
Having starting with Llama 3.3, I moved on to other models: Open AI’s GPT 5 and then Anthropic’s Sonnet 4.6. To assess and critique responses from each, I adopted the mindset of a person reluctant to begin a content audit and looking for AI help to kick-start action on this, and used this as a roleplay to review and compare the responses.
The response from Llama 3.3. was too generic to convince me to start auditing
Despite a glitch in formatting the response, Llama 3.3. provided some general information about content auditing. Its response included an initial assessment of the site, which made objective judgements of the homepage (assessing it as ‘well-structured, with a clear introduction’), the navigation menu (labelling it ‘easy to use with links to key sections’), content quality (noting it as ‘high-quality, informative and engaging’) and accessibility (pulling out that the site had an accessibility statement).
The response then acknowledged that in order to do a thorough content audit on the site in question there was a need to provide guidance on the aspects to focus on, and rounding off, the response noted some of the aspects a typical content audit would examine: content quantity, content quality, accessibility and consistency across different parts of a site.
Reviewing this response, I felt that if I was a site owner looking to improve my site, this response could very easily persuade me that I didn’t need to bother auditing it at all – and I wasn’t left any wiser about how to meaningfully provide guidance on the aspects to focus on if I did decide to proceed with an audit.
Likelihood to convince me to audit: 3/10
Open AI’s GPT 5’s response was detailed and thorough but a bit overwhelming
As would be expected from a reasoning model, the response when I used GPT 5 was more detailed than Llama 3.3’s. Arranged in two sections, the first part included preliminary observations using the site URL provided – which were split into the following categories:
- Information architecture and navigation – commending the site’s positioning on user intents, yet suggesting improvements with more explicit audience pathways and call-to-action signposting
- Content coverage and freshness – using the site’s range of content types to assess content structure and assessing freshness based on recency of news items
- Call to actions – acknowledging that the site had were multiple calls-to-action which could be combined into primary CTAs for each audience
- Accessibility and usability – calling out the need for more meaningful and less duplicated alt text on the site images
- Trust and governance signals – recognising the role of the site’s strong branding to demonstrate authenticity
- SEO – suggesting the site’s page titles and meta descriptions were reviewed for relevance
The second part of the response outlined a plan for a full audit. The plan began by posing three questions to be answered:
- What are the primary goals? e.g. increase visitors, boost traffic, showcase outputs
- Who are the priority audiences? (with suggestions of the different groups)
- What’s in scope? (suggesting either the entire site or specific sections)
The response then provided a suggested series of 7 steps to follow to complete the audit:
- Inventory and crawl (with detail of the inventory to start with and the tools to complete the crawl)
- Editorial quality review (with detail of criteria to score each page – such as inclusivity, accuracy, audience-fit etc)
- UX and IA review (with suggestions to evaluate navigation labels and connections between content and pathways to achieve top tasks)
- Accessibility (with suggestions of accessibility checking tools to use as well as manual checks to complete)
- SEO and technical (with suggestions to check data points like metatags, internal links, robots files and performance)
- Analytics (with suggestions to use analytic data such as bounce rates, page views, site searches etc)
- Recommendations and roadmap (detailing prioritisation measures to make the identified site changes)
It rounded off suggesting a template structure for the content inventory (and offering to create this template as a Google Sheet), an idea for a scoring rubric, and reiterating different tools to use (naming products like Screaming Frog, Google Analytics and SEMrush). To conclude, it offered to do the audit, requesting answers to the three initial questions and asking for access to a sitemap, external tools and governance documents (such as style guide and brand voice guidelines).
Reviewing the response, which was presented as text content, I found it a bit too much to scan and digest, and putting myself in the place of someone facing content auditing with some reluctance, it felt that this large amount of detail could be off-putting to even start the process. I could appreciate the relevance of suggesting of external tools, but knowing that each of those would require setting up accounts and logins, as well as going through a process to integrate with Open AI, it had the effect of making me mentally push content auditing to the bottom of a to-do list.
Likelihood to convince me to audit: 6/10
Anthropic’s Sonnet 4.6’s response gave structured guidance which was easier to follow
The output from the Anthropic model stood out compared to the others from ELM since it was not just plain text, instead, it had been structured with headings, tables and icons. This made scanning and digesting this response much easier than the others.
The response was structured into 10 sections, as follows:
- Site overview – table summarising the site name, owner, primary purposes and date of confirmed content
- Information architecture – list of the top-level navigation elements with an assessment of what worked well and issues identified in a bulleted list
- Content inventory – table of pages, with noted audiences, details of last update (if known) and inferred status (on a red, green and amber scale)
- Calls to action – list of CTAs on the homepage, with assessments and recommendations
- Content freshness – table containing list of most recent items in each section and related assessment
- Accessibility – list of accessible features that were present, table of items that needed checking and recommended accessibility tools to make the assessment
- SEO and technical – table containing an SEO assessment, considering elements like meta descriptions, structured data, mobile responsiveness – each with an allocated status, associated notes and combined recommendations
- Content quality and tone – list of observations and recommendations
- Priority recommendations summary – table of priority actions with an effort/impact score
- To complete the full audit I need – list of requirements to complete the full audit (including sitemap, CMS page list, Google Analytics data, etc.)
Content within each of these sections was comparable to that included in the GPT 5 response, but in the Sonnet 4.6 response, these sections referred directly to information from the site in question – for example, instead of making general assessments of site aspects like navigation and content coverage, it contained direct detail about the site itself with mention of specific pieces of content, noting what was already known (for example, referring to issues noted in the accessibility statement). Arranging this in tables with use of icons and red, green and amber priority indicators made it easy for me to pinpoint what to address and with what urgency, and having this consolidated in the final table was easier to parse than reading it in text format (as in the GPT 5’s response).
Reviewing this response from Sonnet 4.6, it was the best of the bunch from ELM, it contained enough detail to help me get started but not so much that it was overwhelming. Putting myself in the position of a time-poor site owner unsure where to start, I had clear pointers of the priority areas and a mix of quick wins and bigger tasks to choose from.
Likelihood to convince me to audit: 7/10
Three sets of two side-by-side screenshots, showing the output from Open AI GPT 5.5 (on the left) compared to the output from Anthropic’s Sonnet 4.6 (on the right) showing Anthropic’s more structured and visually appealing approach to presenting the responses
As well as ELM, I experimented with Chat GPT and Claude to provide content audit help
Models within ELM can also be accessed through interfaces provided by AI products. Two of the best-known AI products are Chat GPT (by Open AI) and Claude (by Anthropic). I was curious to see the responses each of these products would provide in response to my query asking for content auditing help so I prompted each of them in the same way. In both cases I used the free versions.
Chat GPT gave a suggested approach and produced initial findings
The response from Chat GPT was comparable to the output from ELM choosing an Open AI model, but the response was structured and annotated with tables and icons, in a style similar to the Anthropic response. It began with a suggested audit approach which set out different areas to consider and related questions. In the next section (headed ‘Initial observations’) it contained an analysis of the site itself, which included site strengths and highlighted the following six areas for improvement on the site in question:
- Homepage is quite academic – pulling out some audience questions that would be relevant to be answered on the homepage instead of the content that was there
- Navigation could be more task-focused – noting that the current navigation was driven by organisational structure rather than tasks users would want to complete
- News archive – picking out the need for recent articles
- Calls to action – acknowledging that many of these are worded to provide information not prompt an active response
- Content consistency – advising a uniform page structure for easier reading
- Audience segmentation – recommending clearer delineation between groups of site users
The response concluded with an example audit table and some content recommendations, organised under ‘Keep’, ‘Improve’ and ‘Review’.
Reviewing this response as a whole, I liked the up-front suggestion of an auditing approach and the example audit table which was then supplemented with detail about the site itself and recommendations relevant to different aspects of the site in question. This helped me visualise what the audit outputs could look like, and I could see ways to get there.
Likelihood to convince me to audit: 7/10
A screen showing the output from Open AI after being prompted for help content auditing. The suggested audit approach laid out in a table with columns for the different areas of auditing and associated questions to ask in the right-hand column
Another screen showing the output from Open AI after being prompted for help content auditing. An example audit table is laid out with rows for each URL, and columns containing associated purpose, audience, quality and recommendations.
Another screen showing the output from Open AI after being prompted for help content auditing. Content recommendations are laid out with colour coding for ‘Keep’, ‘Improve’ and ‘Review’.
Claude presented interactives to actively take me through an audit process
Anthropic’s Claude interface took a different approach to the other AI products, immediately asking about the main goal of the audit with options to pick. I picked ‘full inventory’. It then asked the desired breadth of the audit with another set of options, from which I picked ‘full site crawl’. The next screen asked me about the output format, and I picked ‘spreadsheet’.
From top to bottom, the interactive screens produced by Claude asking about the main goal of the audit, the breadth of the audit and the format for the audit output. Each has multiple choice answers to select
After approximately 2-3 minutes, with a running commentary of what was being done, Claude presented a list of ‘headline findings’ of bugs, orphaned content, out-of-date content and broken content (dead links, incomplete calls-to-actions etc). It also produced a downloadable Google Sheet with two tabs – the first containing a summary of the issues to be fixed, their locations and a reason why they needed to be fixed, the second with a full inventory, with one row for every URL in the site and columns logging the page title, content type, date, status (with colour-coded options: ‘Keep’ (green), ‘Update’ (amber), ‘Rewrite’ (dark amber), ‘Review’ (blue) and ‘Critical’ (red)) and recommended actions for each page.
Screenshot of the audit output spreadsheet produced by Claude, showing one row per page URL and different columns logging content types, date, status and recommended actions.
Reviewing this response, it was by far the most effective at making me feel I was starting the content auditing process and supporting me through it. Being presented with the opportunity to answer questions upfront meant I could control how the audit went, and having a spreadsheet created for me to review saved a lot of effort making this from scratch by scraping the site for a sitemap and page metadata.
Likelihood to convince me to audit: 9/10
Conclusion: If you’re putting off auditing your website, try getting AI to help
If you’re looking to improve your website, doing a content audit is one of the best steps you can take – working out what you have, what’s working and what’s not will put you in a strong place to move forward and make positive changes. Depending on how well you know your site, and how much time and resource you have to spend on it, you may appreciate some help to get started auditing its content.
From my quick experimentation with the different AI tools available I learned that, with the exception of the small non-reasoning model, each AI product offered something to help initiate the content auditing process of a website. At a basic level, asking AI to look at your website will pull out something you may not find yourself such as an out-of-date page, an irrelevant CTA or an accessibility glitch. AI can also take a zoomed-out look at your site that you may be too close to adopt yourself – such as reviewing your navigation pathways, appraising your audience segmentation or analysing the consistency of your content voice and tone.
If you’ve never audited a website before, AI can offer a lot of guidance to get you started, and whether you choose to do it yourself taking recommendations from the AI on tools and/or processes or whether you prefer to hand the heavy-lifting to AI to get your audit spreadsheet started – leaving you to deal with the more nuanced decisions that require specialist knowledge and experience, these tools can save you time and effort and may just be the nudge you need to do this all-important website maintenance task.