❌

Normal view

There are new articles available, click to refresh the page.
Before yesterdayMain stream

Just do it for me? Balancing human and AI input in content improvement

6 November 2026 at 14:07

Working with student interns and contributing to the Drupal community, I’ve experimented with AI in various  tools designed to support content improvement. I reflect on what I’ve learned about peoples’ preferences for engaging with AI tools and consider wider design challenges for web content to support co-browsing.

Writing good quality, user-centred content has always been a challenge, and accordingly, a suite of tools evolved to address this; Hemingway, Grammarly and many more. With their ability to learn patterns and predict word sequences, the advent of LLMs provided a new direction for content improvement. Where previous tools had used fixed rules, algorithms and formulas to flag content problems and score sentences, LLMs changed the playing field as they were able to be prompted to rewrite and create text and to adapt words and sentences not only in line with fixed rules, but also aligned to different interpretations of style.

The UX Service’s content design course, Effective Digital Content, and our ongoing Content Improvement Club meet-ups are both designed to help staff publishing digital content for the University. It’s been encouraging to see a positive uptake for both training formats, however, we realise we can do more to help staff publish the best quality content they can, and to this end, for the past few years it’s been a privilege to support a series of interns in the UX Service to look at how we can innovate and apply AI to the persistent challenge of writing good quality, user-centred content.

Thinking beyond – embracing the spirit (or grasping the nettle, depending on how you look at it) of AI innovation, it’s sensible to note that, more broadly, the browsing landscape of the web is increasingly shifting to include AI agents as well as humans, giving cause to consider what using AI to design content to be consumed by AI might look like. That’s the subject of another blog post (or several).

For now, as I begin another phase of work with a new student in the role of AI and UX Innovation intern (Viktor Sharha), it felt timely to pull together what I’ve learned so far.

Designing, creating and editing content happens in various ways

When designing any user-centred assistive features, a good place to start is the current workflow or user journey people adopt to achieve the task in question. In other words, to look at the stages content editors currently go through to make sure their content is as good as it possibly can be.

One of my earliest research tasks when I joined the University was to usability test the Editorial Style Guide, to find out how publishers used it to keep their content consistent in EdWeb. This work gave me insight into the type of workflows publishers followed to get compliant content published. Some publishers worked in Microsoft Word, using the in-built spelling checker for typos, and cross-checking against a printed-out copy of the style guide for University-specific errors before pasting content into the EdWeb body text. Others went straight to EdWeb, checking their content line-by-line with a PDF version of the guide. Others relied on colleagues giving their work the once-over before pressing publish. Some used Grammarly and Hemingway, some didn’t.

Overall, there wasn’t a dominant content-publishing workflow, with unified friction points waiting to be alleviated through a standard user-centred design process. Instead, there were varied approaches, habits and ways of working which people had adapted over time to suit their specific circumstances and to address the areas they found difficult as individuals. Tools were used at different stages of the process, and to a greater or lesser extent depending on personal preferences.

Therefore, designing content improvement tools isn’t one-size-fits-all

It follows that, since people follow different pathways to create, design and edit content, tools to assist this process need to be developed in different ways to suit varied preferences. Since 2025, interns in the UX team have prototyped our own different content improvement tools to test with publishers, and we’ve also tested open-source tools that exist already, developed within the Drupal community. From our testing and research, we found pros, cons and nuances associated with different tools. The summary below pulls together detail of the tools and how they compared.

Tool 1: Drupal Editoria11y – no AI, flags errors in the editing interface

Editoria11y is a Drupal module built by John Jameson from Princeton University that automatically checks content inside the editorial interface. It was originally designed to catch accessibility issues, but it can be configured to check content against other rules too, such as institutional style guide conventions.

What it does and how it works: Editoria11y checks content as authors type, and also on previews and published pages. It includes an indicator bar to show how many issues are on the page and has inline markers to show where each issue is, as well as a dashboard summarising site-wide errors. When an issue is opened, a pop-up modal explains the problem and suggests a fix.

What worked well: In tests, publishers were able to make sense of the inline markers and understand how these related to the indicator bar for the whole page. They liked the tracked change visualisation in the pop-up modals, and were able to work through successive modals, accepting or ignoring the suggested changes.

What needed improvement:  Some publishers found the number of markers overwhelming, and since the pop-ups covered the text, it was difficult to judge each recommendation in context. In the pop-ups, some of the rule descriptions used formal grammatical terms (such as ‘ordinal suffixes’) which were unfamiliar to some publishers meaning they were confused about the problem to be fixed. Since rules were applied mechanically, there were instances where errors were missed, or where awkward fixes were suggested (such as swapping ‘1st floor’ for ‘1 floor’ due to straight removal of the ordinal suffix) – these degraded publishers’ trust in the tool.

Read more about what we found testing Drupal Editoria11y

Embedding the rules: What we learned UX testing a style guide helper tool built with Drupal Editori11y

Tool 2: ELM-powered content design chatbot – with conversational advice

This tool was developed by student Mostafa Ebid during a 2025 summer internship. It connected ELM, the University’s AI platform, to EdWeb2, so publishers could get help with their content by conversing with the University’s suite of LLMs without leaving the Drupal editing interface.

What it does and how it works: This tool sits in a chat sidebar next to the main Drupal editorial interface and offers three modes: Write Content, Design Content and Proofread. In response to a user prompt to improve content, it activates an orchestration of agents (each with specific tasks such as checking the style guide, structuring pages, profiling audiences) which read the page content and act on it accordingly, publishing the revised content in the chatbot window.

What worked well: In tests, publishers understood how the tool could be used as a writing partner, and were impressed with the way it restructured content with headings, bullet points and calls to action and hyperlinks.

What needed improvement: When faced with the empty chatbot interface, some publishers weren’t sure where to start, and they didn’t notice the three modes to work on content. Some were also unsure of the sources the chatbot was using for its revised content and skeptical that the revised content contained hallucinogenic content. They would have appreciated tracked changes to be able to pinpoint what had changed and found the long thin sidebar chat interface awkward to review the updated text. The chatbot was also limited to reviewing body text content, whereas publishers needed it to review content in other page components used in Drupal.

Read more about what we found developing and testing the ELM-powered content design chatbot:

Initial insights from UX testing our Drupal AI content assistant tool

Tool 3: QuillMark v1 and v2 – some AI, clusters errors with correction suggestions

QuillMark was an AI-enhanced tool which worked in Drupal to check content against the University’s Editorial Style Guide. Shlok Gupta built and tested two iterations of Quillmark during his 2026 summer placement as AI and UX Innovation Intern.

What it does and how it works: QuillMark sits in a side panel of the Drupal editorial interface and scans content on the page in two stages. The first stage uses code-based pattern matching (without AI) to catch straightforward errors. The second stage brings in AI to find deeper issues (such as passive instead of active voice) and to suggest fixes. Findings from both stages are displayed in the side panel – each finding describes the problem, links to the relevant rule in the style guide and offers an ‘Apply fix’ button (which can be reverted). Findings are grouped by category, and this panel also includes an indicator of the environmental impact and cost of the scan.

What worked well: In tests, publishers understood that the tool was there to help them correct content on the page, they liked the environmental and cost indicator, the suggested fixes and the links to the style guide reassured them that the tool was applying University-specific rules.

What needed improvement: Publishers reported feeling overwhelmed by the number of findings on a single page and they wanted a way to deal with related findings and fixes in bulk, to avoid going through them one-by-one. Related to this, they wanted

Read more about what we found developing and testing the QuillMark tool:

QuillMark, iterated. How user research helped us evolve an AI-enhanced Drupal style guide assistant

Tool 4: Drupal AI Content Review – full AI, with a plan provided to improve content

As part of the Drupal AI Initiative, I’ve contributed to shaping the self-explanatory Drupal AI Content Review module. This module plugs into a Drupal site, references content rules and good practice from context items in Drupal Context Control Center (within the same site) and uses the configured AI provider to review site content

What it does and how it works: The feature reads the page content and checks it against criteria set by the site owner, such as tone and voice guidelines, inclusive language rules and target readability levels. These rules are stored in the Drupal Context Control Center, along with examples of good and bad content. The review gives the page a score for each criterion and recommends fixes to improve it. Accepted recommendations become an improvement plan, which is applied in the editorial interface. Re-reviewing the page shows the updated scores, with the option to keep making improvements. Over time, the rules can be refined by adding more examples of good and bad content to the Context Control Center.

Read more about Drupal AI Content Review on Drupal.org

Read more about the Drupal Context Control Center on Drupal.org

View a demonstration of AI Content Review in a mocked-up university website as part of the Drupal AI demo on the Drupal Forge platform

What worked well:  In tests of this module within the AI demo, people liked seeing the red, amber and green indicators against page content for an easy visual highlighting areas to be worked on, and they liked being able to select a checklist of recommended improvements to apply to improve their content.

What needs improvement: The screens explaining the recommended improvements were difficult to scan read, and it was also difficult to compare the original text and the text with the recommended improvements side-by-side (despite colour shading to denote which was which). The stage where the recommendations were turned into an improvement plan felt like an unnecessary extra step, it would have been better to have implemented the recommendations directly. Also, it was presumed that implementing the plan would have corrected all errors resulting in a score of 100%, without the need for further work and re-review. Finally, it wasn’t immediately clear how to use the Drupal Context Control Center and the AI Content Review together.

When designing AI-enhanced tools, consider three AI design maxims

Taking on board variations in blending human effort and automated operations, there are three approaches to designing AI features and AI-assisted workflows worth considering. These are defined below, with ideas for how they may apply to content improvement.

1.      Human-in-the-loop

In this set-up, humans typically share the task-load with AI – actively working alongside an automated operator to supervise or influence actions, and to approve AI decisions before they happen (ensuring AI outputs are not released without the green-light from a person).

Human-in-the-loop tends to be adopted when:

  • there is a need for human judgement to get something correct
  • the consequences of an error could be high-profile or damaging
  • trust in an automated tool is still being built, and the boundaries of its capabilities are still being realised
  • the goal is for the system to facilitate shared learning between the person and the automation

For content improvement, human-in-the-loop is appropriate in these scenarios:

  • ensuring correctness of important factual data – for example, prices, dates, contact details, terms and conditions
  • checking the tone of sentences in line with inclusive language guidance
  • verifying accuracy of high-stakes published content such as legal information, policy wording or written instructional content
  • rewriting complex topics in plainer language
  • writing meta descriptions or page summaries to be accessed by search mechanisms
  • translating content into other languages

2.      Human-on-the-loop

These systems involve the AI or automator acting on its own, with the human reviewing and spot-checking at intervals, monitoring output quality over time, and intervening to correct failures periodically, where required.

Human-on-the-loop tends to be adopted when:

  • there is a high volume of low-risk tasks to be completed, which would be tedious for humans to do themselves
  • consequences of errors are minimal
  • there’s a need for tasks to be completed quickly, and including human-input at each task would be too time-consuming
  • consistency is required across tasks, and there’s a risk of human error introducing variability

For content improvement, human-on-the-loop is appropriate in these scenarios:

  • fixing straightforward style rule violations such as correcting spelling errors
  • categorising and tagging text content (for example, to facilitate its discovery in search)
  • scanning content for readability and accessibility, to flag or report areas to be addressed

3.      Human-out-of-the-loop

In this arrangement, the human sets up the AI to act autonomously, and leaves it to proceed, with no routine review.

Human-out-of-the-loop tends to be adopted when:

  • there are very low-risk, behind-the-scenes tasks to be completed where errors have little or no consequence and can be fully reversed

For content improvement, human-out-of-the-loop is appropriate in these scenarios:

  • autocorrecting typing errors
  • detecting content duplication
  • auto-optimising media content to defined parameters

Choice, pacing and transparency are constant design requirements

Testing four different tools has showed that options always need to be available, whatever balance of human and AI the tool includes. People approach AI differently based on preferences and past experiences. Some love nothing more than a good grappling of a sticky sentence, some just want an agent to do the work for them, and many sit somewhere in between, happy to have some AI help as long as they have the final say.

Alongside humans in, on and out of the loop, it’s important to remember the fourth option: human only, where people can choose to do their content unaided, using the methods they relied on before AI came along. AI shouldn’t be forced on anyone, so all tools need to come with a ‘no thanks to AI’ option.

AI is a fast-evolving and non-deterministic technology, so at any point in time it may work differently to the way it did before.  Responding to challenge from a design perspective, it’s important to factor in pacing. People need to be taken through how a tool works, and the options available. They need time and space to experiment and try it out, so they can learn for themselves how it can work for them. Pacing not only matters in the way tools are introduced, but also in the way their outputs are presented, to avoid the overwhelm that comes when AI does the heavy lifting all at once, and so people know what changed and how, and they have a way to accept, decline or edit what it produced.

If people don’t trust an AI tool, they won’t use it. Earning trust requires transparency, explainability, relatability and reliability – so AI tools need to be upfront about what they can and can’t do, with clear, intuitive interfaces and labels. Reliability will always be hard to promise given AI is probabilistic, so instead of accuracy guarantees which may fall short, a better approach is to design for experimentation, so people can find their feet and make their own judgements.

Designing for humans and AI together – metadata, markdown and MCPs

Looking at the broader digital landscape, the challenge widens from designing tools that help humans make content to designing websites with content ready for both humans and agents to use.

In co-browsing, people are increasingly arriving at a website with an agent alongside them, handing off some tasks while keeping control of others and wanting to see what their agent is doing. That means firstly deciding which tasks belong to humans and which one to AI, and secondly designing for each. Humans need intuitive interfaces. Agents need data they can read and act on without ambiguity:

  • metadata, such as schema markup that tells an agent what a page is about, who it’s for and how it connects to other content
  • markdown versions of pages and llms.txt files, which give agents clean, structured text without layout and navigation getting in the way
  • MCPs (Model Context Protocol servers), which let sites expose what they can do as well as what they say, from simple actions declared directly in HTML to more complex ones written in JavaScript – in preparation for use by the emerging WebMCP browser standard.

Labels matter for humans and agents alike, because words are becoming a currency that machines read as well as people. Alignment on terminology facilitates the creation of a single source of truth, to prevent the human experience from drifting from the agentic experience. On another level, humans need a way to consistently communicate with AI – to give AI the rules and instruct it and to train it with new insight. ‘Rosetta stone’ mechanisms like the Drupal Context Control Center will play an important part in designing for human and AI interaction and collaboration and I am excited to keep working and researching in this area to address these evolving design challenges.

 

QuillMark: The Technical Details Behind My Drupal Style-Guide Assistant

I previously introduced QuillMark, a tool I built to help web publishers check content against the University’s editorial style guide. In this post, I explain the technical design and methodology behind the prototype.

Introduction

My name is Shlok, and this summer I am working as an AI and UX Innovation Intern in the Information Services Group (ISG). During my internship, I have been developing QuillMark, a prototype tool designed to help web publishers check their content against the style guide. This post outlines the system design, methodology and AI pipeline behind the prototype, including how deterministic checks, language models and a MapReduce approach work together to improve reliability while keeping publishers in control.

 

QuillMark highlights violations in your existing text rather than generating an entirely new version. This:

  1. avoids the risk of AI making unnecessary changes to unrelated parts of the page, which would require a manual audit each time
  2. allows publishers to clearly see each violation and accept or reject suggested changes
  3. reduces output tokens, lowering both hallucination risk and cost

 

Problem: The style guide contains a large number of rules, around 60–70. If all rules are sent to the LLM at once, the model may not check the text against every rule reliably.

 

Research: https://openreview.net/pdf?id=R6q67CDBCH

 

Solution → MapReduce (Divide and Conquer: Map → Collapse → Reduce)

 

System Design and Methodology

 

Pipeline:

  1. Deterministic checks to flag blatant violations
  2. Fine-tuned small model flags likely obvious violations
  3. Large model audits the final page deeply for complex deeper violations

 

The deterministic checks are used to flag “obvious” violations that are directly detectable using regex or code. For example, this could include the use of a forbidden word, or a spelling convention such as using “benefited” instead of “benefitted”. These issues can be detected using a direct search of the page and do not require AI.

 

The second stage uses a smaller fine-tuned model. This model would be fine-tuned on the Style Guide rules and training examples. It would again be used to detect relatively “obvious” violations, but ones that are not simple enough to be detected reliably through deterministic checks. These issues are not highly complex, so they should still be visible to a smaller model.

 

After the publisher fixes the deterministic violations and the violations found by the fine-tuned small model, there are likely to be remaining deeper and more complex issues that both previous approaches did not pick up. The final large model can then focus mainly on these more complex issues. This means it spends less attention on obvious violations and more attention on issues that are easier to miss.

 

This improves the process in two ways.

 

First, it allows the final large model to focus on deeper issues. If the whole page was passed directly to the large model from the beginning, it would likely flag many obvious violations first and might miss some subtler issues. By fixing the most obvious issues earlier, the large model can focus more on smaller, judgement-based, or easily missed problems.

 

Second, it may reduce cost compared with running the large model multiple times. One alternative would be to run the large model once, fix the obvious issues + some deeper ones it finds, and then run the large model again to find the remaining deeper issues. In the first run, the large model may spend much of its output on obvious violations. In the second run, after those obvious issues are fixed, it may be able to look more deeply. The proposed approach tries to achieve a similar effect more cheaply by using deterministic checks and a smaller fine-tuned model before the final large-model review.

 

This pipeline is shown in Figure 1.

 

Style Guide Violation Detection Pipeline
Figure 1: Style Guide Violation Detection Pipeline

 

Using MapReduce to reduce instruction-following degradation

Based on personal experience with large language models, as well as existing research such as the Curse of Instructions, we cannot rely on a large language model to follow a very large number of rules at once. In this case, the style guide contains around 60–70 rules. If all of these rules are passed to the model in a single request, the model may not check the page against every rule properly. It may follow some rules, ignore others, or miss violations because there are too many instructions to apply at the same time.

 

To solve this, I propose using a MapReduce approach, which is a divide-and-conquer technique: Map → Collapse → Reduce.

 

The rules would be broken into smaller chunks, with each chunk containing a fixed number of rules. Each chunk would then be sent as a separate request, asking the model to check the page only against that smaller set of rules. This allows the model to focus on fewer rules at a time and check them with more attention.

 

After all the chunks have been processed, the responses would be gathered together and combined into one report. This is the collapse stage. The combined findings would then be passed through another large language model request in the reduce stage. This final request would clean up the output by removing duplicate findings, resolving overlapping issues between chunks, and filtering out possible false positives or hallucinated violations.

 

This approach makes the checking process more reliable because it avoids asking the model to apply all 60–70 rules at once. Instead, each group of rules is checked more carefully, and the final report is produced by combining and cleaning the results. This helps ensure that all rules are checked with more equal rigor, rather than relying on the model to remember and apply every rule in one large prompt.

 

The MapReduce approach is shown in Figure 2.

 

MapReduce Approach for Style-Guide Rule Checking
Figure 2: MapReduce Approach for Style-Guide Rule Checking

 

Update: Large-Model Cleanup from the Reduce stage was excluded from the prototype as the probability of duplicates occurring is low and the cost of such a cleanup is relatively high with minimal benefit.

 

Read more about the technical system plan: https://blogs.ed.ac.uk/website-communications/building-quillmark-testing-a-drupal-style-guide-assistant-with-web-publishers/

Watch a video demo: https://media.ed.ac.uk/media/t/1_yozad8ie

 

Future enhancements:

  • Strengthen Deterministic Regex-matching with more training data (web pages)
  • Audit deterministic findings using a cheap, local AI

Building QuillMark: testing a Drupal style-guide assistant with web publishers

I introduce QuillMark, a Drupal prototype designed to help web publishers check content against the University’s editorial style guide. I reflect on how I developed and tested the tool with publishers, focusing on what their feedback revealed about usability and how it will shape the next version.

Introduction

My name is Shlok, and this summer I am working as an AI and UX Innovation Intern in the Information Services Group (ISG). During my internship, I have been developing QuillMark, a prototype tool designed to help web publishers check their content against the style guide. This blog post explains the problem I was trying to solve, how I built and tested the prototype, what I learned from publishers and what I plan to do next.

 

The problem

Our style guide contains more than 60 rules covering areas such as spelling, punctuation, tone, formatting and terminology. Previous research showed that web publishers found it difficult to remember and apply every rule while writing and editing content.

 

This is understandable. Publishers are often working under time pressure, and checking a page manually against dozens of rules adds a significant cognitive burden. Even experienced publishers can miss small details, particularly when they are concentrating on the meaning and accuracy of the content.

 

I began exploring whether a tool could make this process easier. The aim was not to replace publishers’ judgement or rewrite their content automatically. Instead, the tool would identify possible style-guide violations, explain the relevant rule and allow the publisher to decide whether to apply or dismiss each suggestion.

 

This became QuillMark.

 

Read more about the Style Guide

 

Why I built the prototype in Drupal

Drupal is where publishers already create and edit web content, so it made sense to build the prototype directly into that environment rather than create a separate tool.

Before starting development, I attended Drupal in a Day to understand the platform, its content-editing interface and how a custom tool could fit into an existing publishing workflow.

 

Building QuillMark within Drupal meant publishers could check their content without copying it into another system. It also allowed me to test the tool in a realistic environment, using the same types of fields, buttons and interactions that publishers encounter in their day-to-day work.

 

Read more about Drupal in a Day

 

Planning the checking process

Before writing the prototype, I planned the system in detail.

 

One of the main design decisions was that QuillMark should highlight individual issues in the existing content instead of generating a completely rewritten version. Rewriting an entire page could introduce unnecessary changes and would require the publisher to audit every sentence. Showing individual findings makes it clearer what the tool has identified and keeps the publisher in control. This also prioritises the HITL (Human-in-the-Loop) framework, which I intend to use in my AI-based projects to ensure that people remain involved in reviewing decisions made by AI.

 

The system design proposed three types of checking:

  1. Deterministic checks for clear violations that can be identified using code or regular expressions, such as prohibited words or spelling conventions.
  2. A smaller AI model for relatively straightforward issues that require more context than a simple text search.
  3. A larger language model for more complex, judgement-based rules involving areas such as tone, structure or plain language.

 

The plan also used a MapReduce-style approach. Rather than asking a language model to apply all 60–70 style-guide rules in one prompt, the rules would be divided into smaller groups. Each prompt could then concentrate on a limited number of rules before the results were combined. This was intended to reduce the risk of the model overlooking instructions because it had been given too many at once.

 

The original design included an additional AI step to remove duplicate findings and resolve overlaps. I left this out of the prototype because duplicates were expected to be relatively uncommon, while the extra model request would increase cost and complexity.

 

Read more about the technical system plan: https://blogs.ed.ac.uk/website-communications/quillmark-the-technical-details-behind-my-drupal-style-guide-assistant/

Coding the prototype

I developed the prototype iteratively, beginning with a small number of style-guide rules and expanding the checks once the basic workflow was working. For development, I used Codex. I gave it my plan and asked it to design the architecture. After reviewing it, I asked it to proceed with the implementation.

 

The first stage used deterministic checks for issues that could be found reliably without AI. These checks searched the content for recognisable patterns and returned the location of the issue, the relevant style-guide rule and, where appropriate, a suggested correction.

 

The second stage used a large language model to identify issues that depended more heavily on context. I divided the rules into smaller prompt groups so that the model could focus on a manageable set of instructions during each request.

 

The prompts asked the model to return structured findings rather than a rewritten page. Each finding needed to include enough information for QuillMark to:

  • identify the relevant content;
  • explain the problem;
  • show the related style-guide rule; and
  • suggest a possible correction.

 

Publishers still had to approve or dismiss each finding. This human-in-the-loop approach was intentional. Style rules can depend on context, and an automated suggestion will not always be appropriate. QuillMark was designed as a decision-support tool, not an automatic editor.

 

Watch a video demo: https://media.ed.ac.uk/media/t/1_yozad8ie

 

Designing the usability test

Once the first iteration was complete, I adapted an existing usability-testing template to create a test for QuillMark.

 

I conducted three sessions with web publishers. Participants were asked to work through a series of tasks covering the main parts of the prototype, including running the checks, understanding the two stages, locating an issue in the editor, reviewing a rule and applying or reverting a suggested fix.

 

I observed how participants used the interface, where they hesitated and whether the information on screen matched their expectations. I also asked how they might use the tool as part of their normal publishing process.

 

The purpose was not only to find technical bugs. I wanted to understand whether the concept itself was useful and whether the interface communicated how the tool was intended to work.

 

What publishers told me

The publishers thought the tool would be useful as a final check before publishing. They said they would still use their own judgement and would not automatically accept every suggestion.

 

Most of the feedback was about the usability of the tool. Some parts were unclear, such as the difference between the two stages and some of the button labels. There was also too much information on the screen, and some issues were repeated. The publishers also wanted to see what a suggested fix would change before applying it.

 

What I learned

The testing suggested that the core idea is useful, but the user experience needs to become simpler.

 

Publishers do not necessarily need to understand which checks use regular expressions and which use an AI model. They need to know what issue has been found, why it matters, what the proposed change is and what action they can take.

 

The sessions also reinforced the importance of designing for scanning. More explanation does not always create more clarity. In a publishing workflow, concise labels, clear states and well-grouped findings may be more valuable than displaying every piece of supporting information at once.

 

Next steps

My next step is to refactor and review the prototype code before making changes based on the usability feedback. This will include reviewing the AI-generated code to check whether each section is needed, whether the same result could be achieved more simply, and whether any parts repeat code that already exists. I will compare the generated code with the rest of the codebase so that it follows the same structure and does not add extra functions or files without a clear reason.

 

Where the code is longer than needed, I will simplify it by removing repeated checks, combining similar sections, and reusing existing functions. I will also keep individual files to a manageable size, generally no more than 1,000 to 3,000 lines depending on the file. Larger files will be split where this makes the code easier to follow, and hardcoded rules will be moved into the database where possible.

 

The next iteration will focus on:

  • clarifying or simplifying the two-stage workflow;
  • reducing repeated and overwhelming findings;
  • improving button labels, status messages and colour distinctions;
  • previewing changes before they are applied; and
  • making the interface more concise and easier to scan.

 

Drupal In A Day: What we learned (and what we still want to learn): reflections from the UX team

Drupal In A Day (DIAD) is an in-person training event, designed as a beginner-friendly hands-on introduction to Drupal, the open-source content management system. When the University hosted DIAD, several of the UX team took the chance to attend.

As content management systems go, Drupal is one of the mainstays. It’s been around for 25 years and is particularly famed for its robustness, reliability and flexibility – so much so it’s trusted to power websites of enterprises, governments and higher education institutions around the world. It’s open-source so there’s no proprietary lock-in – the functionality, features and innovation available in Drupal comes from a thriving worldwide community of contributors – as individuals, agencies and organisations.

One of the best places to learn about what Drupal is, what it stands for and what it aims to achieve is the blog site of its founder, Dries Buytaert.

Dries Buytaert blog

In 2025, Hilmar Kári Hallbjörnsson taught the first Drupal In A Day on the final day of DrupalCon Vienna.  Hilmar had been teaching Drupal to students at the universities of Reykavik and Iceland for many years and felt strongly that teaching Drupal to new generations, to inform them of its capabilities, and to inspire them of its potential was something the Drupal community needed, as a way to nurture the longevity and sustain the future of Drupal. Through a tremendous effort, he made the first Drupal In A Day happen in Vienna in October 2025. It was very well-received, successfully establishing the Drupal In A Day format going forward.

Read more about the first Drupal In A Day in Hilmar’s blog post from 2025

Drupal in a Day: Vienna 

Hilmar travelled to Edinburgh to deliver Drupal In A Day with our own Web Development Team Manager, Gareth Alexander at the start of June 2026.  Nick (Senior Content Designer), Shlok (AI and UX Innovation Intern) and Hannah (Digital Content Style Guide Intern) from the UX team signed up for the event along with other LTW interns and staff from the wider University with an interest in learning about Drupal. Here, they reflect on their experiences of the day.

Nick’s reflections 

I attended this training to bring my understanding of Drupal up to date. The last time I set up a Drupal site was in the early 2010s, and a lot has changed since then. With Drupal providing the backbone to EdWeb 2, I wanted to get to grips with some of the terminology and concepts that underpin conversations within our team about what our central CMS can do. 

I was particularly interested to learn more about Drupal CMS, a new service that helps you set up a Drupal site more quickly and with less need for technical understanding. 

After getting the required software set up on my laptop (thanks Kirsten), I clicked along with Hilmar and Gareth as they walked us through the various sections of Drupal CMS. We worked through steps to create a new content type and we created different Views to display the same content. We tweaked image formats. We also learned how to attach tags to a piece of content and how these tags can act as a filter for overview pages. 

By the end of the day, I hadn’t suddenly become a Drupal expert. But I did have a better understanding of how EdWeb 2 works behind the scenes. In particular, I felt like I knew more about how EdWeb 2 takes the content you enter when you create a page and presents this in different ways elsewhere on a site. 

Alongside that, I would now feel more confident in setting up a new Drupal CMS site, and I now have a better understanding of what this system offers in comparison to other ways of operating a website. 

Finally, the day gave me a better sense of how the community aspect of Drupal is central to how this project keeps going. Hilmar and Gareth emphasised that there are various events in the calendar where people working with Drupal can meet up and collaborate. That’s a powerful message: that anyone is invited to learn more, become part of the community and contribute to how this system works. 

Shlok’s reflections

Before starting my internship, Drupal was a completely new concept to me. I did not really know what a content management system was, how Drupal worked, or why it was used by universities and other large organisations. Because of that, Drupal In A Day was a really useful introduction for me. 

I thought the format worked well because going through everything in one day helped keep a flow of information. It was quite intense and sometimes fast-paced, but that also meant we were able to cover a lot of concepts in a short amount of time. I would say I was able to follow around 70% of the session, which felt like a good start considering Drupal was completely new to me. 

One of the parts I found most useful was the introduction to Drupal itself. It helped me understand what Drupal is, what a CMS is, and why the University uses it. I learnt that Drupal is valued because it is stable, secure, flexible and open source, which makes it suitable for large and complex websites like those used by universities. This helped me understand why Drupal is still used by many major institutions. 

I also found it interesting to learn about the Drupal community and the different ways people can work with Drupal. Before the session, I assumed Drupal was mainly for developers. However, I learnt that people can build careers around Drupal in different ways. Some roles involve coding and development, while others focus more on design, content, user experience, training or project work. That helped me see Drupal as more than just a technical platform. 

One point that really stood out to me was when we were told that Drupal has a steep learning curve at the start. This was useful to hear because it made the difficulties I was facing feel expected. Since many people find Drupal challenging in the beginning, it was reassuring to know that not understanding everything straight away was normal. It made me feel more comfortable continuing to learn. 

During the practical parts of the day, I learnt about some of the basic Drupal features, such as creating and managing content pages. At first, this was quite confusing because the concepts were new to me. However, as the day went on, I started to become more comfortable with the terminology and the way Drupal is structured. 

Another part that I found very helpful was the follow-up assignment where we had to build something from scratch. After learning so much in one day, I think it was important to have a task that allowed us to apply what we had learnt. It helped me consolidate my knowledge, see which parts I had not fully understood during the session, and become more comfortable with Drupal before starting work on my internship prototypes. 

One thing I would have liked to learn more about was the command line and the use of terminal commands. We touched on some setup and development processes, but I think spending more time on what the commands do and how they fit into the Drupal workflow would have helped me. I would also have liked to learn more about the coding side of Drupal, especially how to build custom Drupal modules. This is particularly relevant to my internship because my work is focused on AI and innovation within Drupal. 

If I could suggest one change, it would be to make Drupal In A Day into a two-day or three-day format. The first day could stay mostly the same, giving everyone a broad introduction to Drupal. The second day could be a slower guided session where participants build something with support from the instructors, similar to the assignment we were given afterwards. A third optional day could focus more on coding and custom module development for those who want to explore the technical side further. 

Hannah’s reflections 

I signed up for the Drupal in a Day training with very little knowledge of the CMS beyond working with existing EdWeb 2 sites, such as when creating prototypes of Style Guide pages or writing blog posts. So, I was interested in increasing my knowledge of Drupal as well as gaining the ability to build a site. Before the training began, I was unsure how well I would follow the instructions as a beginner, but Hilmar, Gareth, and all of the members of the Drupal community who were present at the training were extremely helpful, informative, and patient when it came to giving assistance at points where I was confused or behind. 

Once I had overcome a technical error with my laptop and decided to use Drupal Forge, I was able to follow along with the instructions being provided and replicate the Artist biography pages that Hilmar and Gareth were demonstrating. The course was fast-paced and detailed, and while maintaining this pace was a challenge, it made the course engaging and the product of the day felt like a genuine accomplishment. 

With the knowledge that I gained throughout the training, I feel that I have a strong foundation that will allow me to continue to develop my skills in building a Drupal site further in the future. In particular, I think the example site that we were creating during the training was incredibly useful in that it allowed us to try out several different aspects of building a site with Drupal, such as different content types, tags, and images, as well as exploring some different design options as well. 

I was impressed to learn about how community centred Drupal is, and that contributions and developments to Drupal are made by users and members of the community. Gareth and Hilmar’s explanations of how this works really helped me to understand why Drupal is as adaptive and intuitive as it is (such as in its security and bug fixes), which is that these developments are a direct result of user experience. Furthermore, Gareth and Hilmar also emphasised the flexibility of Drupal, due to the ability to install ‘recipes’ that customise the functionality of sites that you are building. 

Overall, I think Drupal in a Day was an extremely useful and practical training session that has equipped me with the skills to build a basic site and further develop these skills whenever I have the chance.  

Emma’s reflections

When it comes to Drupal, I am largely self-taught – pretty much everything I know has been gleaned from reading, attending/watching sessions from Drupal events, and (predominately) pestering people in-the-know with my questions. When Drupal In A Day came to the University, I pondered whether I should attend. I’ve contributed to the community since 2022. I’m on the Drupal leadership team. Surely I should know Drupal by now? I took a split second to reflect and realise that you never fully know Drupal. There’s always something new to learn, something to challenge what you thought you understood, or something to clarify an area you weren’t quite sure about. Conscious of not taking a space away from a complete Drupal beginner, I opted to sit in on the event, to follow along, while working on other things.

The day combined practical exercises with general Drupal knowledge. Hilmar and Gareth began with the basics, covering key parts of the process to get started and familiarise with Drupal – like setting up DDEV, using Drupalforge and creating a Drupal.org profile. As the day went on, it was great to see quick progression to experiment with some of Drupal’s flagship modules, especially Drupal Views which holds huge power for presenting and displaying structured content in a range of adaptive ways.

Drupal CMS, Drupal’s low-code product provided the playground for learners. Having worked on Drupal CMS since it began, I found it heartening to see it being used to introduce Drupal to new audiences, to help them learn what Drupal has to offer and to help them start ideating on how they might use it. Adopting a UX perspective, I tuned into comments from fellow attendees about Drupal CMS, noting observations to follow up and logging areas for UX improvement to carry forward in my ongoing Drupal UX contributions.

Reflecting on the day as a whole, it helped attendees achieve what is often the hardest thing about Drupal: Getting started. All too often people new to Drupal can find it overwhelming, and they find when everything is possible it’s hard to choose a direction. Drupal In A Day sets learners on a path to start discovering Drupal and making it their own, in other words, planting a seed from which the community can continue to grow.

Embedding the rules: What we learned UX testing a style guide helper tool built with Drupal Editoria11y

Our Editorial Style Guide contains many conventions and we know from research that publishers struggle to remember to apply them. Could automation help? We experimented embedding style guide rules into a Drupal module that checked content against the rules in the editorial interface and suggested corrections when the rules weren’t followed.

As part of a concentrated effort to make it easier for publishers to apply our style guide rules, Mostafa Ebid, a student who came to work with us in summer 2025, installed Editoria11y, an open-source Drupal feature and configured it with selected rules from our University style guide. Applying this feature to a site with demo content, he tested it with University staff to see how well it worked, and how useful and usable they found it.

Read more about our ongoing style guide work:

Collection of blog posts about the Editorial Style Guide

Drupal Editoria11y is an open-source accessibility checker

Editorial accessibility ally (shortened to Editoria11y) is a module developed by John Jameson from Princeton University that enables automated content checking directly in the editorial interface. Originally created to help content editors catch accessibility issues, its configurable architecture makes it well suited to embedding other kinds of rules, such as institutional style guide conventions. The checks can run in real time as content authors type, and can also be applied to content in published pages and previews, to flag issues to be corrected.

Editoria11y can include more than 50 built-in content tests covering image alt text, link quality, heading structure, and general content issues. Flagged issues are noted at the page level by an indicator bar, which includes a count and a categorisation of the type of issues (default categories are headings and alt text). Through this bar it is possible to identify the precise inline location of the problematic content with visualisers. Each visualiser is associated with a modal that contains detail about the problem element and plain language tips explaining how to fix it.

In addition to the on-page alerts, it is possible to review issues across a site via a reporting dashboard which can be configured to log recurring issues or most problematic pages, as required.

Read more about Editoria11y on Drupal.org:

Editoria11y Accessibility Checker project page

We designed tests to learn if Editoria11y would be useful and usable to embed style guide rules

Editoria11y presented itself as a mechanism to enable web publishers to check their content against style guide rules, and we wanted to understand the kind of experience this provided in EdWeb2. In particular, we wanted to understand:

  • If Editoria11y could effectively check content against our style guide rules
  • How Editoria11y presented results of the checks in the editorial interface
  • If publishers could use Editoria11y to correct content that misaligned with the style guide

Moreover, we wanted to learn if the addition of Editoria11y to the EdWeb2 editorial interface improved the publisher experience or not.

We picked deterministic rules about dates and numbers for the tests

Like many Drupal features, Editoria11y is very flexible and customisable to fit a range of different use cases. Since it was new to our publishers, we were keen to avoid overengineering it, taking care to configure it to present the minimal information necessary to achieve our testing goals.

The University’s Editorial Style Guide contains between 60 and 70 separate rules, of which fewer than half are deterministic (in other words, can be applied directly without editorial judgement). It made sense to include deterministic rules in the tests, to enable us to accurately assess how effective Editoria11y was at checking them.

Rules in the dates and numbers section of the style guide seemed a good fit for the tests, so these were configured into Editoria11y.

We set up Editoria11y to display an indicator bar, visualisers, modals and a dashboard

The module was set up to display the indicator bar, the inline visualisers and the reporting dashboard. Content that contained sentences with dates and numbers was saved in draft in a demo interface which had Editoria11y applied. In the test scenario, participants were asked to assume they had been asked to review this draft content before it went live and to use a checker tool to help them with this.

Screenshot of the Drupal Editori11y tool in action, showing the yellow indicator bar at the bottom right of the interface, tallying the total number of style guide errors on the draft page

Screenshot of the Drupal Editori11y tool in action, showing the yellow indicator bar at the bottom right of the interface, tallying the total number of style guide errors on the draft page

 

Screenshot of Editoria11y showing on the left, the indicator bar at the bottom right of the page, and on the right, the editorial interface revealing the locations of the style guide errors (activated when the indicator bar is pressed)

Screenshot of Editoria11y showing on the left, the indicator bar at the bottom right of the draft page, and on the right, the same draft page, but revealing the locations of the style guide errors (activated when the indicator bar is pressed) and an example of the modal giving detail of the error and the suggested fix

 

Screenshot of Editoria11y showing on the left, the display with the indicator, and on the right, the reporting dashboard summarising all the errors

Screenshot of Editoria11y showing on the left, the display with the indicator, and on the right, the reporting dashboard summarising all the errors (which opened in another window in the interface)

Watching participants use Editoria11y, we learned what worked and what didn’t

Tests were carried out with seven participants. Observing how they made sense of the different parts of Editoria11y and interacted with it, we were able to draw conclusions about how useful and usable it was to support the context of web publishing in EdWeb2.

Participants understood the relationship between the indicator and the visualisers

The first part of the tests involved showing participants the draft content in the editorial interface with Editoria11y applied. All of them were able to understand that the number of style guide errors on the page were tallied in the indicator bar at the bottom right of the page, and that they could use the indicator bar to reveal precise locations of the errors on the page, together with a modal for each error, explaining the error and the suggested fix.

Placement of the visualisers and modals often made it hard to read the draft content

Taking an overview of the visualisers in the draft content, participants commented that they felt a bit overwhelmed to see so many. Some felt the placement of the question mark visualisers made it difficult to ascertain which parts of the content needed to be corrected, which was slightly helped by yellow outlines around the text. Placement of the modals masked the text beneath, however, meaning participants found it difficult to engage with the original context of the errors to be corrected to assess whether to accept the suggestion or not

Screenshot of Editoria11y showing a modal detailing the error about writing dates with ordinal suffixes

Screenshot of Editoria11y showing a view that some participants found overwhelming: indicator bar, question-mark indicators, yellow outlines and an open modal

Descriptions of the errors in the modals were not clear to some participants

Reading the information in the modals, some participants were unclear of the error it was pointing out. Specifically, in a modal describing the rule to write dates without including ‘th’ or ‘rd’ after numbers was written as ‘Write dates without commas or ordinal suffixes’. Some participants were unfamiliar with what ‘ordinal suffixes’ were, but seeing the suggested change presented as a ‘before and after’ tracked change with the error crossed out (in red) and the suggested change presented in green helped participants understand the proposed correction.

Screenshot of Editoria11y showing a modal explaining the rule to not include ordinal suffixes when writing dates

Screenshot of Editoria11y showing a modal explaining the rule to not include ordinal suffixes when writing dates

Participants were able to use the modals to correct some style guide errors

When they had read and understood the errors being highlighted, most participants were able to assess whether to accept the suggested fixes and apply them based on the information contained in the modal. Several participants entered into a flow of using the arrow keys in the modals to clicking through the different errors and accepting the fixes, especially when the fixes related to a recurrent error – for example, capitalising the first letter when writing days of the week. They appreciated a notification alerting them that the fix had been applied, which temporarily appeared at the top right of the screen, although some said they would have appreciated this notification to remain for longer as it was easy to miss. They were also unclear whether they needed to save after applying each fix, or if this could be done when they had worked through all of the fixes on the page.

Screenshot of Editoria11y showing a notification in the top right of the interface to confirm the fix had been applied

Screenshot of Editoria11y showing a notification in the top right of the interface to confirm the fix had been applied

Not all errors were fixable with automatic rule application – some required interpretation and judgement

Viewing the suggestions in the modals, several participants noted a problem with the fixes being based on automatic application of the style guide rules. Specifically, relating to the rule about omitting ordinal suffixes for numbers, there were several instances where it didn’t make sense to apply this rule directly. For example, a date written as ‘5th of December’ contained a suggested fix to remove ‘th’ but not to remove ‘of’. Another relating to ‘4th year’ of studies suggested removing the ‘th’ but not writing ‘fourth’ as a way to make the sentence readable, and similarly, another advised removing ‘st’ from ‘1st floor’ which was an incomplete fix.

Screenshots of Editoria11y showing instances where the ordinal suffix corrections couldn't be directly applied

Screenshots of Editoria11y showing instances where the ordinal suffix corrections couldn’t be directly applied

Some participants’ trust in the checker waned when they realised it didn’t catch all the errors

Of those who took part in the tests – some were more familiar with the rules of the style guide than others, and this impacted the trust they placed in Editoria11y. When they reviewed the Editoria11y outputs against the draft content and spotted corrections that hadn’t been picked up or flagged by Editoria11y, they were inclined to disregard the tool and check the content for themselves to ensure the text was completely compliant.

Screenshot of Editoria11y showing a style guide error where the pound sign is missing from an amount that the tool has not picked up

Screenshot of Editoria11y showing a style guide error where the pound sign is missing from an amount that the tool has not picked up

Few participants said they would use the dashboard as they felt that was for a site administrator

Navigating through the different options on the indicator, participants were able to access the dashboard interface, which was titled ‘Content Accessibility Issues’. Reviewing what was there, in sections called ‘Top issues’, ‘Pages with the most issues’ and ‘Recent issues’ most participants said they didn’t feel they would use this feature, and presumed it would be for someone with full responsibility for the whole site.

Editoria11y has potential as a style guide helper tool, but would need contextual refinement

Taking the findings together, Editoria11y showed promise as a way to embed the style guide rules into draft content, to avoid web publishers needing to navigate away from the editorial interface to check the rules in the guide itself to then apply them. Participants understood what the indicator bar was there to achieve, and liked the idea of being able to check off corrections to their content through the modals. Some identified areas for improvement included:

  • Refinement of the style guide rules embedded in Editoria11y, to include examples, exceptions and applications based on scope and context
  • Embedding a more complete set of style guide rules into Editoria11y to help build trust in the tool
  • Adaptation of the modal options to include ‘Accept with edits’ as well as ‘Accept’ and ‘Ignore’ to account for instances where editorial judgement needed to be applied to the suggested correction
  • A way to show corrections in categories (for example, all those relating to numbers together, all those relating to punctuation together and son on) which could be addressed by the editor in sequence, to avoid overwhelm in the interface
  • Suggestions for correcting content when reworked sentences were required – powered by an LLM such as ELM

As well as Editoria11y surfacing style guide rules, there is potential to use it for its intended purpose, as an accessibility checker, containing checks about the heading hierarchy and alt text on images. If this was to be included as well as style guide rules, however, progressive disclosure of the information should be used to avoid the interface becoming too cluttered.

Many of these areas overlap with work currently progressing in open-source Drupal, firstly develop a Context Control Center – as a way of handling rules to enable AI-assisted content production, and secondly to develop an AI Content Review feature, capable of reviewing content against defined rules and conventions.

Read more about these Drupal developments in my related blog post:

Think like a machine: How building a Drupal context-handling feature is providing a new lens on content design and style rules 

 

Stop chasing, keep researching: Why continuous contextual learning is the only way to build useful AI features

AI development keeps evolving as do the ways people seek to use AI. Traditional software development runs the risk of trying to perfect AI features people won’t use. Revisiting our previous AI research helped me tease out new opportunity spaces for AI features to help with content design tasks.

Last summer, Mostafa Ebid joined the UX team for a summer internship and built an AI assistant tool which integrated the University’s main AI provider ELM into EdWeb2, our Drupal content management system. The idea for the tool came from hearing University describe the difficulties they experienced when publishing web content. The tool included the capability to write content, design content and proofread, and was designed to work by typing prompts in a chatbot interface, right-aligned to the main part of the editorial interface. When prompted, an orchestration of AI agents were triggered to read textual content and use ELM to make suggestions for improvement based on what the publisher had asked for. Improved text was displayed in the sidebar chatbot interface for the publisher to review and consider using.

Read Mostafa’s blog post about how he developed the tool:

Integrating ELM with EdWeb – Building an AI tool for publishers

Initial tests of the tool with publishers revealed some potential, but identified the need to do further tests, specifically to understand limitations around the user interface display and to learn if the tool was useful to publishers in the context of content they were familiar with (as opposed to generic stock content that was used in the first round of tests).

Read my blog post about the initial tests of the tool:

Initial insights from UX testing our Drupal AI content assistant tool 

Mel Batcharj accessibility tested the tool, focusing on keyboard navigability, and Nick Daniels ran tests with two publishers in October last year. I recently reviewed the findings of this research to consider advancements in Drupal AI in the past 12 months, to assess whether the original premise for the tool was still valid, and to think about residual content design challenges AI could potentially help with.

Advancements in Drupal AI have resulted in improved AI features

The Drupal AI Initiative began in April 2025 with a group of participating organisations making a commitment to collaborate to build the future of AI in Drupal. As a result of the initiative, various workstreams began to shape Drupal’s infrastructure to support AI, to experiment with new innovations and to improve the UX.

Read more about this workstream:

Drupal AI Initiative project page on Drupal.org

A more accessible AI chatbot is now available as a Drupal recipe

Our original AI content assistant tool was built into a panel of the editorial interface as a custom build, which worked well when using a mouse, but which accessibility testing showed was restrictive when navigating using a keyboard. Since the tool was built, however, accelerated development in the wider Drupal AI community prompted the creation of an open-source AI chatbot freely available to apply. Adopting this chatbot was preferable as it avoided the need to maintain custom code and it was possible to use it with a keyboard only.

There’s an active Drupal issue to address the need for the AI chatbot interface to be expandable

Several of the participants who took part in Mostafa’s tests of the tool last year found it awkward to scroll through the AI chatbot output as it was presented in the narrow left-aligned interface. The same problem had been noted in the wider Drupal community, and therefore I raised an issue to have this rectified, which is being worked on as part of the AI Initiative task backlog.

We did more tests on our AI tool – this time using participants’ own content

In the first round of tests, four participants were all presented with the same piece of content (on the topic of safety procedures) and asked to use the AI tool to improve it in specific ways (such as writing it for user needs, making link text better, and so on). This approach turned out to be limited, as since participants were unfamiliar with the content, they were unable to assess whether the outputs of the AI tool were an improvement on the original content or not.

In a subsequent round of tests we therefore adopted a looser approach – asking participants to supply a piece of content they were already working on, and then asking them to use the AI tool to improve it to suit their needs. The results from these tests were more indicative of how useful publishers found the tool to make content improvements.

Results of these tests highlighted how the AI tool needed to change

Since we placed participants in a situation where they were using AI on content they knew well and could critique and appraise authentically, the results of these second-round tests gave clearer indications of what worked with the existing tool, what didn’t and where AI could be best applied to help with content design tasks.

Participants didn’t notice the content options in the AI tool, or the help text

The tool contained three different content options: Design Content, Write Content and Proofread, presented in a dropdown menu in its interface.

Initially, participants didn’t notice these options and used the default Write Content option.  When they later experimented with the Proofread option they found no discernible difference between these options in terms of outputs, which led them to believe that a simpler version with a single conversational interaction option would be preferable.

The tool defaulted to reading the content on the page, and working on this when prompted. Participants were initially unclear that this is how it worked, and they didn’t notice the help text to enable or disable this mechanism presented in the tool interface. Taken together, this feedback suggested that a simpler version of the AI chatbot, such as the one from the Drupal recipe, would be easier for publishers to use.

Close-up screenshot showing detail on the AI assistant tool chatbox

Close-up screenshot showing the content options and the help text on the AI tool

The AI tool had some value as a writing partner to suggest restructures to textual content

Responding to the task to experiment with the AI tool to improve their content, the participants quickly got used to how the tool worked, and recognised its use as a writing partner to prompt about their content and receive suggestions for improvement in a conversational way.

Prompts they used to improve a page of content in the body text field included:

  • ‘Rephrase copy to condense, highlight key messages and make it accessible to pet owners looking to join practice’
  • ‘Proofread copy so that it appeals to pet owners non clinicians’
  • ‘Turn this page into web ready content. It needs to be concise, easy to scan, readable to a wide range of audiences’

These prompts resulted in edited versions of the page content, typically including structural elements like headings, bullet points and calls to action, delivered in the chatbot interface.

Screenshot showing the output from the AI tool before and after a prompt to rephrase the content for a specific audience (before on the left, after on the right)

Side-by-side screenshots showing the prompt entered in AI tool to improve content for an audience (on the left) and after (on the right), with the output response to the prompt.

The tool lacked capacity to tweak or iterate on previous versions of content – which participants wanted

Once they had reviewed the tool’s initial outputs, both participants entered conversational turns with the tool, asking it to perform successive tasks on the content it had previously produced, to edit it further, in line with their specific requirements and rules.

Prompts they used to tweak initial AI outputs included:

  • ‘Remove adjectives and exclamation marks’
  • ‘Remove brackets’
  • ‘Remove any unnecessary words, fix typing errors, suggest improvements for SEO’

With every new prompt in the conversation, the tool produced a fresh output, meaning the publisher was left to review a succession of different versions of the edited content, presented in the chatbot interface, without any indication of what had been changed. Participants found it difficult to review edits, as they would usually do when working on a piece of content, in order to compare the ‘before’ with the ‘after’ – ultimately to assess whether the AI tool outputs were to their satisfaction.

They said they would have liked the tool to have presented the edits in a ‘tracked changes’ format that they were familiar with from word processing programmes.

Screenshots showing outputs of the tool before and after a prompt to iterate on improved content

Side-by-side screenshots showing a prompt entered to improve existing content (on the left) and (on the right) the output from this prompt – showing a new version of the content

Participants didn’t really need the tool in the interface as they tended to edit text content elsewhere

When describing their usual content process, participants said they would usually prepare their content in a word processing programme like Microsoft Word rather than edit directly in the Drupal editorial interface. There were several reasons they chose this method – a key ­­reason being the need to involve others to check (and in some cases, sign off) their content in preparation for the website. They were more familiar with referring others to check their content or proofread it when it was in the Microsoft suite, than when it was in the Drupal editorial interface.

Furthermore, Microsoft Word accommodated the addition of comments and tracked iterative changes to pieces of content which was not possible within the Drupal editorial interface. This content preparation habit suggested that while the AI tool was useful to suggest content edits, this would have been more useful before the content was in the interface, and therefore could be achieved by pasting content to be edited into a browser-based AI tool or app (such as ELM).

Within the interface, the tool only had use as a ‘final check’ mechanism, to catch any typos, errors or style misalignments before the content was ultimately published.

The tool needed to be able handle more than text as pages were typically made of multiple elements

Reviewing the test set-up and comparing it to their usual ways of working with EdWeb2, the participants said the pages they worked on would usually be made up of more than just textual content in the body text field. They would typically work on pages with multi-column layouts, making more extensive use of Drupal paragraphs or including structural elements like accordions, feature boxes and cards. They were interested to know how the tool may make appraisals or suggest improvements for those sorts of pages to help them arrange their content in appropriate ways.

We identified new opportunities for AI content publishing features

Extrapolating on the feedback from the tests, several use cases and scenarios for applying AI to content design tasks emerged, which will help inform our ongoing work to apply AI to make content design tasks easier for publishers.

AI-assisted content structuring

Describing their typical content writing workflow, participants said they found it difficult to move from a text-based editor like Microsoft Word into the Drupal editorial interface where they needed to make use of Drupal Paragraphs as well as page elements like accordions to structure the content. Potential areas for AI development could therefore include:

  • A mechanism to convert textual content into appropriate structural elements
  • A way to make suggestions for accordion labels or structure
  • A feature to ensure uniform creation of cards or feature boxes
  • A means of cross-checking style consistency of pages made of multiple elements or with a specific layout

AI- assisted content design for SEO/GEO/AEO

Having their content picked up by search engines or being machine read was something participants wanted, but they were unsure how to write, tag and structure their content to make this happen effectively. Potential areas for AI development could therefore include:

  • A way to have their content analysed for SEO effectiveness, based on signals like content quality, scannability and key word alignment
  • A mechanism to suggest content changes aligned to specifically defined SEO goals and target user engagement measures

AI assisted content reviews – against specific style conventions and contexts

As well as ensuring their content followed the rules of the University’s Editorial Style Guide, both participants mentioned other conventions that they needed to apply to their content, that existed at a more local website level. For example, one participant’s site had a rule not to use brackets or exclamation marks, or to overuse adjectives, so they would have found it helpful to have a way to cross check content against these rules before publishing.

The Drupal Context Control Center is an emerging feature designed to handle the application of context rules within a site across various scopes and use cases, and Drupal AI Content Review is a related feature, designed to appraise content against given context rules and conventions. Together, these Drupal features may be a good fit to help University web publishers make use of AI to shape their content with the uniformity they require.

Read more about the Drupal Context Control Center and its development in my recent blog post:

Think like a machine: How building a Drupal context-handling feature is providing a new lens of content design and style rules

Read about AI Content Review on Drupal.org

We’re changing the direction of the AI tool based on what we’ve learned

Last summer it seemed certain that an in-interface AI chatbot content assistant helper was what we needed to build. A few improvements to the UI to make it expandable and navigable with a keyboard and it would be ready to go. As it turned out, things had moved on, and these problems were addressed by the wider Drupal community. This meant we could go back to our research findings to re-examine how AI could be best applied to aid content design tasks, and to consider how it could best fit into existing workflows of our publishers to assist them with difficulties they faced. As we continue with internships this summer, we’re excited to re-focus and plan more research to explore some of these emergent opportunity areas. Aligning with in-progress Drupal AI developments, we’re open to learning how we can apply and adopt the work of the Drupal community to our University digital publishing context.

Think like a machine: How building a Drupal context-handling feature is providing a new lens on content design and style rules

AI tools to support content tasks are becoming more and more widespread. As part of my contributions to open-source Drupal I’ve been researching how to prepare and package content design and style rules that these tools can use effectively.

Using AI to help with content design tasks (and indeed, any other type of tasks) is now standard practice for many. As anyone who has experimented with it knows, the value the AI can provide is largely dependent on the prompt it receives, and the contextual data and information it is given to work with.

The way you provide your chosen AI with context affects the results you get

I have encountered two main methods for providing AI with context data. The first is the ‘prompt and pray’ approach, where you start with vague instructions and then engage in back-and-forth to drip-feed the AI the necessary information in the conversational turns. The second is the ‘context engineering’ approach, where you front-load information to proactive guide the AI, for example, defining its role, setting explicit goals, providing background and adding contextual constraints and rules.

Context engineering is more proactive than prompt and pray

Prompt and pray is better suited to a standard LLM interface or chat assistant (like ELM or Claude), and is the preferred choice when you’re looking to brainstorm and experiment, and are happy to take variable outputs. Context engineering relies on a more structured interface, designed to handle workflows (like Claude Skills) and is therefore the best choice if you’re looking for consistent, predictable outputs within set parameters, but it requires you to prepare your context data upfront, and for your chosen AI mechanism to have a way of handling it.

The Drupal Context Control Center is a management system for context data

Context data is a term given to the structured, site-specific knowledge – such as brand guidelines, regulatory requirements, editorial rules, and terminology – that tells an AI how you want it to work, to ensure its outputs reflect your actual preferences and standards rather than generic defaults.

In January 2025, our team was fortunate to spend a day of Drupal AI experimentation with specialist company Freely Give, in which we applied AI solutions to some of the web content management and design problems faced by the University. One of our experiments involved prototyping a basic style-guide checker into EdWeb2.

Read about this early experiment in my blog post:

An automated Editorial Style Guide? Experimenting with Drupal AI Automators

Looking back, this experiment was a very basic form of context handling, and since then, following the launch of the Drupal AI Initiative in June 2025, there have been rapid developments in thinking about AI and context. The Drupal Context Control Center (CCC) was initiated specifically for the purpose of enabling AI to handle context data and I am proud to have worked on its design, collaborating with Kristen Pol, Aidan Foster and others from the AI Initiative.

Read about the Context Control Centre on its project page on Drupal.org

To design the CCC architecture we brainstormed context application scenarios

Starting with a blank page, we started trying to come up with the main building blocks that the CC needed to have, but we found it very difficult. Drupal is phenomenally flexible which meant every function we wanted could be achieved in various ways. We quickly moved from thinking in the abstract to considering real-life scenarios when people would want to use AI to make use of context data to control what happened with their content which helped us tease out the tasks we wanted the CCC to support.

An example scenario was as follows:

Content creator wants to refresh a prospective student recruitment campaign. They ask the AI, integrated in the content management system, to help. Acting on the instructions, the AI prepares a sample campaign page which includes a slogan that doesn’t match the voice and tone and contains images from winter when the campaign is for summer. The content creator rectifies this by providing the voice and tone guidelines and updated imagery. The AI tries again, it’s better but this time there’s a word spelled differently from the style guide. The creator uploads the style guide rules. The page looks good, but the content creator wants another version to compare. The AI comes up with a version 2. The content creator wants to track interactions with the first version and swap to the second based on performance. To do this they connect their Google Analytics Acquisition reports and give the AI instructions of the thresholds to look for, to initiate the change.

We defined building blocks of the CCC and the relationships between them

Considering what would be needed to support the scenarios we came up with, we identified that the CCC needed to house several types of data and we thought about the dependencies and interactions the CCC would need to support.

1.     Data items that set out ‘the What’

First and foremost, the CCC needed to include the context items (rules and guidelines) such as:

  • Brand guidelines
  • Voice and tone rules
  • SEO keywords
  • Campaign or project briefs
  • Style guide
  • Image bank
  • Pattern library

In addition, it needed to contain subcontext items (more refined versions or child/subsets of the context items) such as:

  • New product brand guidelines
  • Seasonal campaign copy
  • Seasonal image set
  • Landing page pattern set

2.     Data that set out ‘the When and How’

As well as the various sorts of rules and guidelines, the CCC also needed to include details of the circumstances and situations in which to apply them. Collectively, these were called the context scopes and they mapped out boundaries and constraints such as:

  • Global (site-wide)
  • Site sections
  • Content types
  • Use cases (for example – writing teaser copy, working with images)

3.     Mechanisms that control how agents select context

Connecting the ‘what’ with the ‘when and how’ relied on AI agents identifying context scopes and selecting appropriate context items to apply. For this to occur successfully, the CCC needed to include some means of facilitating as well as guardrails to steer suitable choices. These included:

  • Context limits (limiters on numbers of context items and tokens)
  • Scope subscriptions (agent configurations to define opt-in or opt-out)
  • Target entities (specified content entities with context built-in)

We tried out terms in practice before deciding on labels

Labelling parts of interfaces is always tricky, particularly within Drupal where certain words have legacy meanings and, owing to its flexibility the functionality associated with terms may change over time. It was especially important to make careful label choices for the different parts of the CCC architecture, given its anticipated universality.

To inform my work on the CCC I decided to re-read ‘Design by Definition’ by Elizabeth McGuane, which book sets out and appraises a range of approaches for deciding upon terminology, labels and names for objects.

When choosing a name to fit a system, the author recommends appraising the options against three criteria:

  • Novelty – how standard or unique the name needs to be
  • Flexibility/mutability – how well the name works in different contexts
  • Memorability – how easily the name is recalled

Applying this idiom helped us decide upon several key CCC terms, including:

Context item: any piece of information fed into an AI powered mechanism (typically an agent) to help the AI’s working memory to produce responses and outputs that are accurate and tailored to the users’ requests.

Context source: origin of any piece of information fed into an AI powered mechanism (typically an agent) to help the AI’s working memory to produce responses and outputs that are accurate and tailored to the users’ requests.

A screenshot showing an interface from the beta release of the Drupal Context Control Center, showing the use cases and the context items (with inter-relationships). Arrows point out these different areas.

A screenshot showing an interface from the beta release of the Drupal Context Control Center showing the main building blocks

For the CCC to work as intended, inputs need to be machine-readable

With key parts of the CCC defined and the architecture mapped out, we started to consider how well the CCC could handle variations of context data loaded into it. This required us to think more broadly about how AI makes sense of information.

In the excellent book ‘Machine customers: The evolution has begun’ by Katya Forbes, the author outlines many use cases where people enlist AI agents to take care of tasks for them and diagnoses the underlying factors necessary for the AI to satisfactorily work on behalf of the people concerned.

In one such use case she breaks down the stages required for a human to take a purchasing decision compared to a machine.

For a human to decide on a purchase, they go through the following stages:

  1. Awareness – ‘I have a problem’
  2. Interest – ‘This might solve it’
  3. Consideration – ‘Let me evaluate my options cognitively and emotionally’
  4. Intent – ‘I’m leaning toward this choice’
  5. Purchase – ‘This feels right’

For a machine to decide, the stages are different:

  1. Query initialisation – parameters received ready to begin search
  2. Discovery – options identified that meet basic criteria
  3. Evaluation – comparing options against weighted parameters
  4. Verification – validating performance claims and reliability
  5. Selection – optimal choice identified based on data

From this comparison use case, it is clear that designing contextual data such as guidance documents to be used by AI requires a different approach to designing content to be read by humans. In particular, writing guidance information that relies on sentiment and/or subjective interpretation will be lost on AI and therefore not applied in the ways intended.

In its current form, our Editorial Style Guide is only partially AI-ready

Last year, in the early days with ELM, members of the UX Service tested ELM’s ability to apply the rules of the style guide to check content. They found it had limited success and found that in some cases, ELM applied its own rules to checking the content which couldn’t be traced to the style guide. Reviewing this experiment in light of the machine customer journey, it becomes clear that for an AI like ELM to faithfully and consistently apply style guide rules, the rules need to be written in a way that demands minimal interpretation, in other words, a deterministic way.

Read John Wilson’s blog about experimenting with ELM and the style guide:

Testing ELM’s ability to return useful results with prompts about the Editorial Style Guide

Adopting a ‘machine-first’ lens to the Editorial Style Guide to analyse content in selected sections, it is possible to pull out some of the deterministic (automatable) rules to compare with non-deterministic (requiring judgement) ones.

Analysis revealed deterministic and non-deterministic rules in the Editorial Style Guide

I completed a quick review of three sections of the style guide to assess how AI-ready they were.

  1. In the Headings and page titles section

Deterministic rules:

  • Do not skip heading levels
  • Do not use H5 or H6 headings (maximum four levels)

Non-deterministic rules:

  • Use word and language your users will be looking for and familiar with
  • Write descriptive headings – avoid vague words
  1. In the Acronyms and abbreviations section

Deterministic rules:

  • Do not put full stops or spaces between letters of an acronym
  • Do not abbreviate Professor to Prof
  • Do not use eg, ie or etc

Non-deterministic rules:

  • Spell out acronyms multiple times on long pages or pages with accordions
  • Well-known acronyms don’t need spelling out
  • Use abbreviations only when better known than the full version, or when space is limited
  1. In the Links section

Deterministic rules:

  • Do not use a URL as link text
  • Do not use ‘click here’, ‘more information’, ‘learn more’ as link text
  • Put links on a new line, not inline in a sentence

Non-deterministic rules:

  • Add details about what a link will do (open in a new tab, require a University login) where relevant
  • Avoid duplicating links where you can
  • Reserve button styling for the most important links

Going through this short exercise with our style guide established that before we can make valid use of AI-powered features like the CCC, there is preparatory groundwork required to some of our content design guidance and rules more AI-readable, and potentially to create a new machine-readable version of the Editorial Style Guide.

I’m looking forward to investigating the potential for the CCC at the University

My work with others in the Drupal community on the CCC has led to a successful release of a beta version, with a release candidate coming soon, planned alongside the regular Drupal AI module release cycle.

Building on our early AI experiments with the style guide, I am keen to lead the UX team to explore the CCC’s potential to handle and apply editorial rules within a Drupal-powered CMS like EdWeb2. Our previous research with web publishers tells us that applying the style guide consistently can be genuinely difficult, and as AI-powered content design tools like the CCC continue to emerge, there is real opportunity to put this to work on the challenges our publishers face.

Read about our previous research with publishers about using the Editorial Style Guide

An analysis of responses to our Editorial Style Guide survey by Hannah Watson

Usability testing the Editorial Style Guide site by me in my previous Content Designer role

Full automation of good content design is unlikely and undesirable but it may be possible to offload some of the more straightforward rules to an AI mechanism like the CCC, freeing publishers to focus their attention on the trickier aspects of content preparation. Taking a machine-assisted view will also provide the chance to revisit existing non-deterministic rules to assess whether some could be made more granular, broken into sub-contexts or context scopes for more precise application.

In line with our broader aim of improving the tools available to content publishers, I am excited to see what the CCC can offer for real-world content design challenges at the University.

 

❌
❌