I’m Grace, the Web Registry Development Intern for the 2026 Summer! As my internship draws to a close, I discuss how my AI Supported Accessibility Testing, and my Web Estate Dashboard projects have progressed. The first project focused on investigating how AI can be used to automate accessibility testing websites. My second project aimed to produce a dashboard for non-technical audience so they could see their website statistics. I reflect on my project, my experience, and what I’ve learned.
AI Supported Accessibility Testing
Almost three months after I started, my internship is wrapping up. My Summer project aimed to investigate how AI can be used to automate accessibility testing, so we can make sure the web estate is in compliance with Web Content Accessibility Guidelines. Currently, most automated testing (programs like axe DevTools) catches approximately 30% of errors, I aimed to increase that accuracy and investigate where AI can be useful. I’ve learned a lot, but there’s still lots more for me to pick up! The goal was to get AI to test for WCAG violations more accurately, and see if it could be a useful tool in the testing process.
Findings
Rather mundanely, AI won’t be taking over the world anytime soon. AI supported accessibility testing still hovers around 30% accuracy.
Strengths
The agent is good at crawling through sites and identifying suspicious elements that commonly produce issues, things like a pop-up widget or a pdf. It also excels at anything related to markup languages, testing guideline 1.3.5, identify input purpose, for example, which looks to see if input fields are labelled correctly, so computers can autocomplete them. These black-and-white guidelines are easily digestible for AI, it’s when a human aspect in included that AI can stumble.
Weaknesses
In conversations with the accessibility team, it was noted that AI struggles with the “grey-areas” that pop up in the testing process. While the full list of WCAG guidelines is lengthy and specific, the primary goal is to make sites usable for all. A guideline may not explicitly outline why a website fails, but if a human finds it inaccessible, that means it’s inaccessible! An element’s context within a website can drastically change whether or not it’s WCAG compliant, this was something that agents struggled with. I found that agents are too scared to get something wrong to think critically or apply executive judgement.
Aside from that, guidelines that measure things like a logical tab sequence—measuring how intuitive it is for someone to navigate a website with a keyboard—should continue to be tested and reviewed by humans.
Methodology
In the beginning, I did some desk research to find the AI agent best suited for the task at hand. I investigated the different LLMs available through ELM, with varying success. They could analyse whatever screenshots I gave them, but that still required me going through a website and selecting information that was relevant. I pivoted to the ChatGPT software, which was then called Codex. Codex could exit its window and crawl through the site itself autonomously, more closely mimicking how accessibility testing is actually carried out. It was a bit of a learning curve, lots of failed audits that I got to learn from. Throughout the summer, I iterated a master prompt to feed to agents, tinkering away at it bit-by-bit. I made it modular, so it was easy to swap out guidelines based on what you wanted to test. Currently, the prompt sits at 7252 words, and each guideline is split into smaller sections.
The AI specifications section contributed the most to the wordcount. You have to be very specific about the exact path you want the agent to take. It’s good that you get to have specific control over the testing, less great when you have to soft-parent an agent through pressing the “Tab” key.
This prompt would then produce an audit report, this is distinct from a completed accessibility report. The audit was produced to help a tester identify issues, not to present findings to a wider audience. During my internship, I gave weekly updates to the accessibility team and I was consistently given really valuable feedback that I could integrate back into the master prompt. At the end of my internship, I started scoring the accuracy of the prompt against preexisting human accessibility reports.
Web Registry Dashboard
The aim of this project was to pull data from a registry service using its API capabilities and present it in a user-friendly dashboard. I wanted to make sure the entire process was cheap, secure, and easy to upkeep, qualities that can be tricky to balance all at once.
I split the project up into 2 phases. Phase 1 covered transferring the data from the registry to a PowerBI dashboard, which was a bit of trail and error, but was successful. Phase 2 covered adding specific filtered to the dashboard depending on the user, and these filters would be added autonomously, without requiring users to manually log in every time. Phase 2 was a lot trickier, and s still incomplete.
Methodology
Phase 1
This really tested, and expanded upon, my technical skills. Through my astrophysics degree, I do a lot of data analysis and presentation through Python. However, the data in a university setting is reliable, easy to obtain, and consistent, this isn’t the case with this project! It was really interesting working with data that has real-world implications and it changed the way I think about my data analysis. I used Python code to make my API calls—the thing that gives me the data from the registry—which confused me at first, but everyone on my team was happy to answer all my questions. I was able to pull the data from the registry and produce a PowerBI dashboard that presents all this data simply.
Phase 2This really broadened my experience with different types of data, especially sensitive data and the security that must come with it. Phase 2 results are still inconclusive, but good progress has been made.
Applying the filters to the dashboard is relatively straightforward. Phase 2’s main difficulty is accessing employees’ names and departments, data which is quite sensitive and requires high admin permissions to access. Naturally, any workflows would have to ensure employee data is handled securely. In order to produce a successful dashboard, the current web registry data would need to be completely reorganised. To create a robust dashboard, the data—including employee data, stored separately—would also need to be accurate.
What I learned
I’m still a long way from understanding the ins-and-outs of accessibility testing, but this project certainly built up my confidence in the subject. It encouraged me to think about the logic of these guidelines in creative ways, to make it better digestible for AI. After all, how would I train an AI to test a guideline if I myself didn’t understand it? While the project didn’t produce earth-shattering findings, the findings are still inconclusive. We should continue to explore tools to make websites more accessible, especially as AI continues to evolve.
The dashboard helped my technical skills grow, and taught me to come at an issue from many different angles as certain avenues were rejected. The dashboard project taught me that there sometimes isn’t a “right” way to do things and that sometimes you have to juggle conflicting considerations in a project.
Overall, I’m really grateful I got to evolve in this way and I’m excited to see what future projects can accomplish!
Login
