We often get asked how to measure psychological safety. And it’s a fair question. Teams and organisations want to know where they stand, and leaders are usually being asked for a number by someone above them.
The trouble is that most of the tools available answer a slightly different question than the one that matters. They tell you how much — a score out of five, a percentile, a place in a benchmark. What they rarely tell you is why, or what to do differently in your next teem meeting, which is in half an hour.
Even a thermometer changes what it measures. A survey about psychological safety changes far more.
The moment we ask a team whether they can speak up, we’ve already told them something about the subject – maybe that it’s safe to discuss it, or maybe that we think there’s a problem. We’ve introduced some vocabulary and have signalled that we consider psychological safety important enough to measure. Done badly, we’ve also taught them that their answers disappear into a spreadsheet and nothing happens as a result – survey theatre.
Rather than try to pretend we’re taking a neutral reading, we’ve designed the survey around the observer effect – the act of measurement changes the thing we’re measuring, so we should make that effect positive rather than negative.
A plan, not a score. The report ends with a short, prioritised set of experiments chosen against what your team actually reported, ranked by fit and effort. Things you can try now, drawn from the practices we use with clients.
It tells you why people hold back. There are many reasons people stay quiet, and each needs a different response. Someone who cannot predict how a comment will land needs something different from someone who predicts it perfectly well and expects it to cost them dearly. Different again from the sense of futility felt by the person who has raised things before and watched nothing happen.
It shows you what is already working. Most surveys only ask what is broken. When people tell us that speaking up here tends to go well, we ask what makes that possible, so the team knows what to protect and do more of.
You hear what people would not say to your face. Anonymity is built into the architecture rather than promised in a policy. No names, no accounts, no timestamps, and results stay sealed until the survey closes.
It draws on over ten years of practice. We’ve worked globally with teams in technology, healthcare, aviation, heavy industry, financial services and more. What works in one of those places usually has something to say to the others, and the suggested actions come from that experience rather than from a textbook.
There is no benchmark against other teams or other organisations. Comparison breeds gaming, complacency and anxiety in roughly equal measure. The only meaningful comparison for a team is with its own past.
And there’s no incentive mechanism, no punishment for the manager who’s struggling against the system, and no reward for a high “score”. A psychological safety score used to punish or reward a manager isn’t measuring psychological safety at all, because everyone knows what the number is for and answers accordingly.
We also show distributions rather than averages – a team is only as safe as the least safe person. A team where everyone sits at three and a team split between ones and fives produce the same mean and need completely different conversations. The report shows you which one you have, one square per person, so the disagreement is visible rather than averaged away.
How the survey works
Pick the statements you want to ask, in your team’s language: adapt the wording, or add your own. Send the link. People answer twelve or so statements in about five minutes on any device, with no account and no app.
When you close the survey, the report opens to you first, then you share the identical report with the whole team. That order matters. It means no surprises for you and no cherry-picking for them.
The question bank is adapted from Amy Edmondson’s work, extended with our own items about voice, power and the reasons people stay silent. If you are doing research, or you need scores comparable across teams and studies, use her original validated seven-item scale unmodified: validation is precisely the property that adaptation destroys. Ours is built for a different job, which is starting a useful conversation in your team.
There is also a free three-minute self-check if you would rather look at your own experience first. It stores nothing at all.
Measure your team’s psychological safety
You can read a full sample report before you ask anyone anything. It is built from synthetic data for a team that does not exist, by the identical pipeline a real team’s report goes through.
Team surveys are free during the preview period while we learn from how people use it. If you try it, we would like to hear what worked and what did not. There is a feedback link on every page, and we read all of it.
“A scientist must also be absolutely like a child. If he sees a thing, he must say that he sees it, whether it was what he thought he was going to see or not. See first, think later, then test. But always see first. Otherwise you will only see what you were expecting.” — Wonko the Sane (Douglas Adams)
Organisations spend a great deal of time and effort trying to understand themselves, through engagement surveys, benchmarks, maturity models and more. And yet many leaders will admit that they don’t feel like they know enough: the dashboards and the metrics aren’t quite the full picture. Many of the things that matter most in an organisation are the things that are hardest to measure directly: things like alignment, buffer capacity, expertise, safety, and ambition.
The more elaborate and expensive our system for understanding something, the less we may actually understand it, because we end up paying attention to the numbers instead of the actual organisation. This is worse than simply not knowing, because it doesn’t feel like not knowing. It feels good, it feels like control.
Riverbank with Himalayan balsam [Tom Geraghty]
Ecologists have spent a very long time on the same sort of problem: we usually can’t measure and quantify the health of an ecosystem directly either; there’s no eco-mometer that we can put in a stream to give us a measure of its health. So ecologists instead read these things indirectly, through what’s present or absent, what is thriving or struggling, what’s changed and what persists. In “Five Ecological Concepts for Working in Organisational Change” I described one of the ecologist’s most useful tools, the indicator species: organisms whose presence or absence reveals the health of a habitat in ways that would otherwise be difficult or impossible to measure directly.
And this is why I think ecology isn’t just a metaphor for organisational dynamics, but one of the richest lenses we have for them. Organisations are complex adaptive systems, and reading complex adaptive systems is exactly what ecology has spent centuries learning to do. When we borrow from it, we’re borrowing practices from a mature discipline built for precisely the kind of thing an organisation is.
And it is a discipline, which is important. Reading organisations ecologically is not a method or a tool. It’s a practice. The difference is that a method can be written in a manual or explained in a training session, whereas a practice has to be developed. What I’m describing here is closer to what an experienced field ecologist does when they walk a habitat — a kind of trained, attentive seeing that draws on their experience and everything they know without consulting a checklist. That takes time, and exposure to many different habitats. And it takes someone occasionally pointing at the thing you walked past and saying: oh, did you see that? This piece describes that way of seeing, not a checklist of things to look for.
First Sight and Second Thoughts, that’s what a witch had to rely on: First Sight to see what’s really there, and Second Thoughts to watch the First Thoughts to check that they were thinking right. Terry Pratchett – The Wee Free Men
The caddisfly Stenophylax permistus, Baltasound by Mike Pennington
What do we mean by indicator species?
Bear with me through a few plants and freshwater invertebrates; because every one of them is teaching a habit of attention we’ll take back to the workplace. All species can tell us something about the habitat they occupy. Even very common species carry information. Nettles (Urtica dioica) typically indicate nutrient-rich, recently disturbed soil and often signal human activity — a place where things were moved or the ground was disturbed. Rye grass, hardy and tolerant of compaction, can suggest grazing or trampling pressure. Rosebay willowherb is sometimes called fireweed, because it will rapidly colonise burnt ground, telling us something about the history of the land – during WW2 it was also known as bombweed, because it thrived in bomb sites. Fruticose lichens, and some fungal diseases such as sycamore tar spot, are particularly susceptible to air pollutants (classically, sulphur dioxide), so we tend to only find them where the air is clean, away from heavy industry and traffic. All these tell us something about the habitat they’re in, and by identifying multiple species in one place, we can triangulate a fairly precise picture of the local conditions without actually measuring anything.
Rosebay willowherb [Tom Geraghty]
(Note, I’m using primarily British examples throughout — the specific species will be very different in other parts of the world, and that is, in fact, kind of the point.)
Some species are more discriminating than others. The organisms that ecologists find most useful as indicators tend to be those with narrow tolerances — sensitive to specific conditions in ways that make their presence or absence genuinely informative. Among the most useful are the “EPT” taxa: Ephemeroptera (mayflies), Plecoptera (stoneflies), and Trichoptera (caddisflies). These freshwater invertebrates are highly sensitive indicators of water quality: they require high oxygen levels and low levels of pollutants. A single kick-sample from a healthy stream, containing stonefly nymphs and caddis fly larvae, tells an ecologist more about water quality than laboratory tests (and far more quickly too), because these organisms are living indicators of conditions, not just a metric analysis of nutrient and pollutant levels.
Now, consider Tubifex tubifex, a small and ostensibly very boring aquatic worm. Also known as the sludge-worm, or sometimes, if it’s feeling fancy, the sewage worm. In a fast-flowing stream that we would expect (and hope) to be oxygen rich, the presence of tubifex could be a warning: it tolerates low oxygen and organic pollution that most other invertebrates cannot, and its presence signals that the conditions might be too degraded for more sensitive species. The same organism in the sediment of a large lake is entirely unremarkable and somewhat reassuring — those conditions suit it fine, and its presence there tells us relatively little. The same species in different contexts tells us different things.
Tubifex tubifex by Stefano on Flickr: https://www.flickr.com/photos/81918877@N00/
This is the first thing to understand about indicator species, biological or organisational: context precedes reading. There is no universal indicator; no signal that means the same thing in every habitat. In organisations we often forget this: we read a behaviour, a metric, a structure, and assume it means what it meant somewhere else. It doesn’t, necessarily. The same signal, in a different organisation or a different stage of the same one, can mean a different or even opposite thing. So the first move is never to interpret the signal. It’s to read the habitat.
Reading organisational indicator species
Let’s explore what that might look like inside an actual organisation. I was once part of the senior leadership team of a fast scaling tech startup and in the very early stages, we had very few managers (we also didn’t have heating, and our primary server was one we pulled out of a skip). Everyone just did stuff, made decisions quickly, rarely asked for permission, and we moved fast. But as we scaled, we hit the usual issues that rapidly scaling organisations do: work got more complex and interdependent, and as a result, tasks and deliverables would more often collide and conflict with each other, which meant that we needed increasing alignment and oversight to make sure we weren’t getting in each other’s way. We needed team and product managers to provide oversight, coordination and prioritisation. This growing concentration of managers as an indicator species was a healthy organisational signal: structure was emerging where there had been early productive chaos, processes were forming around the places where coordination at scale was breaking down, and the indicator species of governance roles were doing what they do — stabilising the substrate so that more long term and mature species could take root. However, two years later the project management layer had thickened beyond what the work required; handovers had become excessively elaborate and approval boards had become blockers rather than enablers. What had been a healthy response to early-stage complexity had begun to signal high institutional anxiety: a defensive response to an environment that felt fearful of failure and far more brittle than it once had.
The same species at different stages of an organisation result in a different reading.
This is where ecological succession matters as a companion concept. We can’t read an indicator without understanding where the system is in its developmental trajectory. An ecologist doesn’t look at pioneer species on recently cleared land and conclude that something is wrong — those are exactly the species we’d expect to see at that stage. The same species on an established woodland floor might be cause for concern: it would suggest a recent disturbance, a localised regression to an earlier successional stage.
Recently disturbed land with pioneer species emerging [Tom Geraghty]
A caution, though, about pushing this metaphor too far. Succession in a woodland tends towards something we can broadly predict, because the conditions are somewhat stable and we have repeatedly watched many woodlands do it. Organisations are not like that: there is no fixed end-state they’re heading for, arguably no climax community, and the trajectory can stall, reverse, or branch. The organisation now is as different to the past organisation as it is to another organisation, so restoration never means turning the clock back to some idealised point; it means asking how to move towards a better future.
Whilst succession doesn’t tell us the end state, it still tells us something important about order. A disused pasture does not become woodland in a single step; it has to pass through stages like scrub first, because some states are preconditions for others. We can’t skip them, however much we desire the end result, because the later stage is not merely later; it’s dependent on what the earlier stage builds.
Organisations work the same way. The mature thing cannot be installed before the conditions that make it possible actually exist. We will not (for very long) get people openly challenging senior decisions in a place that hasn’t first built the more basic conditions that make challenge safe and survivable; we cannot introduce otters into a river that cannot yet support them. So succession is useful here not as a ladder with a known top rung, but as a reminder of two things: that the same signal means different things depending on where a system has come from, and that some things simply cannot happen before others.
Presence isn’t establishment
One of the subtler lessons from ecological restoration work is the difference between presence and establishment. A conservation project might introduce specialist species (for example, round-leaved sundew [Drosera rotundifolia] to a recovering peat bog) as part of a restoration programme. To a passing observer, their presence looks like success: the specialist indicator species are there, therefore the habitat must be recovering and healthy. But an ecologist must look more carefully: are these plants thriving and reproducing? Is the substrate actually supporting them, or are they merely persisting on the back of the intervention, dependent on continued artificial implementation and support to survive?
Sundew. Photo by Ian at https://naturalbornblogger.co.uk/?p=1888 (CC BY-NC 4.0)
Organisations do this constantly. Fixating on “what worked there” instead of considering “what will work here?”. Adopting the “Spotify model”, implementing 360 degree reviews, putting a poster up telling employees we have a “Just Culture”. These are introduced species. Their presence may tell us relatively little, because the question we need to ask is whether they are self-sustaining: whether the conditions in the substrate actually support them, whether they are spreading and reproducing without continued top-down mandates, whether they’re actually the right species for this habitat at all. If the answer is no, what we have is a display, not a thriving habitat. The trap is flipping causality, in thinking that because many healthy organisations run regular team retrospectives (for example), that running regular retrospectives means we’re a healthy organisation.
“The more one learns of this intricate interplay of soil, altitude, weather, and the living tissues of plant and insect (an intricacy that has its astonishing moments, as when sundew and butterwort eat the insects), the more the mystery deepens. Knowledge does not dispel mystery.” ― Nan Shepherd, The Living Mountain
This is where “rewetting organisations” comes in. In environmental restoration of a peat bog (incidentally, from the Gaelic bogach, meaning soft), we don’t begin by introducing specialist species. We begin by restoring the substrate: slowing down change, raising the water table, allowing sphagnum moss to re-establish, so that the conditions exist in which native species can return and thrive of their own accord. The specialist species come last, not first, and their arrival is evidence of recovery rather than its cause. Organisations that naively introduce practices they’ve seen others do, carry out restructures that mirror a different organisation, or apply goals and targets without attending to the substrate that supports them (including capabilities, power structures, incentive systems, institutional memory, and more), are at risk of planting sundews in dry, drained peat. They may, if we’re lucky, persist for a while, but they will not thrive.
What are we actually looking for?
An ecologist asked to assess a habitat doesn’t arrive with a fixed checklist to tick items off. They arrive with a trained capacity for attention, developed over years of looking at many different habitats in many different conditions, and they read what is in front of them against everything they know about what could and should be there, what has been there, and what the current conditions suggest about where things are heading.
Some signals are generalist: ubiquitous enough that their presence tells you relatively little. Others are discriminating: their presence or absence in this habitat, at this stage, under these conditions, is genuinely interesting. The skill is knowing which is which; and that knowledge is inseparable from knowing the habitat. Organisational “seeing” means looking at, and doing our best to understand, the context first – before interpreting what the signals may be telling us. If we read the signal without reading the organisation first, we risk confidently misreading it.
With that caveat stated, some organisational signals tend to warrant attention:
A proliferation of committees and change advisory boards often (but not always) indicates an institutional fear of failure: we’d rather not do anything than try something that incurs risk, and the accountability sink of decision-making is distributed so widely that responsibility becomes impossible to locate. Heavy, bureaucratic handover processes between teams frequently signal low inter-team trust: paperwork and process act as a surrogate for trusting relationships. An abundance of individual metric targets for performance, particularly where they are tracked publicly and tied to reward, often suggests that leadership doesn’t trust the workers to be intrinsically motivated, and tends to suppress exactly the lateral knowledge-sharing that collective performance requires.
But (and this is the tubifex point) none of these signals means the same thing in every habitat. Individual targets are appropriate in some cases where there is little interdependence between people. Heavy duty approval boards and signoff may be appropriate in safety-critical, highly complex contexts such as nuclear power. Read the context first. Always read the context first.
The positive organisational indicators
What’s the organisational equivalent of otters?
Otters returned to many British rivers as water quality improved — because the conditions finally supported them. They sit near the top of the freshwater food chain, and their sustained presence signals that the entire system beneath them is functioning: clean water, healthy invertebrate populations, plenty of fish, stable bankside vegetation. We cannot fake otters. We cannot introduce them into a degraded system and expect them to hang around for long. Their sustained presence is evidence of genuine, systemic recovery.
Otter at the British Wildlife Centre, Newchapel, Surrey by Peter Trimming, CC BY-SA 2.0 , via Wikimedia Commons
The organisational equivalent is the signal that could only exist if everything beneath it was working. Not the signal that has been managed into existence and kept on life support, but the one that emerged because the conditions were finally right. A junior team member openly disagreeing with a room full of senior leaders and being thanked for it, not “performance managed” afterwards. Genuine productive dissent surfacing in formal meetings rather than only in corridors and car parks. Lateral knowledge-sharing and ideation across teams that are nominally in competition with each other for resources or recognition.
These things cannot be “performed” sustainably. They can’t be introduced by policy – we cannot simply tell people to “speak up”. These signals are the otters: evidence that the substrate has genuinely changed, that the water is clean, that something has actually been restored rather than imposed or mandated. If we are regularly seeing these things, they are typically very good signs of organisational health.
The measurement-industrial complex
We’re all familiar with organisational surveys — the engagement survey, the pulse survey, the psychological safety survey. Ecologists do surveys too, but the two things share little more than a name. When an ecologist surveys a habitat, we’re exploring, looking, seeing, sensing: reading what is present and what is absent, what is thriving and what is struggling. A staff survey, by contrast, is a snapshot, abstracted from the habitat, often in language that doesn’t quite fit the context.
There is by now a substantial industry built around the measurement of organisational health — engagement platforms, culture diagnostics, psychological safety scales, maturity frameworks, and benchmarking services of all kinds. Much of this is well-intentioned. Some of it is useful, particularly when applied carefully, longitudinally, and in combination with richer and more qualitative reading. But the industry as a whole rests on an assumption that the ecological approach exposes as problematic: that the same indicators mean the same things across contexts, and that the path to understanding an organisation runs through the consistent application of validated instruments.
This isn’t true. At least not primarily. The more elaborate and expensive our system for understanding something, the less we may actually understand it: because the apparatus of measurement tends to substitute for the discipline of attention rather than supporting it. When we have a dashboard, we watch the dashboard. When the dashboard tells us that engagement is 7.2, we manage towards 7.8. And in doing so, we invoke Goodhart’s Law in its most destructive form: the indicator becomes the target, and in becoming the target, it stops being an indicator. We end up improving nothing, or, in many cases, actually making it worse (you may already be thinking of cobras as indicator species!). The sludge worm at the bottom of the dashboard keeps trying to tell us the actual conditions, but nobody is paying any attention to it because it’s boring and small.
The measurement-industrial complex trades in what Aristotle would have called episteme: systematic, generalisable, transferable knowledge. The kind that (at least claims that it) can be packaged, certified, and applied by someone who wasn’t there and doesn’t know the habitat. What the organisational ecologist1 develops over time is closer to phronesis: practical wisdom that is irreducibly situated, inseparable from experience and context and judgement, and not available as a downloadable checklist. We can learn the names of the EPT taxa in an afternoon. Knowing what their presence or absence means, in this particular stream, at this particular time of year, given what happened upstream last year, takes much longer, and it takes practice.
What the organisational ecologist actually does
An ecologist doesn’t “fix” ecosystems: we work to restore the conditions in which the system itself can recover, and then we pay attention to what emerges. Organisations are complex adaptive systems, not machines to be upgraded by swapping a component in the right place.
Silver birch establishing on heathland [Tom Geraghty]
The capacity of being an organisational ecologist, of “seeing things properly”, develops over time, through practice and attention and the accumulation of many habitats read carefully. It is not a generic framework to be applied or a checklist to be ticked off. A real woodland restoration plan is a strange document to an organisational eye. It works across decades, sometimes with no end point at all, and contains almost no deadlines. Its phases overlap rather than run in sequence. Its operations are conditional: we fell trees only if they’re diseased, we intervene only where needed, and we support regeneration where it appears. A phase is complete not in Q3 but when the assessment suggests it is — when the system itself says so. The plan describes conditions and directions, not dated milestones, because habitats don’t consult plans. Neither do organisations.
None of this resolves into a simple organisational checklist, and it would betray the whole argument if it did. But there are dispositions, habits of attention, that the organisational ecologist develops.
Weread the habitat before the signal, because the same signal means different things in different places.
Weconsider where the organisation is in its own development, because a structure that is healthy in a young system may be pathology in an old one.
Weimplicitlydistrust the introduced species: the practice imported because it worked elsewhere, and ask whether it is actually thriving or being propped up.
We triangulate across many weak signals rather than trusting a single strong number, because a number is the feeling of knowledge and rarely the thing itself.
We look for the signals that can’t be faked: the ones that could only exist if the conditions beneath them were sound.
And we attend to the substrate, because we’re working with a living system that will not consult our plan, and the most useful thing we can do is often to make the ground more hospitable and pay attention to what grows.
Most of all, we keep looking. The habitat changes, the stage moves on, the indicator that meant one thing last year means another now. We’ll never stop trying to look at things properly, because the work is never finished.
See first, think later, then test. But always see first.
A different way of seeing isn’t something one can be handed; it’s something we develop, over time, in the field. But it helps to start somewhere. That’s what our Thinking Like An Ecologist workshop is for: not a new set of tools, but the beginning of a different kind of seeing.
1* Note: Through all of this I should be clear that I mean something quite different from the sociological field of “organisational ecology” (Hannan and Freeman), which studies why whole populations of businesses are founded and die under impersonal environmental selection. In this piece I mean something smaller and almost opposite in spirit: reading and tending the living dynamics inside a single organisation, on the premise that its conditions can be shaped.
Seven principles for measuring people without mistaking them for data.
In theory at least, we measure psychological safety because we want more of it. That, in fact, is often part of the problem. The act of measuring changes the thing being measured, and not always for the better: as soon as a survey score becomes something a manager is rewarded for, we stop measuring safety and start measuring how much influence that manager has over the answers (such as whether they decide to simply fire the psychologically unsafe members of the team!).
Measurement isn’t neutral. Someone chooses what to count, what it means and how to present it, and those people are rarely the ones being counted. A heat map of red and amber squares may tell us where safety is low but it cannot tell us why, and it does not tell the person filling it in that anything will change as a result. In fact sometimes it suggests the opposite, because one of the fastest ways to teach people that speaking up is pointless is survey theatre – asking them how psychologically safe they feel and then doing nothing about it.
This newsletter issue is a collation of all our primary pieces on measuring psychological safety using both quantitative and qualitative approaches, what the risks are (and they are significant!), why there’s a tendency to measure the easy thing, not the important thing, and what to think about before we even start.
The seven principles below are our attempt to codify our fundamental approach to measurement, inspired by UKRI’s principles of ethical research. They’re constructed about psychological safety, but they apply to most, if not all, aspects of surveying and measuring people in organisations.
Psych Safety Principles of Ethical Measurement in Organisations
Measure to benefit, not to surveil. Ask only for what you need and only what you’re going to use. What people tell you should serve their working conditions and not be turned back on them.
Make participation voluntary and informed. Tell people what they’re being invited to contribute to, who will see it and what happens next. Declining should cost nothing.
Be honest about power. Measurement is not neutral. Name who’s asking, who’s answering, and what the gap between them means for what gets said.
Protect the people who answer. Confidentiality must be built into the process, not an afterthought. Nobody should be worse off for having told the truth.
Focus on learning, not judgement. Read scores as data about conditions, looking for learning opportunities, not who can be fixed or blamed.
Work with integrity and transparency. Be honest about what the data can and can’t tell you. Don’t overstate certainty and don’t bury inconvenient findings.
Close the loop, or don’t open it. People have given you their time and their honesty. Tell them what you found and what you’re doing about it, or don’t ask at all.
Further reading
1. Measuring Psychological Safety The fundamental question: what are we actually capturing when we measure psychological safety, and what does the measuring do to it? This piece works through the epistemology (why averages obscure the least safe person, why the act of measuring shapes what gets valued) and ends on caution: reach for qualitative understanding over metric fixation.
2. Goodhart’s Law, Campbell’s Law and the Cobra Effect When a measure becomes a target, it stops being a good measure. Goodhart’s Law, Campbell’s Law and the Cobra Effect explain why: even just the existence of a metric results in optimising for the metric, not the thing it was meant to stand for. Applied to safety surveys, this is how a programme designed to increase safety can end up creating the appearance of it instead, masking much more serious problems.
3. The Streetlight Effect We sometimes search where the light is, not where the thing is. Measurement gravitates towards what is easy to count rather than what matters, and those choices are never neutral: deciding what to measure is deciding whose experience counts. A short, sharp argument that the convenience of a metric and its importance are two entirely different, and easily confused properties.
4. Beyond Metrics John Snow found the Broad Street pump by going beyond the numbers, not by collecting more of them. This piece uses his 1854 cholera map to show that over-reliance on metrics produces measurement-shaped knowledge rather than reality-shaped knowledge, and that the things which matter most are often precisely the ones that resist counting.
5. The Strange Confidence of 360° Feedback Every problem in this edition shows up in a single tool. Jade traces 360° feedback from 1930s officer selection to its 1980s trademarking, finds an evidence base far thinner than its ubiquity would suggest, and shows how averaging many biased ratings does not cancel the bias but smooths it into something harder to challenge. Anonymity, she argues, cannot be a shortcut to safety. 360 Feedback is the real world example that this whole edition points towards.
6. The Qualitative Measurement of Psychological Safety A survey can tell you where safety is low; a conversation can tell you why: the competitive bonus scheme, the threat of redundancies, or the unclear purpose behind a team. The best research is talking to people: Jade makes the case that words are not softer data than numbers, and offers concrete ways to gather them.
7. Measuring Psychological Safety: Questions First Before we measure anything, a pre-flight checklist. What is the measurement for? Longitudinal or cross-sectional, quantitative or qualitative? This piece walks through the questions worth answering before we send a single survey. Poorly designed measurement is worse than none at all.
See also
Explore this Trail on the Psych Safety Knowledge Map, and take detours into relevant academic papers too. If this edition leaves you wanting more, here’s two more from the archive. A team is only as safe as the least safe person makes the case that reporting safety as an average actively hides the one person it most matters to. And Reflections Part Three: The Safety to Dissent is where survey theatre is explored, and where the decisions about what and how to measure are shown to be usually made by people other than those being measured.
A few months back, I was preparing for a workshop with a client, and they shared, anonymised, their staff engagement survey data. We both sat on our respective screens, peering at a heat map of red, orange and the occasional shiny green rectangles, and I realised the data itself told me nothing useful for planning the session.
I could see from the data why they had identified psychological safety as an area for improvement, but beyond that the numbers themselves gave me no context, nuance or insight. So I started to ask questions – how long have the team been working together? What are their challenges? How do they communicate with each other? Really quickly, a picture started to take shape. The team had previously worked under a tyrannical manager who had since left, but in whose shadow they were still dwelling. Communication only happened in one-to-one conversations or private slack channels – the whole group chat was largely silent. Lack of clarity about what the team’s real purpose was meant that prioritisation was always contentious and conflict-ridden, and decisions were often undermined by other team members. From a relatively short conversation, I had a much stronger starting point for planning my session. I still didn’t know everything, of course, more always emerges in the workshop, but I had a much clearer picture of where to start.
Psychological safety is a felt phenomenon. It’s something we experience, in context, for reasons that are unique to us and the team. Yes, we might be able to measure headline figures about psychological safety, and we can even dig into some of its different ‘flavours’ – are people more likely to feel safe sharing an idea than asking a question for example? But we still don’t know why until we actually start talking to people.
There’s an assumption baked into a lot of workplaces that numbers are rigour and words are just impressions. People are often apologetic about offering their observations and might caveat them with, “This is just anecdotal,” or “Obviously this is only my take on it.” Which is good practice, of course, to acknowledge the limitations of our perspective and the biases in our take on the world. But I notice we tend to be less inclined to do the same with quantitative survey data. We run the risk of treating an engagement survey completed by only half the team at 5pm on a Friday, while they’re rushing and vaguely resentful of yet another survey, as if it’s more truthful than a set of thoughtful observations and reflections from someone who has worked in the team for years. It’s not, it’s just different, and ultimately it’s just more countable.*
Terry Pratchett put it best: “The best research you can do is talk to people.” Talking to people, and listening to them, with openness and curiosity, gives you things numbers can’t. You might discover that no one really speaks up because they know only the boss’s opinion counts. This means it’s not necessary that anything ‘bad’ will happen as a result of speaking up, but that it’s futile, so why bother. That’s a very useful insight into how the team works. Perhaps someone else tells you they’ve always taken responsibility for speaking up even when it’s hard or unpopular, and when you press a little, it turns out what they actually mean is that they don’t think they’d be fired anyway. This tells us that for this person, the stakes aren’t as high as they might be for others, which is quite different to either the whole team feeling safe or that person being particularly ‘psychologically brave’. None of this gets captured in survey scores, but it’s invaluable for planning next steps for a team.
The point, I think, is that we’re often scared of qualitative measurement because, unlike surveys, which have become accepted as standard practice across organisations, we don’t always know how to do qualitative measurement of psychological safety – and whether it “counts”.
What qualitative measurement looks like in practice
There are rigorous ways we can go about formal qualitative measurement of psychological safety if that’s what we want to do. However, I think it’s also important to acknowledge that we’re already doing it. In every conversation we have with someone about how they’re getting on with colleagues, in every team meeting we’re part of and every interaction we observe, we’re gathering information, rich qualitative data, about the team. If our focus is psychological safety, we can find out a lot this way. All of this helps build our picture of what the organisation is like. It’s not a complete picture, but neither is a survey score.
“Trying to measure the traditional way with metrics is a suboptimal (at best) approach for complex properties or phenomena like safety or culture. A metric will provide you limited information and more often than not mislead you. A better way of ‘knowing’ is to aim for rich descriptions.” – Carsten Busch
While many organisations speak the language of ‘numbers’, and while an alarming statistic might open the door, research suggests it’s qualitative data – people’s voices and stories – that actually shifts attitudes. Numbers justify decisions; stories make us believe in them. In my experience, even with the most quant-driven folks, what really brings a data story to life is actual quotes – using what people have actually said to build a picture of what’s behind the numbers.
If we do want to more formally gather some qualitative data about psychological safety, what could we try? One of the major constraints in qualitative measurement or research is that it takes time – and to put it bluntly, if we get loads of qualitative data, loads of words, it’s going to take a lot of time to analyse well. But there are manageable ways we can integrate more qualitative exploration into our measurements. A few things come to mind, from the ‘easiest’ to the more involved:
If we’re going to do a psychological safety survey anyway, we can at least add an open text comment field – pop a box in to ask people why, or give them an opportunity to share some qualitative nuance behind their answers.
Observe what’s already happening. When we have the opportunity to join team meetings or join sessions, what do we notice? Who’s speaking? Who isn’t? What sorts of questions, ideas and concerns get raised? What happens to them? What kinds of opportunities and channels do people have to speak up? Do they have “think time”? Can they share in writing instead of verbally, or after the meeting as well?
Set up short interviews, being mindful that this will likely be less effective if we’re interviewing people we directly work with or whose career trajectories we have power over. We also need to think carefully from an ethical standpoint about what we will do with the data and make sure we’re clear with our interviewee – can we guarantee anonymity? But if we can navigate all that, we will find a lot out, through asking people questions like:
How safe do you feel speaking up in this team?
What might stop you from sharing your ideas?
Can you tell me about a time that you held something back? etc.
– As an aside on this one – rather than having to start from scratch with interviews, there are also often wonderful rich sources of interview data held by organisations in exit interviews – what have people shared in those that might relate to psychological safety? And if your org doesn’t do them, why not? They’re an ideal opportunity to gain really candid feedback about what it’s like to work for the organisation – from people who are leaving, so don’t have so much to lose.
Focus groups. Again, these take some setting up, and a neutral and skilled facilitator, but on a group phenomenon like psychological safety, bringing together small groups to talk about their experiences of psychological safety in the organisation could generate some really rich data.
The most useful thing you can do before trying to change the conditions in a team is understand what they actually are. A chart or heat map won’t give you that. Talking to people, and listening to what they say, just might. When we listen to understand, rather than to try to close a gap on a dashboard, it’s an act of curiosity and an act of care. And if we don’t care, we shouldn’t ask.
*As a slightly tongue in cheek aside, I’d like to point out the fist of five – where people hold up fingers to indicate how psychologically safe they feel – is considered a valid measure in some quarters. If that clears the bar, I’ll take our qualitative observations with some confidence.
Measuring the really important stuff is hard. How do we measure inclusion, or safety, or happiness? We can probably all agree that these things matter, but when it comes to measuring them, where do we start? In fact, some would go as far as to suggest that what matters can’t be measured.
As sociologist Willam Bruce Cameron explained in 1963:
“It would be nice if all of the data which sociologists require could be enumerated because then we could run them through IBM machines and draw charts as the economists do. However, not everything that can be counted counts, and not everything that counts can be counted.”
And yet, much of the world of work remains fixated on measurements, on pinning everything down to quantitative metrics, as if the only way to prove something matters is indeed to measure it. But because measuring the important stuff is either very hard, or impossible, I’m increasingly convinced that a lot of the time (maybe most of the time?) what we are doing is not measuring what matters, but measuring what we can, and making it matter.
And of course this is not just a technical problem of how to measure the difficult stuff, but a moral one – who gets to decide what’s important? When we flatten people’s lived realities into metrics, who decides what we quantify and what we disregard? Who has that institutional power? Measurement choices are about power. They determine whose experience is rendered visible, whose risks are legitimised, and whose realities remain structurally invisible.
“Measurement is a technology with power, harnessed by those able to use it.” Sally Engle Merry, The Seductions of Quantification
There are so many small and large examples of this. We know there are organisations who measure the number of story points completed by software developers as a way to monitor their productivity. Or they record how “active” someone is whilst at their desk by measuring their mouse movement. Consultancies use consultant “utilisation” as a measure of both productivity and efficiency (and nobody records it honestly anyway). Even the typical annual employee engagement survey is an example – we ask what is convenient to ask and will fit into neat bar charts ready for reporting and year-on-year comparison. We measure what we can, or what we already have the data for, but this is no guarantee that the numbers we generate are a good reflection of what actually matters. Customers, patients, and service users don’t care about hours spent on a task or what time someone arrived at the office. They care about good outcomes. Am I being well looked after? Am I treated as a whole person, not a collection of symptoms? Does this product meet my needs, maybe even the needs I didn’t know I had? But these are much harder to measure, so we tend to look for other things to measure instead.
Activity does not necessarily equal value, just as compliance does not necessarily equal safety.
In safety-critical contexts, we often count incidents and audit compliance, while the more significant determinants of risk such as time pressure, understaffing, misaligned incentives, fear of blame and more, remain largely unmeasured. As Carsten Busch reminds us:
“An absence of accidents is not the same as the presence of safety.”
But what is easier to count becomes what is taken most seriously. We look for answers where it’s easiest to look, not where the answers are most likely to be.
Looking Where the Light Is: The Streetlight Effect
This idea is known as the Streetlight Effect – a term that dates back to at least 1942, when it appeared in a Mutt and Jeff comic strip by Bud Fisher, published in the Florence Morning News. The idea however predates 1942 considerably, with a story of the Islamic folklore character Nasreddin, who loses a ring in a dark room of his house but instead looks for it in the yard because there is “much more light” out there.
A police officer is patrolling a neighbourhood when he sees a drunk man crawling around underneath a streetlight. The officer asks if there is a problem. The man explains that he dropped a quarter, and is trying to find it. The officer has a look, and after finding no quarter, he asks the man where exactly he dropped it. The drunken man replies that he dropped it two blocks away. When the confused police officer asks him why he is searching here instead of there, the man replies, “Because the light is better here.”
It’s hard to look in the right place, so he looks in the easy one instead – even though it’s the wrong place. The drunk man chooses convenience. Organisations, more often than not, institutionalise it.
From bias to design principle
The drunken man chooses where to look based upon convenience, whilst in organisations, the Streetlight Effect becomes almost a measurement design principle based upon four “streetlight” criteria:
Visibility (we’ll measure what we can see)
Legibility (we’ll measure what easily fits into reports and dashboards)
Defensibility (we’ll measure what can be justified to management, regulators, or exec teams)
Appearance (we’ll measure what makes us look good – compared to other people, teams, or organisations)
Together, these criteria determine not just what is measured, but what is believed, funded, and prioritised. Anything that falls outside them risks becoming organisationally invisible.
And these criteria often distract us from the much more important question of “is this information useful?” Does this data actually tell us anything? More often than not, data simply begs more “but why?” questions. Were we even looking in the right place to begin with?
Organisations fall into this trap all the time – we gather the data that we can, not the data that actually matters. Psychological safety is far from immune to this – just because we can run a psychological safety survey, should we? Even if we successfully measure how safe people feel in their team, at a certain point in time, for any given value of “team”, what are we going to do with that information? At best, surveys give us more questions than answers; at worst, they can lead us down the wrong path altogether or distract us from the real issues at play.
Measurement Theatre
One reason for our tendency to measure what we can, and then make it matter, is the seductive nature of “measurement theatre” – it feels good, productive, busy and (importantly) highly visible to measure stuff. It can feel like we’re doing something valuable and having an impact. Nobody ever got sacked for generating more data. We can create impressive looking dashboards, slide decks, reports and analyses, and use these to elevate our own status and visibility within the organisation by presenting it to senior leaders or sharing the findings with the whole organisation. Yet if we never do anything meaningful with this information, or if the measurement was pointing us in the wrong direction anyway, measurement can ultimately allow organisations to appear proactive and responsible without actually confronting harm, inequality, or imbalanced power.
This happens in academic research too, not just organisational research. As Battaglia and Atkinson argue in their analysis of medical research, the pressure to publish, secure funding, and advance careers increasingly drives people toward “safe” work: studies that affirm existing paradigms rather than challenge them. They describe this explicitly as a form of the streetlight effect: studies look where it is easiest and most defensible to gather data, not where the most important questions actually lie.
What gets measured is often what careers can be built on. It influences what can get funded, resourced, and noticed, and it risks distracting us from the things that really matter.
“It is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth.” – W. Edwards Deming, The New Economics.
Methods for seeing in the darkness
What’s harder, but often far more instructive than looking where the light is, is going to the places where the answers are more likely to reside. This usually means actually talking to people, and inviting them to speak and share what’s really going on for them in the messy, lived complexity of their work. We need to explore the dark, as well as check what we can see in the light.
And there are tools that can help us here. In our work, it often involves approaches like action research (learning whilst doing the work), listening carefully to stories (sometimes described as narrative capture), and qualitative sense-making, where people surface meaning together, rather than having it extracted from them by analysts. Underpinning all of this is a simple belief: that organisational knowledge should be participatory instead of merely observational – measurement should be in service of learning, not the other way around.
“What gets measured gets managed — even when it’s pointless to measure and manage it, and even if it harms the purpose of the organisation to do so” Simon Caulkin in 2008, summarising VF Ridgway 1956.
Measurement doesn’t just describe reality (or our idealised version of it), but it shapes reality. And if metrics shape reality, then choosing what to measure is an act of governance. So by only measuring what is easy – the things that we can see under the streetlight – we don’t just notice those things, we make them important. We alter the very substrate of the system by influencing attention, behaviour and decision making.
In that sense, the Streetlight Effect becomes a form of organisational self-deception. It preserves the status quo, reinforces hierarchies, and legitimises “power over”, all while appearing neutral and “objective” And if we take that seriously, then the begging question isn’t what we can measure, but what kind of organisation and what kind of world are our measurements shaping?
“There’s beggary in the love that can be reckoned.” Shakespeare; Antony and Cleopatra.
References
Fisher, B. Mutt & Jeff Florence Morning News, 3 June 1942, p. 7
1963, Informal Sociology, a casual introduction to sociological thinking by William Bruce Cameron, Page 13, Random House, New York. (Google Books snippet view)
Battaglia M, Atkinson MA. The streetlight effect in type 1 diabetes. Diabetes. 2015 Apr;64(4):1081-90. doi: 10.2337/db14-1208. PMID: 25805758; PMCID: PMC4375074. https://pubmed.ncbi.nlm.nih.gov/25805758/
Deming, W. E. (1994). The New Economics for Industry, Government, Education (2nd ed.). Cambridge, MA: MIT Center for Advanced Engineering Study. https://amzn.to/4swn0YX
Carsten Busch. If You Can’t Measure It… Maybe You Shouldn’t: Reflections on Measuring Safety, Indicators, and Goals Paperback – 30 Oct. 2019 https://amzn.to/4jw1L5r
Merry, S.E., 2016. The seductions of quantification: Measuring human rights, gender violence, and sex trafficking. University of Chicago Press. https://amzn.to/4pnGmMZ
Login
