AI Governance Has to Earn Trust

I have spent a lot of time writing about how artificial intelligence is collapsing the boundaries between traditional technology roles. The premise felt liberating: with an intelligent assistant at your elbow, a single person can reach across product management, architectural design, and implementation to carry an idea from conception to production. In that world, the old specialty silos begin to dissolve, and the “builder” emerges as a natural description for someone with broad, end-to-end ownership.

It is an exciting thesis for engineering leaders. But when I reached out to Dr. Cat Hicks to ask how research could keep pace with an engineering role that was spilling over into adjacent disciplines, she expanded my thinking well beyond what I had already published.

As a psychologist who studies software teams and developer experience, Cat looks past job titles to examine how people actually experience their work and understand their professional identities. She separates how developers see themselves from how they believe their colleagues and managers categorize them. In doing so, she gave me a richer set of questions to examine the categories we use to understand the people doing this work.

When someone's capabilities expand, organizational reality rarely produces a neat, equitable promotion. Your skills might stretch into architecture or user research, but colleagues often continue interpreting your contributions through the narrow lens of the job title you were hired into. Worse, management frequently rewards new capability with an insidious prize: piling more uncredited work onto your plate. I wanted to understand how organizations could do better than that.

Who gets recognized?

In Becoming a Full Stack Builder, I described an individual capable of taking a bounded product problem from initial user discovery all the way through design, coding, release, and operational refinement. I still believe that expanded reach is genuinely empowering, provided deep domain expertise and specialist guidance remain accessible when complex decisions demand them.

Yet stamping a new label on someone leaves the hardest social questions unanswered. Cat pointed out that a product manager in one organization might be deeper in particular technical work than an engineer in another. Job titles are notoriously noisy proxies for competence. If we simply toss out existing roles and replace them with "builder," we risk importing the exact same biases and misunderstandings under a trendier name.

In her research, Cat emphasizes measuring identity beliefs explicitly, distinguishing between “how you see yourself” and how you believe other people categorize you. A woman working in product might perceive herself as a rigorous technical problem-solver who understands systems architecture, while feeling acutely aware that engineering peers expect her to stay confined to roadmap decks and stakeholder updates.

Learning what colleagues actually think requires going to them directly and asking. A developer's perception of peer expectations matters, but it often diverges from what teammates report. The definitive verdict comes from the organization's concrete actions: who gets handed the mission-critical architectural project, whose technical objections are taken seriously in a design review, and whose invisible labor is credited during performance evaluations. Those accounts frequently tell radically conflicting stories.

Cat pointed me toward a 2022 Perspective by Mina Cikara, Joel E. Martinez, and Neil A. Lewis Jr. on incorporating context into social psychological theory. Their work demonstrates how social categories acquire meaning and practical consequences through institutional settings and relationships. Applying that lens to engineering teams clarifies the stakes: who holds the power to confer recognition, and what opportunities flow from that judgment?

One engineer might relish the chance to take on broader product strategy and customer interviews. Another might want to dive deeper into database internals and kernel tuning. A healthy engineering organization must make room for both paths, respecting the individual's own aspirations rather than forcing everyone into a prefabricated corporate template.

The extra work has an owner

I pressed Cat on the practical operational consequence in a follow-up post:

When people become capable of contributing beyond their established roles (with the help of AI, of course), what helps organizations recognize that capability fairly, without simply expanding everyone’s workload?

She replied, “A really good question and I don’t feel I have the answer.” She noted that getting leadership teams to reflect on that question had proven valuable in her consulting, and pointed to her essay, “Organizational Fairness is a precondition for good problem-solving cultures.”

Her answer strikes at the core tension of modern AI adoption. When an engineer uses an AI assistant to generate code at triple their usual pace, they experience a personal burst of productivity. But that code does not exist in a vacuum. It flows downstream into code review queues, test pipelines, and deployment environments. If teammates are not equipped with equal leverage, one person's rapid generation becomes an unmanageable pile of unfinished verification for someone else.

This is the exhausting phenomenon Cat calls “Slop Mountain.” The engineer who taps a prompt and generates five hundred lines of plausible-looking syntax gets praised for velocity, while the senior peer who spends three hours reviewing edge cases, untangling race conditions, and writing missing tests gets zero credit for the cleanup. Cat also highlights the double standard where junior developers are chastised for leaning on AI tools by the very organizations demanding rapid AI adoption.

Her framework reminds us that organizational fairness is not an abstract human-resources platitude. It governs what people receive, how decisions are reached, and whether people feel respected. Above all, it forces us to see who actually absorbs the hidden costs of technological change.

That insight connects directly to what I want from agentic engineering. Slop Mountain is not an inevitable byproduct of artificial intelligence; it is the direct symptom of an incompletely redesigned delivery system.

Build past Slop Mountain

A software team that is already underwater reviewing human-written code has an architectural bottleneck long before AI enters the building. If you introduce automated code generation without touching the surrounding review, QA, and release machinery, the review queue inevitably balloons faster than the team can read. What executive dashboards celebrate as a leap in coding speed is experienced by the engineering floor as a crushing backlog.

That is an immature implementation of AI. When I design an agentic workflow, I start with the desired business outcome: dependable software reaching users safely, at a human cost that remains sustainable over years. From that outcome, I work backward through the entire software development life cycle (SDLC). Dropping AI into the code-authoring step while leaving review, security verification, and release unchanged is an engineering failure. The whole SDLC needs to be reimagined.

Think of it like bolting a high-powered internal combustion engine onto a wooden horse-drawn carriage, leaving passengers to rattle around inside the same fragile cabin. You have dramatically accelerated what pulls the vehicle forward, but you have left the surrounding chassis to absorb the violent shocks. That is the exact mismatch occurring across engineering organizations today: companies supercharge code generation while leaving human reviewers to absorb the impact in an unchanged process.

The goal of modern systems design is to move the human constraint upstream toward intent, architecture, and specification. Humans should decide what deserves to be built, define acceptable boundaries, and establish the evidence required to prove success. An evaluated software factory then absorbs that specification, executing implementation, automated review, quality assurance, and bounded deployment. Human judgment remains indispensable, but its role shifts from manually inspecting syntax to designing controls, setting policies, and handling novel exceptions.

To make that factory dependable, AI must earn authority far beyond basic code completion. Generating a diff is a trivial capability. Reviewing a pull request, validating security constraints, verifying performance under load, and managing a blue-green deployment are entirely distinct capabilities carrying severe consequences. Each increase in autonomy must be earned through repeatable evidence, verified against hard baselines, bounded by explicit guardrails, and supported by a tested rollback mechanism. When an automated check fails, authority must immediately pull back.

The five-level ladder I used for agentic site reliability engineering formalizes that progression. Assisted investigation and supervised delegation sit at the bottom, establishing trust before any system is granted autonomy within bounded operational corridors. Higher levels introduce autonomous remediation with human escalation, and eventually sustained operation under active monitoring. The evaluation applies to one specific capability in one environment; earning autonomy to triage a test failure grants zero authority to modify a production database schema.

This is an engineering discipline, not a promise that every company will transform overnight. Many sensitive workflows will require human eyes indefinitely. The objective is to ensure that as AI accelerates implementation, it also absorbs the corresponding verification burden, preventing one person's speed from becoming a colleague's burnout.

In this architecture, QA engineers and senior reviewers retain immense consequence. Under the earned-autonomy approach I’ve described, their job evolves into designing adversarial scenarios, calibrating automated evaluators, testing authority tripwires, and converting production anomalies into permanent regression gates. An automated reviewer requires rigorous evaluation of its own. We must prove whether our factory can distinguish a dependable architectural change from a merely convincing one.

Governance has to earn trust, too

Cat’s research on organizational fairness pushes this logic into territory that corporate AI governance usually ignores: the human systems that decide how technology is deployed.

In her work, Cat measures whether employees believe their concerns will be taken seriously, and whether leadership accounts for differing team contexts rather than enforcing blunt mandates. While her fairness battery continues through formal validation and reliability testing, the profound opening for engineering leaders is treating worker experience as first-class evidence about the governance environment itself.

That evidence belongs alongside technical telemetry and uptime metrics. Are senior engineers carrying invisible overtime to clean up AI pull requests? Can a developer challenge an arbitrary AI adoption mandate without being branded a dinosaur? Do teams understand why a specific model or tool was granted deployment privileges, what boundaries restrict it, and what failure would revoke its license?

Those questions point to a hard truth: trust in AI is meaningless without trust in the human decisions governing it. An automated tool can operate with technical precision inside a corporate process that developers experience as arbitrary, coercive, and unfair. We should investigate reports of developer disenfranchisement with the exact same rigor we bring to an infrastructure incident.

A periodic survey is only a starting signal. Engineering leaders should correlate developer sentiment against review durations, rework rates, escaped production defects, and records of what happened when an engineer raised an objection. High satisfaction cannot excuse a failed security gate. Equally, an executive dashboard glowing green with automated pull requests cannot hide the human cost if those metrics were achieved by burning out senior staff.

Imagine a team where automated CI passes consistently, yet senior reviewers report surging after-hours work. Honest governance investigates both data points. Perhaps the test suite is blind to subtle runtime regressions. Perhaps the review policy routes every automated pull request to the same two overburdened tech leads. Perhaps a team's reported productivity jump is being propped up by volunteer labor that never shows up on a JIRA burndown chart. Each reality demands a structural fix.

Opening a feedback channel is theater unless people see tangible evidence that speaking up changes decisions. Leaders must publish the criteria behind AI adoption, explain the conditions that would alter policy, log developer concerns publicly, and demonstrate visible course corrections.

In that light, fairness feedback becomes another layer of earned autonomy for leadership itself. The executives and managers governing AI must prove that they can distribute operational burdens equitably, respond to frontline evidence, and reverse bad decisions. Their managerial authority should be subject to the same continuous verification they impose on automated systems.

This is the deeper lesson Dr. Cat Hicks offered me. She showed that professional identity and organizational fairness are not soft distractions from technical architecture; they are the bedrock that determines whether an engineering organization can thrive. Earned autonomy cannot end at the boundary of a software container. It must encompass the social contract between technology leaders and the people doing the work.

The engineer whose role is expanding deserves to know how their contributions are being valued, to challenge the metrics used to evaluate them, and to see tangible evidence that their voice carries weight. A human-led software factory owes its people a governance process they can inspect, question, and trust.

For anyone who wants to explore Cat’s insights more deeply, her book, “The Psychology of Software Teams”, is available now directly through her site. I am grateful to her for the conversation, and for handing me better, harder questions to bring to my own practice.