The Provenance Problem — Why AI Governance Is Collapsing Into Proof of Human Control

It did not begin as a governance discussion.

No one sat down and declared that the system had changed. There was no formal shift, no industry-wide signal, no moment where everyone agreed that something underneath the way software was being built had quietly broken.

It appeared the way real constraints usually do.

Unannounced. Unlabeled. Easy to miss if you were not paying attention.

A line item in an MSA.

Not a sweeping requirement. Not a mandate for constant visibility. Just a clause that gave the client the right, if something went wrong, to ask a question later.

Where was AI used in producing this?

That was it.

And in the moment, it didn’t feel like much.

Engineering moved past it quickly. Another tracking problem. Another layer to instrument. Something that could be solved with the right combination of tooling and discipline. You could almost feel the answer forming before the question had fully settled. Route inference through a controlled layer. Tag commits. Capture prompts. Store logs. Build the audit trail. Move on.

It fit inside a pattern they already understood.

Legal didn’t experience it the same way.

They weren’t hearing a process question. They were hearing a future confrontation. You could feel the hesitation before anyone articulated it. If that question gets asked later, can we answer it in a way that holds? Not internally. Not for ourselves. For someone who is trying to challenge it.

Because if the answer doesn’t hold, the rest of it stops mattering.

It doesn’t matter how the system was built.

It doesn’t matter whether it works.

It doesn’t matter if everyone involved knows exactly what happened.

If it can’t be defended, it doesn’t exist in the way that matters.

Same clause.

Two interpretations that did not overlap.

And the tension wasn’t coming from disagreement.

It was coming from the fact that both interpretations were valid, but they were answering different questions.

That realization does not arrive cleanly. It shows up as a gap. The conversation keeps moving, but nothing resolves. One side continues to build toward an answer that feels complete. The other keeps returning to something that refuses to collapse into a solution.

What are we actually proving?

Not what happened.

What are we proving.

That question takes time to land. And once it does, it begins to reorganize everything around it.

Because the instinct is to answer with what feels measurable.

We are proving whether AI was involved.

It sounds precise. It sounds like something you can instrument. It suggests a boundary that can be defined and enforced. You can already see the system that would support it. Flags. Tags. Traceability. A report that answers yes or no.

And then you try to apply it.

The function that started as a suggestion and ended as something else. The bug that was understood in a browser and fixed by hand. The idea that came from a model but was shaped, rewritten, and refined until its origin no longer matters. The fragment that was generated, edited, and absorbed into something new.

You start trying to draw the line.

And every time you draw it, it shifts.

Not because something is wrong.

Because that is how the system behaves.

If you follow it all the way through, the answer becomes uncomfortable.

Everything is touched.

And a question that resolves to everything stops being useful.

So the system sharpens the question.

Not whether AI was involved.

Whether a human can claim what was produced.

That shift does not feel dramatic.

But it changes the problem completely.

Because ownership is not something you infer.

Ownership is something you defend.

And the moment you feel that shift, the idea that this is a tracking problem starts to fall apart.

You are not building visibility anymore.

You are building something that has to hold under pressure.

That is the Provenance Problem.

Not provenance as a record.

Provenance as a structure that can withstand examination.

A chain that connects intent to outcome in a way that does not collapse when someone starts pulling on it. Not just what happened, but how it happened, why it happened, where decisions were made, and who is accountable for the result.

The instinct that follows is familiar.

Capture more.

If there is uncertainty, increase visibility. If something might be questioned later, record it now. Prompts. Outputs. Intermediate steps. Histories. Build a system that retains everything so nothing has to be reconstructed.

It feels responsible.

It feels like control.

But that instinct was built in systems that behave differently.

And you can feel the mismatch as soon as you try to rely on it.

The same prompt does not produce the same output over time. Models evolve. Context shifts. What was captured at one moment does not replay cleanly later. The relationship between input and output drifts in ways that are not fully controllable.

And even if it did not, you encounter something more basic.

The system cannot fully observe itself.

The code that never touched the repository. The conversation that never entered a logging layer. The sequence of prompts that shaped a direction instead of producing a discrete artifact. The decisions that existed entirely in someone’s head before they showed up in the work.

This is not an exception.

This is how the work actually happens.

Which means the idea of complete traceability is not just difficult.

It is incompatible with the way people operate.

And this is where the instinct to log everything begins to fail.

Because it mistakes data for proof.

A record tells you what was captured.

It does not tell you whether what matters can be defended.

And governance, when it is tested, is not concerned with whether information exists.

It is concerned with whether it holds.

That is where the shift becomes unavoidable.

From completeness to coherence.

Completeness attempts to eliminate uncertainty by capturing everything. It produces volume. It creates the sense that nothing can be questioned because everything has been recorded.

Until it is tested.

Then the weaknesses appear. Too much noise. Too many gaps that matter more than what was captured. No structure that explains the system in a way that survives scrutiny.

It breaks.

Not because the data is wrong.

Because the data was never the answer.

Coherence takes a different path.

It accepts that not everything can be captured and focuses on what must be made clear.

What tools were used. What constraints governed them. Where human control was applied. How outputs were evaluated. Who stands behind the result.

It does not attempt to reconstruct every step.

It makes the system understandable.

And that is what survives.

There is a separation inside this that becomes clear under pressure.

Provenance is not governance.

Governance defines what is allowed before the work begins. The rules. The constraints. The boundaries that shape behavior.

Provenance is what remains after. The explanation. The evidence that those boundaries were respected.

One establishes the conditions.

The other demonstrates that those conditions were met.

For a system to hold, the two have to align.

If governance defines something that provenance cannot demonstrate, the system fails under examination.

Not because it did not work.

Because it cannot be explained in a way that justifies its outcome.

The instinct is to reach for something familiar.

Treat it like dependency management. Track what was introduced. Track where it came from. Track how it integrates. You do not need everything. You need enough to explain the system.

That instinct is grounded in experience.

It is also where the analogy breaks.

A dependency is stable.

You know what it is. You know how it behaves. You know what version you are working with.

It anchors the system.

An AI output does not do that.

It shifts.

It evolves.

It does not provide a fixed point.

And when that becomes clear, the analogy stops working.

Because in a system where the machine does not provide stability, something else has to.

The only thing left that can do that is the human.

Not assumed.

Not implied.

Demonstrated.

Where the intent began. Where it changed. Where it was constrained. Where it was validated. Where someone made a decision that shaped the outcome into something they are willing to stand behind.

That is the chain.

And that is where ownership now lives.

This is where the system remains unsettled.

A coherent chain satisfies a significant amount of internal and regulatory pressure. It demonstrates process. It demonstrates control. It shows that the system is not operating without boundaries.

But it has not yet been tested everywhere it will be.

The difference between passing an audit and surviving a challenge is not abstract. One asks whether the system followed its own rules. The other asks whether those rules hold when someone is actively trying to break them.

That second test is coming.

You can feel it.

In contracts. In diligence conversations. In questions that did not exist three years ago and now appear without warning in rooms where decisions are made.

This is how systems change.

Not through formal resets.

Through pressure that arrives early, unevenly, and quietly until it is no longer avoidable.

That line item in the MSA was not an anomaly.

It was an early signal.

Not about tooling.

About ownership.

The system is moving from a world where building something was enough to a world where explaining how it was built is part of what gives it value.

That shift does not stay contained within engineering.

If ownership cannot be demonstrated, the asset weakens. If the asset weakens, the narrative around it weakens. And when the narrative weakens, capital begins to move differently.

You do not feel that immediately.

You feel it later.

In diligence that takes longer. In questions that cut deeper. In confidence that erodes just enough to change decisions.

By the time it is visible, it has already been happening.

The teams that see this early will not try to capture everything.

They will decide what must be true and build systems that make those truths visible and defensible.

They will understand that the problem was never about tracking the machine.

It was about establishing the human.

Everyone else will continue to log.

Continue to store.

Continue to build systems that feel like control.

And eventually realize they have built a record of activity without an answer to the only question that mattered.

Because the question was never about what the machine did.

It was always about whether the human can still claim it.