Toby Allen

Six checks later: the on-behalf-of bug that took two sessions to find

· Auth0, MCP, AI Agents, AI-Assisted Development

This is part five of building the live demo for my apidays Australia 2026 workshop with Claude Code. Part four covered five bugs a real click found in one afternoon that every automated check had missed. This post is about a sixth, that took a lot longer than an afternoon: on-behalf-of (OBO) token exchange, the RFC 8693 flow the chatbot backend needs to call its own MCP server as the signed-in citizen rather than as itself, refused to work at all, for two sessions, against a build plan that had been checked - repeatedly, adversarially, against Auth0's own documentation - and still couldn't tell me what was wrong.

The demo's chat backend needs to reach its MCP server carrying the caller's own identity, not a generic service identity, so that Auth0 FGA can evaluate every tool call against the actual citizen or caseworker asking, not against "the chatbot" as an undifferentiated principal. OBO exchange is exactly the mechanism for that: swap a token you already hold for one scoped to a new audience, with Auth0 preserving the original subject on the exchanged token rather than substituting the exchanging client's own identity. I'd built this for one hop already - submit_appeal calling a downstream case service - and marked it done. Generalising it to a second hop, chat backend to MCP server, is where this story starts.

The error that meant nothing

The chat route logged failed_on_behalf_of_token_exchange in Auth0's own event stream, description: "This client cannot exchange access tokens for this audience." That sentence names two things - a client, and an audience - and reads like it's telling you one specific relationship is wrong. It isn't. It's the same error Auth0 returns for several genuinely different misconfigurations, and figuring out which one you've got requires checking a property the error text doesn't mention at all.

I asked Claude to work through the documented requirements for OBO exchange one at a time. It did, methodically, over what turned out to be six separate rounds across two sessions:

  1. No error handling at all, first. The very first symptom wasn't the token-exchange error - it was silence. connectToMcpServer() had no try/catch, so any failure threw an unhandled rejection Next.js turned into a bare 500 with no message. Claude wrapped it, and only then did we get to see any real error text.
  2. The model then made up an explanation. Once errors stopped crashing the route, the system prompt - written back when the only failure mode anyone had seen was an FGA denial - told the model to interpret every tool failure as a permissions problem. It started telling me my access was "out of date" for an error that had nothing to do with permissions. Claude rewrote both system prompts to quote a system error's exact text rather than paraphrase it, which is what actually got the real error into my hands for the first time.
  3. The real text, once visible: "Not supported JWT type in subject token." Auth0Client had never been configured with authorizationParameters.audience, so login was issuing an opaque access token, not a JWT - and RFC 8693 exchange needs a real JWT as its subject_token. This had been silently breaking both OBO flows in the app, the new one and the original submit_appeal one, the whole time. Added the missing audience config.
  4. That fix produced error four: "This client cannot exchange access tokens for this audience." The obvious read is a missing client grant. Checked - a correctly-shaped grant already existed. Checked its subject_type anyway and found it had been created as "client" instead of "user" - a real mistake from an earlier session, invisible until this was the first time anyone actually pushed a real token through the flow. Recreated it correctly.
  5. Same error, still. At this point I copied a real, fresh access token straight out of the app's own debug token viewer and replayed the failing request by hand, rather than continuing to infer anything from the app's behaviour. The same error came back for every audience I tried - including the token's own existing audience, and the case-service audience that was supposedly already working. That detail should have been the whole answer: if asking for a token's own audience back still fails, the problem can't be about the destination audience at all.
  6. An extensive re-verification pass that confirmed everything and explained nothing. app_type: resource_server, resource_server_identifier set, token_exchange.allow_any_profile_of_type correct, token_endpoint_auth_method set, the client grant's subject_type and scopes, the target API's subject_type_authorization policy, is_first_party - all individually correct. A theory about a tenant-level feature gate looked promising until Auth0's own changelog and current documentation ruled it out directly - OBO for this use case had been generally available since May. Every documented requirement, checked twice, against a live tenant, still failing identically. That's where the previous session ended: handed back with a full breakdown of everything verified, nothing resolved.

The field nobody thought to question

I came back to this fresh and read Auth0's own OBO documentation and a support article addressing this exact error family, side by side with the actual client configuration in the tenant. The requirement that had been checked five times over is real - a client must be app_type: resource_server with resource_server_identifier set, and it needs a user-delegated client grant to the destination audience. What none of those five checks had verified was what value resource_server_identifier actually needed to hold.

It isn't compared against the audience you're requesting in the exchange call. It's compared against the aud claim already sitting inside the token you're handing in as subject_token. The client performing an OBO exchange has to be registered as the resource server matching whatever token it's being handed, completely independent of where that token is headed afterwards. Our OBO client had resource_server_identifier: https://chatbot-backend.apidays.local - a resource server nothing in the app had ever actually issued a token for. The citizen's real access token, the one being handed in as subject_token, was audienced for https://smartgov-app.apidays.local, the app's own login audience. Two different strings, silently required to be the same one, checked five times for existence and never once for equality.

This also explains the one clue that should have cracked it earlier - the failure that persisted even when requesting the token's own existing audience back. That's not a coincidence about the destination; it's proof the client was never eligible to perform the exchange on that token at all, no matter what audience it asked for afterwards.

Trying to fix it directly hit one more wall: resource_server_identifier can't be changed with a PATCH once a client exists.

{"statusCode":400,"error":"Bad Request","message":"Payload validation error: 'Additional properties not allowed: resource_server_identifier'.","errorCode":"invalid_body"}

It's set once, at creation, full stop. The fix wasn't editing the broken client - it was creating a new one with the identifier right from the start, re-pointing the client grants at it, and swapping the app's environment variables over to the new client's credentials.

A second, entirely independent setting was still needed on top of that: the target API also has to have Allow Skipping User Consent for Verifiable First-Party Clients switched on, since the exchanging client is first-party and Auth0 otherwise expects an interactive consent step that an OBO exchange has no way to satisfy. That one I actually found myself, mid-session, from the same support article, faster than the model had gotten to applying it.

The chat backend's OBO client must be registered as the resource server matching the citizen token's own audience - a separate requirement from the client grant authorising the destination audience it's exchanging into

Why a plan built on the real docs still missed it

The build plan for this project tracked OBO faithfully, in the sense that mattered to everyone reading it: a checklist line, "OBO flows for agents," moved between unchecked, checked, and corrected as the actual scope of what was built changed. It was even caught out once already - an earlier adversarial review found the line had been marked done on the strength of one hop when the requirement was general, and got corrected. That's a plan doing its job.

What it never did, because nothing ever asked it to, was encode the mechanism the documentation actually describes. "OBO flows for agents: yes/no" is a claim about intent. "This client's resource_server_identifier must equal the subject token's own aud" is a claim about a specific field's specific value, and it was never written down anywhere as a thing to be right or wrong about - not in the plan, not in a comment, not in a script. The original OBO client for the case-service hop was created by hand, live against the tenant, and whoever set that value at the time either got it right by chance or the mismatch simply hadn't been exercised with a real token yet. When the pattern was generalised to a second hop, the new client's identity was set up separately, and nothing forced a check that it satisfied a rule nobody had named.

A plan can be accurate about what exists and still be nowhere near sufficient to prevent a specific class of bug, if the thing that actually breaks was never a line item the plan had any occasion to be accurate or inaccurate about. The Auth0 dashboard's own "Add Application" flow, tucked under an existing API, avoids this entirely by construction - clicking that button links the identifier for you, so the relationship is never something a person has to state correctly by hand. Do the equivalent step through the Management API instead, as this project's own tooling did, and that structural safety net is simply gone, with nothing standing in for it.

Final Thoughts

Five of the six rounds in this chain were real, necessary, individually correct fixes - the kind of debugging that looks like clean progress the whole way through, right up until the very end when it doesn't resolve anything. The sixth wasn't a new bug at all; it was the same field, checked for the wrong property, twice, by two different sessions, because "is this set" and "is this set to the value the request in front of you actually needs" look identical until you write down which one you're asking. I'd rather have caught that from the plan than from a support article and a 400 response - the next time OBO gets set up for a new hop in this app, that equality is going in as an actual assertion, not a step performed correctly from memory.

With OBO actually working end to end, the next stretch of work on this demo is deliberately less technical: giving it some real branding and a bit of humour, since a room full of people watching a live authorisation demo deserves more personality than a bare admin console with grey buttons.