Who owns the AI bill: FinOps levers vs engineering levers

Cloud FinOps spent a decade on instances, idle disks, and reserved capacity. Those still matter. Generative AI does not behave like them. The meter is tokens, retries, context, and loops. Procurement can see the invoice. It cannot rewrite the prompt.

This note is a working split: what a central FinOps / Cloud Financial Management team can centralize, and what still requires engineering changes in the model gateway, RAG stack, or agent harness. Sources are current through mid-2026.

The shape of the problem

Inference prices collapsed while enterprise spend exploded. Stanford’s 2025 AI Index reported GPT-3.5-class inference falling from about $20 per million tokens in late 2022 to about $0.07 by late 2024. Menlo Ventures put enterprise generative AI spend at $2.3B (2023), $13.8B (2024), and $37B (2025). The FinOps Foundation’s State of FinOps 2026 survey found 98% of respondents now manage AI spend, up from 63% the year before; AI cost management was named the most important skill to build (Pollick, 2026).

That is the Jevons trap: cheaper tokens invite more features, fatter context, and extra judge/guardrail calls. Unit price down, total bill up.

A useful identity, adapted from Pollick:

monthly cost ≈ calls × (input tokens × input price + output tokens × output price) × (1 + retry/agent multiplier)

Every term is a design choice. Finance does not set max_tokens or whether the agent may loop eight times.

Ofir Zukovsky’s LinkedIn piece on production LLM margin makes the same architectural point: one frontier model for every request is the default that quietly destroys gross margin; stacked routing, caching, compaction, batch, and semantic cache are what cut 47–85% of spend without touching quality on the cheaper paths (Zukovsky, 2026).

Two layers, not one team

StackSpend draws a clean line:

Layer	Question	Typical owner	Output
LLMOps	Did the workflow behave correctly?	AI / platform engineers	Traces, prompt versions, eval scores, latency
LLM FinOps	Did it run at the right cost?	Eng leaders + finance + platform	Spend by provider/model/feature, pacing, anomalies

You can have excellent traces and still no idea which product owns the Bedrock line. You can have a daily AWS bill and still not know a prompt change doubled output tokens.

Treat cost per successful task as the product metric. Cost per token is a vanity number: output tokens often cost several times input, and a cheaper model that fails and retries is more expensive (Zukovsky, 2026).

What FinOps can centralize

These are the levers that look like classic Cloud Financial Management. A platform or FinOps team can own them once for the company, as long as engineers emit the right tags.

1. Attribution that hits the invoice

On Amazon Bedrock, native attribution lands in Cost Explorer / CUR 2.0 as billed dollars (aggregated per usage type per day), not per prompt (AWS, Track usage and costs):

• IAM principal attribution — per user or team on the bill.
• Application inference profiles — tag a profile ARN used instead of the model ID; activate those tags as cost allocation tags (application inference profiles).
• Projects / Workspaces (bedrock-mantle) — application-level tags on newer APIs.

If a gateway calls Bedrock for everyone, the bill shows the gateway role. FinOps should require either per-tenant RoleSessionName / session tags, or accept that invoice dollars stay coarse and per-user truth lives in logs.

FinOps owns: tag taxonomy, activating cost allocation tags (not retroactive; ~24 hours to appear), Cost Explorer views, Budgets, Cost Anomaly Detection, and a rule that untagged inference is a defect.

2. Reconciling CUR without lying to yourself

CUR 2.0 splits Bedrock into input, output, cache-read, and cache-write usage types, plus in-region vs cross-region prices. Summing only input+output undercounts caching workloads (understanding CUR data). CUR still does not carry requestId; join invocation logs to CUR at model + usage-type grain.

FinOps owns: the monthly reconciliation playbook and the dashboard that shows cache-write as a first-class cost, not a footnote.

3. Commercial shape of capacity

Once usage is steady, FinOps (with capacity engineering) can choose:

• On-Demand vs Batch (~50% off for async jobs).
• Provisioned Throughput / Reserved service tier for throttle SLAs (this is a premium for predictability, not a discount — see the Bedrock playbook).
• Global cross-region inference (~10% vs geographic in AWS’s 2026 practitioner series) when data residency allows.

These are centralized decisions. Enabling them still needs a one-line model-id / inference-profile change in the gateway.

4. Budgets, showback, and “design-time” unit cost

FinOps can force a cost-per-request or cost-per-ticket target into the feature spec, same as a latency SLO. Pollick’s operating loop: attribute → budget at design time → gate in CI with evals → then optimize. A budget without a failing build is a wish.

What FinOps cannot do alone: write the pytest that fails when median cost per successful task exceeds $0.04.

5. Provider and region strategy

Multi-cloud or OpenRouter-style routing, enterprise discounts, and “which account may call Opus” are policy. Enforcement belongs in the gateway, which is a platform engineering artifact FinOps should sponsor, not a spreadsheet.

What only engineering can change

These will not move if you only buy RIs or nag teams about tags.

Model routing and “don’t default to the frontier”

Frontier vs mid-tier can be 5–50× per token. RouteLLM (ICLR 2025) reported up to 85% cost reduction while holding ~95% of GPT-4 performance on routed workloads (Zukovsky, 2026). Bedrock Intelligent Prompt Routing claims up to ~30% inside a family (AWS IPR). Custom classifiers (even “under 2k tokens and no code → Haiku”) are 20–40% in AWS’s high-volume framework (re:Post Part 1).

Owner: platform + product engineers. FinOps can report “% of tokens on premium models” but cannot classify intent.

Prompt structure, cache checkpoints, context compaction

Provider KV cache only hits if static prefix (system, tools, RAG corpus order) is byte-stable and sits before dynamic user content. A 5-minute cache write can cost 1.25× uncached input; 1-hour TTL 2.0×. Hit rate under ~50% can be cost-negative (re:Post Part 1; prompt caching docs).

Compaction (summarize after N turns, retrieve snippets not PDFs) is prompt/RAG work. Semantic cache is an application service.

Owner: whoever owns the prompt template and the agent loop.

Harness: retries, max steps, judges, max_tokens

Agent steps multiply cost linearly. A quality judge doubles calls. Client errors still burn tokens (and cache writes) before 4xx; retries 2–5× that waste. AWS treats client-error rate >10% as a P1 cost leak (re:Post Part 1).

Bedrock also reserves max_tokens against TPM and some Claude models use a 5× output burndown on quota even though you are billed for actual tokens (operations playbook). Oversized max_tokens is both a cost and a throttle problem.

Owner: harness / LLMOps. FinOps should demand a hard step ceiling as policy.

Distillation, RAG index hygiene, eval-gated model swaps

Teacher→student distillation (Bedrock: up to 75% cheaper, ~2% accuracy loss on some RAG tasks, AWS cost optimization) is an ML engineering program. Knowledge Base indexing frequency is a data-engineering cost, not an FM token line, but it still sits on the AI bill (AWS CFM Bedrock post).

A RACI you can actually run

Lever	FinOps / CCoE	Platform / gateway	Product / prompt owners
Cost allocation tags, AIP, IAM mapping	A	R	C
CUR vs logs reconciliation	A	C	I
On-Demand vs Batch vs Reserved	A	R	C
Default model allow-list	C	A	R (exceptions)
Prompt cache layout	I	C	A
Agent step / retry caps	C (policy)	A (enforce)	R (tune)
Per-request metadata (team, feature)	C (schema)	A	R
Cost-in-eval CI gate	C	A	R
Distillation / custom models	I	C	A

R = does the work, A = accountable, C = consulted, I = informed.

What to do this quarter

1. Stop one undifferentiated Bedrock line. Application inference profiles or Projects, plus invocation-log metadata for feature/request grain.
2. Publish unit cost, not just $/month. Pair it with P90/P50 user-cost ratio if you have a SaaS SKU (heavy users can be 10–40× median at the same subscription price).
3. Put a cost assertion next to quality evals. Same PR, same fail.
4. Cap the harness. Max model calls per task; alert on cap hits.
5. Treat cache-write as a budget line. If CloudWatch CacheWriteInputTokenCount dominates CacheReadInputTokenCount, engineering owns a prompt-structure incident, not a “AWS is expensive” narrative.

FinOps for AI is still FinOps: inform, allocate, optimize. The optimize step moved into git.

Sources

1. FinOps for AI: Why LLM Cost Is an Engineering Problem — Pollick, Jun 2026; FinOps Foundation 2026 stats, unit cost, CI gates.
2. 5 Levers that cut LLM spend 85% — Zukovsky, Jun 2026; routing, cache, compaction, batch, semantic cache.
3. LLMOps vs LLM FinOps — StackSpend, Mar 2026.
4. Track usage and costs in Amazon Bedrock — AWS docs.
5. Effective cost optimization strategies for Amazon Bedrock — AWS ML blog, Jun 2025.
6. Optimizing Amazon Bedrock costs at scale (Part 1) — AWS re:Post, Jul 2026.