AI Inference Cost Belongs in COGS Not OpEx

**Inference cost is COGS, not OpEx.** For an AI startup, the compute you spend to run a model in production and generate the output a paying customer consumes is a direct, variable cost of delivering the product, so it belongs in cost of goods sold (also called cost of revenue) and reduces gross margin. Only the compute used to build or improve the model, such as training and fine-tuning, has a credible claim to sit in operating expense as research and development.

Is inference cost COGS or OpEx for an AI startup?

Inference cost is COGS. Inference is the compute an AI startup spends to run its model in production and generate the response a paying customer consumes, so it is a direct and variable cost of delivering the product and belongs in cost of goods sold, not operating expense. The matching principle in accrual accounting requires costs to be recorded against the revenue they generate, and because inference scales with usage and therefore with revenue, it matches revenue closely. Booking it anywhere else, such as inside research and development or a generic infrastructure line under OpEx, overstates gross margin and hides the true unit economics of the business. The narrow exception is model training and fine-tuning, which build a durable asset and can reasonably sit in R&D.

Why inference is a cost of revenue and not an operating expense

The line between COGS and OpEx is not a matter of taste. Cost of goods sold captures the costs you cannot avoid if you want to deliver one more unit of the product. Operating expense captures the costs of running the company that do not scale one to one with each unit sold, such as sales, marketing, general and administrative spend, and most research and development. The test is direct attribution and variability. If the cost rises every time a customer uses the product, it is almost certainly a cost of revenue.

Inference fails no part of that test. Every prompt, every generated image, every agent run, and every token returned to a paying user triggers a metered charge from a model provider or a GPU bill from your own cluster. You cannot serve the request without paying for the compute. That is the textbook definition of a cost of revenue.

This is not a novel treatment invented for AI. Traditional software companies already book cloud hosting, content delivery, third party API fees required to run the product, and payment processing inside cost of revenue. Inference is simply the newest and largest line in that same category. Revenue recognition standards such as ASC 606 and IFRS 15 govern when you record the revenue from a customer contract, and the matching principle then pairs that recognized revenue with the direct costs of fulfilling it. Inference is one of those direct fulfillment costs. Treating it as anything else breaks the match between what you earned and what it cost you to earn it. If you want the fuller picture of how these costs move together, our primer on unit economics from day one walks through the same logic at the level of a single customer.

The one real exception: training versus inference

The cleanest way to keep your books honest is to separate the compute that builds the model from the compute that serves it.

Training and fine-tuning can sit in R&D

Training a foundation model, fine-tuning an open weight model on your own data, or running large evaluation sweeps creates a durable asset that you expect to use across many future customers. That spend behaves like research and development, and it is defensible to classify it as an operating expense, or in some cases to capitalize it, rather than to bury it in the cost of this quarter's revenue. Training is lumpy and forward looking. It does not scale with today's usage.

Inference is delivery, so it is COGS

Inference is the opposite. It is consumed the moment a customer is served and it never becomes an asset. A run you paid for yesterday delivers no value tomorrow. Because it is tied to the act of delivery, it is a cost of revenue in the same way that server time is for any hosted product.

Two gray areas are worth naming honestly. Inference spent serving free tier or trial users can be argued into sales and marketing as a customer acquisition cost, though many teams still book it in COGS for simplicity and conservatism. And experimental inference during model development, such as prompt engineering and internal testing, can travel with R&D. The paid delivery path, which is the part that actually determines your gross margin, is COGS without ambiguity.

What hiding inference in OpEx does to your margin

The temptation to park inference outside COGS is strong because gross margin is the number investors, acquirers, and your own board watch most closely. Move the largest variable cost off the cost of revenue line and your reported gross margin can appear to jump materially on paper. Nothing about the business changed. You simply stopped telling yourself the truth.

That truth is what makes AI companies structurally different from classic software, and it is why the classification matters so much more here than it did for a pure SaaS product.

Read that with inference correctly booked as COGS and a 50 to 60 percent gross margin is the honest baseline you plan against. Read it with inference hidden in OpEx and you will convince yourself you are running an 80 percent margin software company, price as if you were, raise as if you were, and then discover the gap the first time compute usage spikes. The classification is not cosmetic. It changes how you price, how much runway a dollar buys, and whether your growth is actually creating value. How much revenue inference consumes varies widely by product and workload, so the safest move is to measure your own number rather than trust a benchmark. The pressure on that number is not static either, because the regional cost picture in LATAM differs from the US default.

How a venture studio books it from day zero

This is where the operating discipline matters more than the accounting theory. Avante Ventures co-founds AI-native companies for Brazil and LATAM, working alongside founders from day zero with capital and hands-on building, and one of the first things that build touches is the financial model. We set up the chart of accounts so inference lands in cost of revenue from the first invoice, not after a later cleanup when a diligence process forces the correction.

The reason is not conservatism for its own sake. When gross margin is honest from the start, three decisions get easier. Pricing gets anchored to a real margin rather than a flattering one, so founders do not underprice a product that costs a quarter of its revenue to serve. Fundraising conversations stay credible, because the margin an investor sees is the margin the company will still show after diligence. And the engineering roadmap gets a clear target, because a cost that sits visibly in COGS is a cost the team is motivated to attack.

Attacking it is the other half of the discipline. Booking inference as COGS tells you the margin problem exists. Defending the margin is an engineering job, largely a routing job: sending each request to the cheapest model that can complete it, caching aggressively, batching where latency allows, and reserving the frontier models for the tasks that genuinely need them. Our model routing and inference cost playbook covers those tactics in depth. The accounting makes the problem visible, and the routing makes it smaller. A studio can do both across a portfolio at once, which is part of why building AI-native companies inside a shared operating stack is more capital efficient than doing it alone.

A quick classification guide

Use this to place any AI compute cost on your income statement.

The durable rule underneath all of it is simple. If the compute is spent to deliver what a customer paid for, it is a cost of revenue. Book it there, watch your real gross margin, and build the company on numbers you will not have to walk back.

• Compute to serve a paying request in production: COGS, every time.
• Cloud hosting, vector database, and third party API calls required to deliver the product: COGS.
• Model training, fine-tuning, and large evaluation runs that build a reusable asset: R&D, inside OpEx, or capitalized where appropriate.
• Inference for internal development, prompt testing, and experimentation: R&D.
• Inference serving free trial users: defensible as sales and marketing, though many teams keep it in COGS for a cleaner, more conservative margin.

Preguntas frecuentes

Is inference cost COGS or OpEx for an AI startup?

Inference cost is COGS. The compute an AI startup spends to run its model in production and serve a paying customer is a direct, variable cost of delivering the product, so it belongs in cost of revenue and reduces gross margin. The matching principle pairs it with the revenue it generates. The only credible exception is model training and fine-tuning, which build a reusable asset and can sit in R&D under operating expense.

Does booking inference as COGS reduce gross margin?

Yes. Because inference is a cost of revenue, every dollar of inference lowers gross margin directly. This is why a16z found AI companies often run 50 to 60 percent gross margins versus the 60 to 80 percent or more of classic SaaS. Hiding inference in OpEx would inflate the reported margin without changing the underlying economics of the business.

Should model training also go in COGS?

Usually not. Training and fine-tuning build a durable asset you reuse across many customers, so they behave like research and development and belong in operating expense, or can be capitalized in some cases. Inference is consumed the instant a customer is served and never becomes an asset, which is why it is COGS. Keeping training and inference separate is the cleanest way to book AI compute.

Why does it matter whether inference is COGS or OpEx?

Because gross margin drives pricing, fundraising, and how much runway a dollar buys. Booking inference as COGS gives you an honest margin to plan against, while parking it in OpEx can inflate reported gross margin materially and lead you to underprice and misjudge unit economics. It also creates a clear engineering target, since a visible COGS line is one the team is motivated to reduce through model routing and caching.