GPT-6 Astra Benchmarks: What OpenAI Publishes and What It Does Not

OpenAI does not publish numerical benchmark scores for GPT-6 Astra in its model documentation. What the docs give instead are qualitative ratings, technical specifications and a positioning statement, and those turn out to be more useful for deciding whether to use the model than a table of scores would be. The model page rates reasoning as highest and speed as fast, and lists a 1,050,000 token context window with 128,000 maximum output tokens.

That absence is worth stating plainly rather than filling with numbers from elsewhere, because benchmark figures circulating without a primary source are frequently wrong.

Key points

• No numerical benchmark table exists in the documentation. OpenAI rates reasoning and speed qualitatively rather than publishing scores.
• The specifications are published and precise: 1,050,000 context window, 128,000 max output, knowledge cutoff 30 April 2026.
• Reasoning effort has five settings, and any benchmark figure is meaningless without stating which one produced it.
• Price is the clearest comparison available, at $10 input and $50 output per million tokens against $4 and $20 for GPT-5.6 Sol.
• Your own evaluation beats any published score, because benchmark performance and performance on your task are only loosely related.

What the documentation actually states

Rather than scores, OpenAI provides a small set of comparative ratings and a positioning claim.

Astra is rated highest for reasoning and fast for speed. It is described as the strongest option for demanding work carried through from beginning to end, with documented use cases covering difficult reasoning problems, software development, computer control, research tasks and producing long documents.

The technical parameters are specific and verifiable.

Set against the rest of the range, the picture is clearer. GPT-5.6 Sol, Terra and Luna all share the same 1,050,000 context window and the same 128,000 output ceiling, at $4, $2 and $0.20 input respectively. The context window is not what distinguishes GPT Astra from GPT Sol. Capability on hard problems is, and OpenAI expresses that as a rating rather than a number.

How the frontier lineup compares on published specs

Since neither vendor publishes benchmark tables, specifications and price are the comparable facts available. Anthropic documents its lineup in the same style, and the Claude models overview gives Claude Fable 5.1 a 1M token context window, 128K max output and a knowledge cutoff of June 2026, at $10 per million input tokens and $50 per million output. Claude Opus 5 sits at $5 and $25 with the same window, positioned for complex agentic coding and enterprise work.

Astra and Fable are priced identically and carry near-identical windows. Both vendors describe their top model in terms of hard reasoning and long-running agent work rather than raw intelligence scores, and both offer a substantially cheaper tier one step down with the same context capacity. That convergence is itself informative: the frontier is currently competing on reliability over long tasks and on cost per outcome, not on a single number.

The practical consequence for anyone choosing is that price will not separate these options, and no published score will either. Your own evaluation is the only thing that does, and via the API it is straightforward to run the same set of tasks through both.

Why the absence is worth taking seriously

It would be easy to treat a missing benchmark table as an oversight, or to fill the gap with figures found elsewhere. Both are mistakes, and the second is the more expensive one.

Numbers circulating for a recently released model come from three places. Some are genuine independent evaluations, run carefully, with methodology stated. Some are early tests run before the model was finalised or at settings the tester did not record. And some are simply invented by content sites competing for search traffic on a term people are looking up. From the outside these look identical, because all three arrive as a confident number in a table.

The reliable filter is provenance. If a figure does not come with the evaluation used, the reasoning setting applied, the date, and ideally a way to reproduce it, it is not evidence. That is true even when the number is correct, because you have no way to tell.

This matters more than usual for a model priced at $50 per million output tokens. A decision to adopt it across a workload is a substantial commitment, and basing it on a number of unknown origin is a poor trade when running your own comparison takes a few hours.

Why the reasoning setting makes single scores misleading

This is the most important thing to understand about benchmarking this generation of models, and it applies well beyond one vendor.

Astra accepts five reasoning effort levels. Higher settings generate more reasoning tokens before producing an answer, which improves results on difficult problems and increases both latency and cost. A benchmark score produced at max effort and one produced at low effort describe genuinely different things, and neither is wrong.

This means any figure you encounter without an accompanying effort setting is close to uninterpretable. It also means comparisons between models are harder than they look, because the competing model has its own equivalent settings, and matching them fairly is a judgement rather than a specification.

The practical consequence is that cost per solved task matters more than score. A model that reaches the right answer at medium effort is better for production than one needing max, even if the max score is higher. Reasoning tokens are billed at the output rate, which is $50 per million here, so the difference between effort levels shows up directly on the invoice.

There is a second reason to be sceptical of top-line figures across models generally. Benchmarks are public, which means they eventually appear in training data, and a score on a well-known evaluation measures something slightly different a year after publication than it did on release. This is not an accusation against any particular vendor. It is a structural property of evaluating systems trained on the open internet, and it is why practitioners increasingly build private evaluation sets from their own work.

What to know before deciding

Several things about this model shape whether benchmark performance is even the right question.

Long prompts reprice the whole request. Above 272,000 input tokens, input and cache rates double and output rises by half, applied to the entire request rather than the excess. A benchmark run on short prompts tells you nothing about the economics of long-context work.

Caching changes the arithmetic substantially. Cached input runs at a tenth of the standard rate. Any evaluation that ignores caching will overstate the running cost of a system with a stable prompt prefix.

Batch halves the price. Work that tolerates a delay runs at 50 percent of the standard rate, and for most workloads that affects model selection far more than a few benchmark points ever would.

Rate limits differ by account tier, from 500,000 tokens per minute at entry level to 40,000,000 at the top. Throughput in practice may be constrained by your tier rather than by the model.

Modality limits are absolute. Images can be read but not generated. Neither audio nor video is handled at all. No benchmark score compensates for a capability that simply is not present, and this is the kind of constraint that ends an architecture rather than degrading it.

How to evaluate it properly

Since published scores are not available, building your own comparison is the sensible route, and it does not have to be elaborate.

• Collect twenty real tasks from your actual workload rather than synthetic examples. Include the awkward ones you would normally handle manually.
• Define what a correct answer looks like before running anything, ideally as something another person could check.
• Run each task at two reasoning settings, typically medium and high, and record both quality and token usage.
• Compare against a cheaper model on the same tasks. GPT-5.6 Sol is the obvious control at two and a half times less on both input and output.
• Count cost per acceptable answer, not cost per call. A cheaper model that needs two attempts may still lose.
• Test with your real prompt lengths, particularly if you sit near the 272,000 token boundary where the pricing changes.
• Repeat the run. These models are not deterministic, and a single pass on twenty tasks can mislead in either direction.

That exercise takes an afternoon and produces information no published benchmark can give you, because it measures the thing you actually care about rather than a proxy for it.

It also produces something durable. A private evaluation set built from your own tasks stays useful when the next model arrives, and it turns model selection from a question of reading announcements into a question of running a comparison. Teams that maintain one make these decisions in hours; teams that do not tend to make them on vibes and vendor claims.

Decision framework

Five questions before choosing this model on capability grounds.

1. Is your task genuinely hard, or just long? Length is handled equally well by every model in the range, at a fraction of the price. Difficulty is what Astra is actually for.
2. Have you tested the cheaper option? Sol shares the context window and costs substantially less. Skipping that comparison is the most common and most expensive mistake.
3. What reasoning effort does your task actually need? Running everything at max is a straightforward way to multiply your bill without improving results.
4. Can the work go through Batch? Half price changes the calculation for any model choice.
5. Are you measuring the right thing? Cost per correct outcome is the number that matters, and it is rarely the same as the number a benchmark reports.

Knowing how to evaluate a model against your own work, rather than accepting a published figure, is a skill rather than a product feature, and it transfers whenever the models change. If you want a structured route in, explore Coursiv AI lessons and check current plan details on the official site.

Your next step

Take the hardest twenty tasks from your actual work, define what a good answer looks like for each, and run them through Astra at medium effort and through GPT-5.6 Sol. Record quality, token usage and cost.

That comparison answers the question a benchmark table cannot, which is whether the difference in capability is worth two and a half times the price for the specific work you do. In a meaningful share of cases it is not, and finding that out costs an afternoon rather than a quarter of unnecessary spending.

FAQ