The record
Nothing here is found. It is built. What is sold is a body of tasks, a way to mark the answers, and a promise: the questions stay secret, so no model can have memorised them, and new ones are added on a schedule. A test of this kind is called an evaluation or a benchmark. It usually comes in three parts: a small public sample, a private part that can be licensed, and a private part only the owner ever runs.
A real example. Vals AI's Finance Agent test, in its second version, has 927 expert-reviewed questions about public company filings 1. Of these, 27 are public, 450 form a private set offered for licence, and 450 are the test set, which Vals says will stay private 1. When a lab hands over a new model, Vals puts the test set through a fixed procedure. Three AI judges mark every answer against a weighted list of checks, and the resulting score goes on a public table. The leading model scores 60.60% 1.
Why it might matter for training AI
This is less about training than about measuring. Labs want proof that a model has improved on work it has never seen. Companies want to know which model handles their own invoices, contracts or support tickets before they spend. Governments want measurement from someone independent.
Purchases have been shown here more clearly than for most records. Epoch AI says OpenAI commissioned 300 mathematics questions from it and owns them 2. Vals, by its own account, is paid by labs to test their models, and raised US$40 million at a US$400 million valuation in August 2026 3. LMArena, which ranks models by public vote and sells paid evaluations, raised US$150 million at a US$1.7 billion valuation in January 2026 4.
There is dissent. Tests published by data vendors are widely treated as marketing, and public tests that never change stop being useful once models have seen them. What is bought is trust and freshness, not the questions.
Who holds it and who may share it
Ownership follows the contract.
Commissioned work belongs to the lab that paid. Epoch says OpenAI keeps ownership of the questions it commissioned and that Epoch cannot share them with others without written permission; Epoch holds back 50 questions whose solutions OpenAI does not see 2.
An independent test belongs to its builder, who licenses parts of it or sells runs.
A test built for one company belongs to that company. PrivateEval sells on exactly that promise: the customer owns and runs the machinery, with criteria its AI suppliers never see 5.
The raw material is the hard part. Refreshing a test takes a continuing flow of fresh real cases, which belong to whoever produced them. SourceX licenses business records for use in evaluation only 6.
In Australia, the law firm Allens publishes a benchmark of AI models on Australian law 7. A one-person public project on Australian financial regulation has a five-item starter set 8. The Tax Practitioners Board has issued guidance on AI use but runs no test 9. No Australian professional body was found running or selling one.
Who is already there
Vals says its tests appear in the model documents of the major labs, and it now offers a tool for building coding tests from a customer's own code 3. Epoch raised US$10.3 million in 2025 from donors and collaborators 10. Andon Labs tests models by letting them run a real vending business 11. Data vendors such as Mercor, Scale and Surge issue their own tests alongside their data sales.
Companies that build on the labs' models make tests too. Harvey's legal agent test has a public part and a held-back part, run by Vals, and the leading model scores 25.42% 12.
For a company that wants to run its own tests, software is cheap. Braintrust lists a free plan, one at US$249 a month, and custom pricing 13.
No price for a single benchmark, for a licence to a private part, or for a test built to order was found.
What would have to be stripped
Little, if tasks are written by experts or drawn from public material. If they are built from a company's real records, the same removal of names and client detail applies as for any business record.
Two risks are particular to tests. A leak destroys the product, so access has to be limited; Vals says it restricts the number of runs for that reason 3. And the tester is paid by the party it marks, a conflict of interest that an unknown tester cannot easily carry.
What stands in the way
- Buyers pay for the name on the result. A newcomer has none.
- Well-funded incumbents already cover law, finance, health and coding.
- Australia-only content has few visible buyers; the one Australian legal benchmark is given away.
- A fixed test goes stale within months, and refreshing it needs a secured supply of new cases.
- Companies may not pay much when the software is cheap.