Skip to content
Slow Variables
Ask
The data market · dossier 32

Scored recordings of people doing practical computer work

The record

This record has to be made. A person is given a task in real software: reconcile a bank feed, build a financial model, dimension a drawing. Their screen is filmed while a log captures every click and keystroke on the same clock. At the end something decides whether the work is right, either a marking scheme or an automatic check on the finished file. One attempt, with its video, log and score, is called a trajectory. What would be sold is a set of trajectories, ideally with many people attempting the same task, so a buyer can see how human results spread.

An illustration, not a real file: a monthly bookkeeping contest sets a task on a made-up company. Three transactions are coded wrongly and one is entered twice. A contestant works through 40 lines of the bank feed, posts a refund to the wrong account, sees the balance is out by $200, finds the slip, reverses it, and submits after 30 minutes. An automatic check compares the closing balances with the answer key: all four faults fixed, placed 20th of 100. The unit for sale is that half-hour recording with the wrong turn left in, the score, and the 99 other attempts.

Why it might matter for training AI

Some AI models now operate a computer the way a person does, by reading the screen, moving the mouse and typing. They are called computer-use agents, and they learn from demonstrations. Vendors' pages show labs asking for office-software work, for checks that are hard to game, and for records in which someone goes wrong and recovers. A scored contest attempt has all three.

Plain recorded hours have been shown to sell, on the sellers' own word. Markov says on its Y Combinator page that it has sold recordings of experts at work to frontier labs 1. No seller publishes a price. No sale of scored recordings from a contest or an exam was found.

Supply is already running ahead of price. p(doom), a research lab, has released more than 600 hours of its own members' screen recordings free for any use 2.

Who holds it and who may share it

Certification exams come closest to the record. Certiport, a Pearson VUE business, runs exams taken inside the live application for Microsoft Office, Adobe, Autodesk and Intuit software, and says it delivers over three million tests a year 3. Its customers are mostly schools, so many candidates are children. Exam owners guard their questions, and no recording product exists.

Proctoring firms, which watch candidates remotely, hold recordings but do not own them. ProctorU told the US Senate in 2021 that the test providers own the session recordings and that it does not use candidates' data to train its own algorithms 4.

Competitions are small and keep files, not recordings. The Financial Modeling World Cup, which also runs an Excel championship, had 363 participants in 2023 5. Contestants upload a finished spreadsheet; its rules do not say that ordinary rounds are screen-recorded 6. Its terms forbid reproducing any part of its website without written consent, and treat competition case files as confidential 7.

Paid recording programmes own whatever their contributors assign to them. A contributor can license a demonstration only if nothing on the screen belongs to an employer or a client. Ownership is clean only when the recording is made on practice data the organiser controls, by a person who has signed a licence.

No Australian holder was identified; local exams were not examined.

Who is already there

Sellers of human recordings are numerous. Datoric lists a set it describes as 250,000 traces with human-reviewed success and failure labels 8. Chakra Labs pays people for recorded sessions in spreadsheet and design tools through a browser add-on called Pango 9. Paradigm Shift AI, Luel, fiund, Gerra and Realset list similar goods with prices on request 10 11 12 13 14.

Spreadsheet modelling, the obvious first subject, is already tested in public. One benchmark turned 113 of the World Cup's cases into questions and answers 15. BlueFin sets 131 finance spreadsheet tasks, and the best model scores under 50% 16. On FrontierFinance, human experts score 77.2% and the best model 70.9% 17.

What was not found is anyone selling many people's attempts at the same task with an objective score attached.

What would have to be stripped

Names, and anything from a webcam or microphone: faces and voices raise biometric-privacy questions this dossier did not examine in law. Children's attempts are out altogether.

The larger risk is other people's property on the screen. WIRED reported in January 2026 that contractors for OpenAI were asked to upload real work from past jobs, and quoted an intellectual-property lawyer saying the lab was "putting itself at great risk" 18. The Wall Street Journal reported that Mercor faces class actions after a breach exposed workers' facial data and screenshots of their computers 19.

What stands in the way

  • Plain recordings are being given away, and nobody has shown that a score earns a premium.
  • Spreadsheet modelling is already covered by several public tests, and models are close to expert scores on some.
  • Labs can generate traces from their own agents.
  • Contests draw hundreds of people, not thousands.
  • Exam bodies and proctoring firms are closed by ownership and by the secrecy of exam questions.
  • Software companies' terms on recording their screens are unread.