Skip to content
Slow Variables
Ask
The data market · dossier 14

Practical exams marked against a fixed sheet

The record

Three records share one form: a person performs a set task against the clock, and an assessor fills in a marking sheet. They do not share a verdict.

Clinical skills stations. A medical candidate has a few minutes with an actor in the role of a patient, and an examiner ticks off a checklist as they go. The record is the recording, the station brief and the examiner's sheet.

Trade practical tests. An apprentice wires up a board or makes a weld, and an assessor from a training organisation ticks a checklist tied to a national standard.

Trade-skill competitions. At WorldSkills events a competitor builds a set project over many hours. Experts mark it part by part. Some parts are marked by measurement, where a gauge reading earns the mark or does not. Others are marked by judgement, where several experts score.

A worked example from the Australian Medical Council, which examines doctors trained overseas. Sitting the clinical exam costs A$3,000 at the test centre or A$3,400 for the online version 1. A candidate moves through 16 assessed stations and four rest stations, each of 10 minutes; 14 count towards the result 1. At a station the candidate reads the case for 2 minutes, then spends 8 with the actor 1. A pass needs 9 of the 14 1. Of the 1,978 candidates in 2018–19, 537 passed, a rate of 27.1% 2.

Why it might matter for training AI

Vendors' pages show labs asking for expert judgement recorded against a rubric, for spoken exchanges with a purpose, and for first-person video of skilled trades. A marked practical exam joins a performance to a grade.

Vendors advertise hourly pay for ungraded video of trade work. For practical exams marked against a rubric, of any kind, no buyer was found.

Who holds it and who may share it

The official clinical exam. The Australian Medical Council runs its exam at its own test centre, which its 2014 annual report says was fitted with recording equipment 3. The Council owns the material and warns of discipline for breaching the security of exam content 1. Medical schools hold their own students' archives.

Paid practice exams. Small Australian tutors, often run by doctors who passed, sell mock exams. AMC Clinical Guide lists a mock exam at A$150 and says it can be recorded 4. OSCE Revisions already sells AI patients and AI grading across more than 1,800 stations, by its own account 5. A tutor's recordings are its own, subject to what each candidate agreed to.

Trade tests. Training organisations hold the evidence, which is mostly paper checklists, photographs and supervisors' reports. A Western Australian government fact sheet mentions video only as a fallback way to confirm a supervisor's report 6.

Competitions. WorldSkills Australia says each two-year cycle has over 700 regional competitions in 34 regions and more than 6,000 entrants 7. The international body's competition rules of January 2026 say finished projects belong to the organiser and to WorldSkills International together, and may not be used without both parties' permission (rule 10.15); filming at a workstation needs the approval of the manager for that skill (rule 13.3) 8. Whether WorldSkills Australia has rules of its own on filming was not established.

Who is already there

  • A Texas medical school put AI grading into routine clinical exams in 2025, published the work in October 2026, and its university has filed patent applications 9.
  • A 2026 preprint compared an AI grader with examiners on 240 transcripts and 8,405 checklist items 10. Where the two disagreed, blinded experts' majority view matched the AI 65.5% of the time and the examiner 27.9% 10. Examiners had been lenient.
  • Tradeye gives away narrated first-person video of plumbing, electrical and heating work, more than 1,020 clips by its count 11. It is ungraded.
  • East China Normal University showed, at a WorldSkills conference in 2026, a system that films students' work from their own viewpoint and grades it against expert rubrics 12.

Clinical recordings and ungraded trade video are both well supplied. Nobody outside China was found selling measured competition work filmed from the competitor's viewpoint.

What would have to be stripped

Every clinical recording shows the candidate, the actor and the examiner by face and voice, so each person's consent matters. One tutor advertises teaching from recent real exam cases 13, and anything built on recalled cases would meet the Council's claim over its content. School students can enter regional trade competitions 7, so some competitors are minors.

What stands in the way

  • Nothing shows that a lab pays more when footage carries a grade than when it does not.
  • For clinical exams, the examiner's sheet is a weak label, schools supply themselves, and AI tutors already exist.
  • Competition footage needs two owners' permission, and a Chinese university has already shown its system at the organiser's own conference.