Benchmarking Large Language Models Against Tumour Boards for Oncology Treatment Recommendations

Brief summary (registry, in English)

BEACON (Benchmarking AI for Clinical Oncology decisioNmaking) is a prospective, multicentre, comparative, blinded, non-interventional benchmark comparing the treatment recommendations of six frontier large language models (LLMs) with those of multidisciplinary tumour boards (RCP) in oncology treatment planning. One hundred standardised synthetic cases (20 per localisation, across breast, lung, urological, digestive and gynaecological cancers) are submitted as identical structured input to two independent tumour boards per localisation and to six frontier LLMs. Every recommendation, human or model, is decomposed into five predefined decision domains (intent, surgery, radiotherapy, systemic therapy with its line, work-up and biomarkers). For each domain, concordance between two recommendations is scored 2 when the same options are selected, 1 when at least one option is shared, and 0 when none is shared; the score is computed automatically from the recommendation forms, without human scoring. The primary endpoint is the mean domain score between each LLM and each tumour board, read against the same score computed between the two tumour boards of the same case, so that humans and models are measured by an identical rule and no respondent contributes to its own reference. A co-primary safety endpoint captures the proportion of recommendations carrying serious harm potential, because concordance alone can conceal dangerous errors. Where the two boards select different options, the case-domain is classified EQUIPOISE and, like every discordance, is adjudicated by an independent panel blinded to the source of the recommendation. Tumour boards record their recommendations during the meeting itself, on a dedicated platform (https://etude-beacon.fr) that reveals a case only once it is presented and time-stamps that reveal, so that no board can read a case in advance. BEACON is designed as reusable, openly licensed, pre-registered infrastructure: all synthetic cases, evaluation rubrics, scoring algorithms, verbatim prompts and raw model outputs are released for full reproducibility.

What you commit to

• Study type: Observational
• Planned participants: 100
• From 1 May 2026 to the primary completion date, 31 Dec 2026
• 5 sites in 1 country
• Visits and tests: ask the trial team, the registry does not list them

Built from the registry fields of ClinicalTrials.gov. It describes the study, not your situation.

Who cannot take part

Copied verbatim from ClinicalTrials.gov and grouped by topic. Only the trial team decides who can take part.

Who the study is looking for

Age in the registry: 18 years to - · Sex: All · Does not accept healthy volunteers

Full criteria on ClinicalTrials.gov

Sites (5)

Sites in registry order. Pick your city on the disease page to sort them by distance.