Research
Technical reports for the models we release.
- 1Sic 1: sentence-level detection of machine-written scientific textDetectorA token classifier that scores every sentence, calibrated on human-written papers and compared with two commercial detectors.
- 2QED 1: academic prose from notes, in the author's voiceWriterA fine-tuned writer evaluated against its base model on organisation, fidelity to the notes and style.
- 3How the two families relatePrinciplesFour rules for building a writer and a detector side by side.
[sic] Report 1
Sic 1: sentence-level detection of machine-written scientific text
Summary. Sic 1 assigns every sentence of an English scientific text a score for machine authorship. At its strict threshold it marks a sentence in 0.05% of human-written research papers and detects 86% of texts written by GPT and Claude models. Because it scores sentences and not documents, it can locate a machine-written passage inside a human document. On that task it outperforms two commercial detectors on the same texts, with a word-level AUROC of 0.92 against 0.79 and 0.65.
1.1Model
Sic 1 is a ModernBERT-large encoder with 395 million parameters and a classification head on every token. Each token is classified as human-written, machine-edited or machine-generated. A sentence's score is one minus the mean probability that its tokens are human-written.
Before scoring, citation markers, links and typesetting artefacts are removed, so that the model judges the prose and not its formatting. Texts longer than the 3,072-token context are split at paragraph boundaries and scored in parts.
1.2Training data
The human text is research articles, reviews, clinical-trial reports and commentary published before 2023, when capable writing models were not yet in general use. The machine text was produced from the same papers by GPT, Claude, Gemini, Kimi, Qwen and gpt-oss models: whole passages, rewrites, grammar edits, and passages spliced into human documents.
People prompt models in different ways, and a system prompt can change how the output reads. To cover this, 2,383 of the machine texts were generated under seven prompting set-ups, from a bare request to a LaTeX editing assistant and a coding agent. The corpus contains 66,631 documents in total.
1.3Thresholds
Sic reports two thresholds, both fixed on a calibration split that is separate from all test data. The strict threshold (0.995) is set so that at most one in 500 human calibration texts has any sentence above it. The sensitive threshold (0.954) is the 99th percentile of scores on human-written words. A text counts as flagged when any one of its sentences exceeds the threshold in use.
1.4Results
Table 1 gives the share of texts flagged at each threshold. The research papers and commentary are human-written texts held out from the training sources. The real-world test set holds 554 texts assembled independently of the training data, from papers excluded from training.
| Texts | Strict threshold | Sensitive threshold |
|---|---|---|
| Written by people | ||
| Research papers | 0.05% 4 / 7,950 | 0.35% 28 / 7,950 |
| Commentary and opinion | 1.0% 6 / 599 | 3.7% 22 / 599 |
| Real-world test set | 0% 0 / 150 | 4.7% 7 / 150 |
| Written or edited by models | ||
| Real-world test set, written by GPT or Claude models | 86% 278 / 325 | 92% 299 / 325 |
| Real-world test set, human text edited by a model | 59% 47 / 79 | 76% 60 / 79 |
| Written by open-weight models | 72% 72 / 100 | 81% 81 / 100 |
On the real-world test set, the area under the ROC curve for machine-written against human-written texts is 0.989. The 95% confidence interval for the 86% detection rate is 81% to 89%. Fig. 1 shows the score of every text in the set.
Detection depends on how much of the text a model wrote (Table 2). Text composed by a model is detected about nine times in ten or more. Rewrites of human text are detected about three times in four, and grammar-level edits less than half the time.
| Kind of text | Claude models | GPT models |
|---|---|---|
| Introduction, discussion, methods | 96–100% | 96–100% |
| Abstract | 92% | 87% |
| Summary of results | 88% | 91% |
| Rewrite of a human passage | 76% | 78% |
| Grammar fix of a human passage | 42% | 43% |
1.5Comparison with commercial detectors
We ran Sic 1 and the two most widely used commercial detectors on identical texts in October and November 2026. Each detector is reported at its default decision, and Sic 1 at its strict threshold unless stated. The commercial detectors are identified as A and B.
Human texts
How many texts written by people does each detector flag? Lower is better.
Level.
Inside a document
In human documents with machine passages spliced in, how well are the machine's words picked out? AUROC per word.
Sic 1 leads.
GPT and Claude
How many whole texts written by GPT and Claude models does each flag?
Level.
Open-weight models
How many whole texts written by seven open-weight model families does each flag?
Sic 1 trails.
Locating machine-written text
The first test uses 30 documents of about 1,000 words: 20 human-written documents with machine-written passages inserted at known positions, and 10 left untouched. The inserted passages come from the model families represented in Sic's training data.
Sic 1 ranks machine-written words above human-written ones more reliably than either commercial detector (Table 3). At its sensitive threshold it recovers 57% of the machine-written words without flagging a human-written word. Its advantage lies in passages of a paragraph or more. Single inserted sentences are not detected.
| Measure | Sic 1, strict | Sic 1, sensitive | Detector A | Detector B |
|---|---|---|---|---|
| AUROC per word | 0.92 | 0.79 | 0.65 | |
| Machine-written words flagged | 38% | 57% | 39% | 42% |
| Human-written words flagged | 0% | 0% | 0.8% | 0% |
| Untouched documents flagged | 0 / 10 | 0 / 10 | 0 / 10 | 0 / 10 |
| By kind of inserted block | ||||
| Machine-written passage, several paragraphs n = 6 | 71% | 83% | 33% | 40% |
| Machine-written paragraph, about 200 words n = 13 | 41% | 68% | 55% | 72% |
| Machine-written run of sentences, about 75 words n = 14 | 0% | 11% | 13% | 6% |
| Single machine-written sentence n = 11 | 0% | 0% | 27% | 33% |
| Light machine edits n = 15 | 0% | 0% | 0% | 0% |
Whole texts
The second test asks the plainer question of whether a whole text is flagged (Table 4). On 37 texts written by GPT and Claude models, the three detectors perform alike. On 100 texts composed or rewritten by seven open-weight model families, detector a flags 92 and Sic 1 flags 72. The two rank these texts almost identically, with AUROCs of 0.998 and 1.000 for composed text, so the gap reflects Sic's more conservative threshold. No detector flagged a human-written text in either set.
| Texts | n | Sic 1 | Detector A | Detector B |
|---|---|---|---|---|
| GPT and Claude models | ||||
| Human-written | 18 | 0 | 0 | 0 |
| Written by GPT models | 16 | 16 | 16 | 15 |
| Written by Claude models, current | 7 | 5 | 7 | 7 |
| Written by Claude models, earlier | 14 | 8 | 5 | 6 |
| Human text edited by a model | 9 | 6 | 4 | 4 |
| Open-weight models | ||||
| Human-written | 100 | 0 | 0 | 0 / 20 |
| Composed or rewritten by a model | 100 | 72 | 92 | 19 / 20 |
| Copy-edited by, or mixed with, a model | 100 | 3 | 10 | 1 / 20 |
1.6Limitations
- Scope
- Sic 1 is calibrated for English scientific prose. Its thresholds do not transfer to other genres or languages.
- Light edits
- Grammar-level edits and single inserted sentences are usually not detected.
- Commentary
- Opinion and editorial writing is flagged more often than research papers: 1.0% against 0.05% at the strict threshold.
- Reference period
- The human text predates 2023. Error rates on later human writing may differ.
Report 2
QED 1: academic prose from notes, in the author's voice
Summary. QED 1 writes academic prose from an author's notes, in the voice of example passages the author supplies. Compared with the base model it was trained from, it follows the author's own ordering of ideas more closely, reuses far less wording from the notes, and preserves mathematical notation and citations almost twice as often.
2.1Model
QED 1 is a rank-32 LoRA adapter on Qwen3.8-27B, an open model with 27 billion parameters. Training has two stages: continued pre-training on the source texts, then supervised fine-tuning on 2,484 writing tasks.
The model performs three tasks. It composes a passage, a section or a whole document from notes; it polishes a draft with light edits; and it revoices a draft, rewriting it in full while keeping its content. Every output is checked for repetition and stray markup before it is returned, and regenerated if it fails.
2.2Training data
Each training example pairs a human-written text with notes derived from it, so that the model learns to write the original from its outline. The notes are given at several levels of detail and in shuffled order, and some examples carry mathematics and citations that must be reproduced exactly.
The texts are posts published before 2023 on four science blogs released under a Creative Commons Attribution licence: Absolutely Maybe by Hilda Bastian, Reciprocal Space by Stephen Curry, Triton Station by Stacy McGaugh and quantixed by Stephen Royle. A further 693 examples use passages by 123 authors of published papers. Each of these is presented with two or three other passages by the same author, which teaches the model to take its voice from examples.
2.3Evaluation
QED 1 is compared with its base model under identical prompts on 32 documents held out from training, and on 40 passages by four authors whose writing the model has never seen. The measures and their thresholds were fixed before the test set was used.
| Measure | Base | QED 1 | Difference | Better is |
|---|---|---|---|---|
| Organisation Rank correlation with the author's order of ideas, from shuffled notes | 0.36 | 0.53 | +0.17 +0.11 to +0.23 | higher |
| Copied wording Share of wording taken from the notes, in excess of human authors | 0.39 | 0.06 | −0.33 | lower |
| Markup preserved Mathematics, citations and emphasis kept verbatim in a rewrite | 48% | 93% | +45 points | higher |
| Stray formatting Documents with unrequested headings, lists or rules | 23% | 2% | −20 points | lower |
| Style, unseen authors Similarity to the author's style, given three example passages | 1.38 | 1.38 | −0.01 −0.20 to +0.18 | higher |
| Claim changes Claims added, dropped or altered per document, by an automated judge | 7.0 | 3.8 | −3.2 −8.9 to +1.8 | lower |
| Length Median length relative to the author's text | 0.93 | 0.95 | +0.02 | near 1 |
| Coherence Rated against the human original by two judge models | 0.67, 0.64 | 0.55, 0.54 | −0.12, −0.10 | higher |
2.4Results
The clearest gains are in how QED 1 uses the notes. Given points in shuffled order, it arranges them much as the original author did, and it rewords them where the base model tends to copy. It keeps mathematics and citations intact in 93% of cases, against 48% for the base model, and adds unrequested formatting in 2% of documents, against 23%.
QED 1 changes fewer claims than the base model, 3.8 per document against 7.0; with a test set of this size the difference is not statistically significant. On authors it has never seen, it matches style from three example passages as closely as the base model does. The automated judges rate the base model's text as more coherent; QED 1 scores 0.55 and 0.54 against the human original, above the acceptance level of 0.40 set in advance.
2.5Limitations
- Fidelity
- QED 1 writes only from its input, but it can still omit a detail or add an aside. Outputs should be checked against the notes; the
comparecall lists the differences. - Voice
- On authors outside its training data, QED 1 follows example passages no more closely than its base model.
- Editing
- Polish makes light edits. Neither polish nor revoice reliably corrects grammar or spelling: on a draft with three planted errors, repeated runs left between none and all of them.
- Long drafts
- The editing tasks were trained on passages of up to about 400 words. On longer drafts, edits become lighter towards the end.
- Register
- Most of the training text is science writing for a general scientific readership. Specialist sections may come out plainer than an expert would write them.
Principles
How the two families relate
QED and Sic are developed side by side. Four rules govern how they relate.
- The writer is not trained against a detector
- No detector score enters QED's training, model selection or evaluation. QED is trained to write as a given human author writes, and it is measured against that author's text.
- No evasion
- Neither model offers a way to make text score lower on a detector, and the tools instruct agents not to attempt it.
- Assistance is disclosed
- Text written with QED is AI-assisted. The API drafts the disclosure statement in the forms journals ask for.
- Evidence, not verdicts
- Sic reports sentence scores with their error rates. It does not make a finding about a person.