[sic] Detector
Sic reports the text exactly as written.
1What it does
Sic scores every sentence of a scientific text for machine authorship.
Each score comes with its position in the text, so a result can be traced to the exact sentence. Sic reports two thresholds: a strict one that almost never marks human writing, and a sensitive one that catches more machine text at the cost of more false alarms. Every response states how often each threshold is wrong.
Sic provides evidence, not verdicts. It does not identify an author, and a score should never be the sole basis for a decision about a person.
2Compared with commercial detectors
We ran Sic 1 and the two most widely used commercial detectors on the same texts. Sic 1 is equally conservative with human writing, performs alike on text from GPT and Claude models, and is better at locating machine-written passages inside an otherwise human document. The commercial detectors flag more text from open-weight models.
Human texts
How many texts written by people does each detector flag? Lower is better.
Level.
Inside a document
In human documents with machine passages spliced in, how well are the machine's words picked out? AUROC per word.
Sic 1 leads.
GPT and Claude
How many whole texts written by GPT and Claude models does each flag?
Level.
Open-weight models
How many whole texts written by seven open-weight model families does each flag?
Sic 1 trails.
Locating machine-written text is what Sic is built for. Inside mixed documents it recovered 57% of the machine-written words at its sensitive threshold without marking a single human-written word. The commercial detectors recovered about 40%.
3Error rates
A text counts as flagged when any one of its sentences scores above the threshold.
| Texts | Strict threshold | Sensitive threshold |
|---|---|---|
| Written by people | ||
| Research papers | 0.05% 4 / 7,950 | 0.35% 28 / 7,950 |
| Commentary and opinion | 1.0% 6 / 599 | 3.7% 22 / 599 |
| Real-world test set | 0% 0 / 150 | 4.7% 7 / 150 |
| Written or edited by models | ||
| Written by GPT or Claude models | 86% 278 / 325 | 92% 299 / 325 |
| Written by open-weight models | 72% 72 / 100 | 81% 81 / 100 |
| Human text edited by GPT or Claude models | 59% 47 / 79 | 76% 60 / 79 |
All texts are held out from training. Methods are in Research, report 1.
4Limits
- Granularity
- A sentence's score reflects the passage around it. Sic locates machine-written paragraphs; a single machine-written sentence in a human paragraph, or a grammar pass, is usually not detected.
- Scope
- Sic is calibrated for English scientific prose. Scores on other kinds of writing are not meaningful.
- Commentary
- Opinion pieces and editorials are flagged more often than research papers (Table 1).
- Low scores
- A low score does not establish that a person wrote the text.
5Use it
curl https://api.etal-labs.com/v1/detect \
-H "Authorization: Bearer $ETAL_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "sic", "text": "Paste the manuscript text here."}'The response lists each sentence with its score, the two thresholds, and the error rates in Table 1. See the API reference, or try it in the playground.
| Model | API string | Status |
|---|---|---|
| Sic Minor | n/a | forthcoming |
| Sic 1 | sic-2026a, alias sic | Available. Up to 10,000 words per call, about 2 s per 1,000 words. |
| Sic Major | n/a | forthcoming |