Is it really an AI tell? Count it first.

People judge writing by the word delve, the em dash, a text with no contractions, lists of three. This page counts 25 of those markers in three kinds of writing. In research abstracts and Reddit posts, the same documents were written by a person and again from the same prompt by four language models. In school essays, 5,867 US students answered seven assignments, and the same assignments were given here to two models, Claude and Llama 3. The page shows the sentences behind every word and phrase count. Several markers change sides between the kinds of writing, and in the school essays the same marker can point to one model and away from the other. It is a measurement, not a detector: it gives no score and cannot tell you who wrote anything.

Published 2026-09-16 · updated 2026-09-18 · 25 markers · 3 kinds of writing: 1,324 research abstracts and 1,338 Reddit posts by the person, against four RAID models and 44 abstracts by Claude · 5,867 school essays by students, against two models run here · re-measured weekly

The problem

Ask how to spot machine-written text and the answers barely change: the word delve, the em dash, no contractions, everything in threes, moreover and furthermore. Readers are documented to judge by some of these, contractions and first person among them (Jakesch et al., PNAS 2023). Teachers, editors, reviewers and moderators use the same markers to decide which texts to doubt.

Almost none of them had been counted where it would mean something: in the same document, written once by a person and once by a model, and in more than one kind of writing. A word list compiled from chatbot output says what the chatbot writes, not whether people write it too, and a count from one kind of writing says little about another. The detector software built for the same job has its own record of calling human writing machine-made; essays by non-native English writers are the documented case (Liang et al., Patterns 2023).

This page does not add another list. It counts the ones that exist.

How it works

The RAID benchmark gave language models the title of a real document and published both the person's text and each model's. This page takes two of RAID's kinds of writing: research abstracts from arXiv, and posts from Reddit, posted in 2021. Every writer covers the same documents, so a difference between columns is about the writer and not the topic. The third kind, school essays, has no RAID counterpart and is built differently: US students' essays to seven assignments, set against essays two models wrote here to the same assignments (below). Each kind of writing is measured on its own, with its own placebo and its own correction. Each marker is a regular expression, or for sentence length a variation threshold, defined in src/markers.ts, and the page runs the same file on anything you paste.

Llama chat loses the most. 191 of its abstracts and 271 of its posts stop mid-sentence at its length limit, and 120 abstracts and 189 posts are not the text asked for. Mistral chat loses 289 abstracts and 95 posts that way. “Pairs” counts the document pairs with the person inside one length band. The school essays have their own writers, counted in their own section.

Each row gets one verdict in each kind of writing. In abstracts and posts it comes from GPT-4 against the person; in school essays, from Claude, told it is a student, against the students. For a word or phrase it comes from the rate test at equal lengths; for a property of a whole text, from the shares on the same documents, or in school essays on the same assignments. GPT-4 marker when GPT-4 is higher (Claude marker when Claude is, in school essays), more human when the person is, careful writing when the two cannot be told apart but both sit above casual writing, not recorded when no text of that kind can show the marker, and no signal otherwise. Next to the verdict, the page counts how many of RAID's four models separate the marker from the person, each decided by the same rules; in school essays, how many of the two models run there, Claude and Llama 3. A model is counted only where there was enough to compare: at least five uses of a word between it and the person, or, for a property of a whole text, thirty pairs and five texts on the rarer side. That count describes; it is not a further test. A verdict describes a marker across hundreds of texts. It says nothing about any single one.

Corrected on 16 and 17 September

On 16 September: the first numbers this project published, in its README and in a thread on X, were partly wrong, because three patterns counted the wrong thing: two measured how the source was typed rather than who wrote it, and one counted possessives as contractions. No first person ignored case, so the “(i)” of a numbered list and the “i” of “i.e.” counted as “I”, and first person came out as a GPT-4 marker. The dash counted only the “—” character, and RAID's human abstracts are plain ASCII from TeX, where people type “---”, a spaced “--” or a spaced hyphen. The published line that the people used no em dash was true only of that character: counted however it is typed, the dash is something the people write and the four RAID models almost never do. No contractions counted every possessive “'s”, which is where the published 89% of human abstracts against 85% of GPT-4's came from; contractions are close to absent in all of them.

The pairing was wrong too. It took texts in file order, RAID's file is ordered by topic, and so the two sides of a comparison were different papers; “crucial” read as careful writing because of that. Two changes were made to the texts themselves. RAID's human abstracts come with line breaks every 79 characters, and those breaks hid phrases from their patterns on the human side only; they are now joined. Quotes and code are now removed from the casual and careful writing before counting. Several other patterns were widened or narrowed in the same pass, and the verdicts now come from the tests described above. The findings about “delve”, “moreover” and “leverage” held. The README lists each change and why it was made.

On 17 September, when the Reddit posts were added, the abstracts went through the checks described above, and the published abstracts numbers moved; no GPT-4 verdict did. The 172 dated abstracts are out of every column, four documents of the earlier 1,500 are no longer in the set, and Llama chat and Mistral chat lost the most texts to the other checks. GPT-4's “delve” went from 1.26 (207) to 0.97 (141) per thousand words, and from 20.2% to 19.4% of its abstracts on the same documents. Its “leverage” went from 2.14 to 2.34, and its “crucial” from 0.74 to 0.79, now in 9.0% of its abstracts against the person's 2.3% on the same documents (it was 8.3% against 2.0%). The person's dashes went from 85 to 75, and the share of the person's abstracts with first person from 0.7% (10 of 1,500) to 0.5% (6 of 1,324). GPT-4's “we” went from 1.77 to 1.59 per thousand. Llama chat's “in conclusion” went from 0.32 (132) to 0.30 (81) and the person's from 4 uses to 3, and the other models now use it once at most, not twice. Mistral chat's “crucial” went from 0.90 to 1.10, still the highest of the four models. The model texts are now picked by an explicit rule, greedy decoding with no repetition penalty, instead of by their order in RAID's file. That rule came in the same run as the checks, and the published data does not separate their effects. One earlier claim was also stronger than the data: the first version called “delve” a GPT-4 word and not an AI word, and Llama chat's own comparison with the person, new in this release, separates it too (q = 0.027), at a sixteenth of GPT-4's rate. The Claude arm lost two documents as dated, one of which it had already lost as remembered, so it is 44 texts instead of 45, and its comparison table covers the 23 documents every writer kept instead of 45.

Every number on this page is from the run of 18 September 2026. That run added the school essays. Each kind of writing is measured and corrected on its own, so adding one leaves the others alone: the abstracts and posts numbers are the same as in the run of 17 September.

What the counts showed

Rates are occurrences per thousand words on the full columns, with the occurrence count in brackets. Each verdict is the one described above: for words and phrases it comes from the comparison at equal lengths, not from the whole-column rates shown here. Every word and phrase count below opens on the page to the matches behind it; the whole-text shares (no contractions, first person, sentence length) show how many texts they rest on.

The same markers in three kinds of writing

The page opens with a grid: for every marker and each kind of writing, a verdict and how many of the models separate the marker from the people. In abstracts and posts the verdict is GPT-4's and the count runs over RAID's four models. In school essays the verdict is Claude's, told it is a student, against the students, and the count runs over the two models run there, Claude and Llama 3. These are the rows that matter most here. “Above” means a model shows the marker more than the person does, “below” less, and “of” counts the models with enough uses or texts to compare.

Across all 25 rows, GPT-4's verdict in the abstracts is GPT-4 marker for 5, more human for 2, careful writing for 6 and no signal for 12. In the posts it is GPT-4 marker for 10, more human for 1 and no signal for 12, and 2 markers are not recorded. GPT-4's verdict differs between those two kinds on 11 of the 23 markers recorded in both. In the school essays, Claude's verdict is Claude marker for 2, more human for 8 and no signal for 15. No placebo split, in any of the three kinds, disagrees on any row.

Between abstracts and posts, several markers change sides. “Moreover” is the people's word in abstracts and GPT-3.5's and GPT-4's in posts, where Llama chat and Mistral chat never write it. “Leverage”, which with “crucial” separates all four models in abstracts, is almost absent from posts. Of the essay scaffolding, “in conclusion” and “it is important to note” give no signal in abstracts, and “not only … but also” is a careful-writing marker there, used more by the person than by GPT-3.5 and Mistral chat; in posts all three separate three of the four models. The dash that points to the people in abstracts gives no signal for GPT-4 in posts. Personal detail and informal spelling, two of the markers readers are documented to judge by, say nothing in abstracts, where nobody writes them, and separate all four models in posts, where people do. What holds in both is mostly shape: three-item lists and sentences of even length. “Crucial” is the one word that runs above the person for most models in both. “Delve” separates in both too, but in posts every writer rarely uses it.

School essays show the same thing inside a single kind of writing. On six markers, one of its two models writes the marker more often than the students and the other less often: “in conclusion”, “not only … but also”, “furthermore”, “crucial”, even sentence lengths and three-item lists. On all six, Llama 3 is above the students and Claude below. The one word or phrase that points to Claude there, “it's not X, it's Y”, gives no signal in abstracts or posts.

In research abstracts

In Reddit posts

RAID's Reddit documents are posts from 2021, each written by a person and again by each model from the post's title. The people's posts are counted and never quoted, here or on the page; the sentences below are from RAID's model texts. GPT-4 writes posts longer than the person (a median of 218 words against 167), and people write posts with “I”, contractions and slang, which makes them a different test from abstracts.

A Claude arm, generated here

RAID has no Claude arm, so one was generated for this project with Claude Opus 5, reached through Claude Code. That is the same system that wrote this project's code, and the arm is marked on the page for that reason. RAID's own prompts for 50 of the papers were given verbatim, one call per document, with one added instruction because the pipeline needs a bare string back: “Return only the abstract text itself.” No style instruction, no examples. All 50 returned texts are published with their prompts in claude-abstracts.json. Five of them reproduce the published abstract: more than half of their five-word sequences come from it. Two are about papers left out as dated, like every writer's texts for those papers, and one of the two is among the five. That leaves 44. The overlap check removed none of the four RAID models' texts. There is no Claude arm for Reddit posts.

Claude's documents are a subset, mostly about image segmentation (36 of the 44 have “segment” in the title), so claude-matched.json counts every RAID writer on the same documents: the 23 of Claude's 44 that every RAID model also kept after the checks. Llama chat lost 17 of the 44, most of them cut off. Per thousand words, occurrences in brackets:

On these documents Claude writes “leverage” once and “crucial” not at all, against the person's 2 and 5, and one “furthermore” where the person writes none; no writer uses “delve”. It writes “moreover” 4 times, which none of the four RAID models does here. On this small set the RAID models use “crucial” about as often as the person (0.82 to 1.35 against 1.12), so the word says little here. Claude's three-item lists, 6.06 (36, in 18 of the 23 texts), run well above the person's 1.78 and at or above GPT-4's 5.11 and Llama chat's 5.55. Across all 44 of its abstracts Claude wrote 8 dashes, in 4 texts, six of them the “—” character, as in: “Learning from scribbles — sparse, quickly drawn strokes marking a few pixels of each class — is an attractive alternative …”. Only 2 of the 8, both “--”, fall in the 23 shared documents. Counts this small move a lot, so read the brackets.

In school essays

The third kind of writing is built differently, and the difference matters for how to read it. There is no shared document. US school students answered seven assignments, and the same seven assignments were given here to two models. A model's essay is compared with a student's essay on the same assignment and of about the same length, not with a rewrite of the same text, so these pairs are looser than the ones in abstracts and posts.

The students. 5,867 essays from the PERSUADE 2.0 corpus, out of its 25,996: the ones written to the seven assignments that need no reading passage, mostly in grade 8 (4,101) and grade 11 (1,342). Three of the seven ask for a letter to the principal. Every essay used here was in the Kaggle Feedback Prize release of December 2021, a year before ChatGPT, and that was checked essay by essay against the official file (kaggle-2021.json). The corpus is CC BY-NC-SA 4.0. The students' essays are counted and never quoted: no essay, sentence or name of theirs is in the repository or on the page.

Why the machine side was written here. Three public sets pair student essays with model essays, and none could be used. OUTFOX sampled its models at temperature 1.3, and most of its essays turn into word salad before they end. Ghostbuster's human essays were published in December 2022 and January 2023, after ChatGPT, so its human column cannot be shown to be human. ArguGPT does not release its human essays. So each assignment was given, with nothing else, to three writers:

• Claude*: Claude Opus 5 through Claude Code, told it is a student in the assignment's grade. 200 essays. This is the column the verdict is read from.
• Llama 3*: Llama 3 8B, run on a local machine through Ollama at temperature 1, told the same. 200 essays. It is the one writer in this kind that is not the assistant that built the project.
• Claude plain*: the same Claude, given the assignment alone. 200 essays, shown beside the others and kept out of every test, so you can see what the student framing moves.

The prompt says nothing about length, style or format, and no student's text is in it. Each essay was written in its own session and is committed with the exact prompt it was given, the model, the date and a hash (claude-essays, llama3-essays). The essays cannot be regenerated word for word: Ollama returned different essays for the same prompt and seed, and Claude Code does not expose its sampling settings. What can be repeated is the measurement, from the essays as committed. The same assistant that wrote these essays also wrote this page and the code that counts them, which is why their columns carry an asterisk. This is the part of the project to be most skeptical of.

No essay was dropped from any column, and none comes near the line for remembered text. Rates are per thousand words over all of a writer's essays, occurrences in brackets; shares are of essays.

What it does not do

• It is not a detector. It cannot tell you who wrote a text, and a text full of these markers proves nothing. The paste box shows which markers a text contains and how often each writer uses them in the kind of writing you pick. There is no score, because the tables do not support one.
• Three kinds of writing, two matched by document. The documents every writer shares are research abstracts and Reddit posts. School essays are matched by assignment, and their machine side was written here. Each kind is measured on its own. The casual and careful writing shows how much of a marker is about the kind of text, but it is not matched by document and each arm comes from a single source. Email, chat, product reviews, and social platforms other than Reddit, such as X, Facebook and LinkedIn, are not covered: this project has not found a set that pairs a person's text there with models writing the same thing and whose terms allow this use. University writing is not covered either: the school essays are US grade 8 to 12 work on seven assignments. The Reddit file RAID used was not filtered for bots or spam.
• Six models, older versions, and columns generated here. Four come from a published benchmark, in the versions RAID used in 2023 (gpt-3.5-turbo-0613, gpt-4-0613, Llama 2 70B chat, Mistral 7B Instruct v0.1); newer versions may write differently. The Claude arm of the abstracts is a small fraction of the others' size and carries the caveats above. In school essays every machine column was written here, by Claude and by Llama 3 8B run locally, and the essays cannot be regenerated word for word. The students wrote before 2022 and the models in 2026, and that kind of writing cannot separate the writer from the year. There is no Gemini arm.
• Markers are patterns, and patterns miss. “Delve” catches the word and not the idea. Irony is invisible to all of it, and quotation only partly visible: contractions, first person and personal detail skip what is in double quotes; the other markers count it. Patterns also catch the wrong thing. Three of them did until 16 September, as described above, and each came to light when the published numbers were checked against the texts. The three-item-list pattern still leans toward writers who use the serial comma. In Reddit posts, two results rest on how people type there: the dash verdict, and Mistral chat's separation on contractions.
• One verdict per kind, and a count that is not a test. The verdict compares one model with the person and nobody else: GPT-4 in abstracts and posts, Claude told it is a student in school essays. The count of models next to it applies the same rules to each model measured in that kind, but it is not corrected across them, and a model with too little to compare is left out of it. Llama chat's “in summary” does not separate from the person even in Llama chat's own comparison, because few of the person's abstracts are as long as Llama chat's. The rates in the tables are whole-column rates and lean on each writer's text lengths; the verdict compares equal lengths, and a word that only short texts use has little to be compared with. Some verdicts sit close to the 0.05 line: GPT-4's dash in abstracts (q = 0.040) and its “delve” in posts (q = 0.038).
• English only. A document is left out of every column when any of its writers wrote it in another language, which removed 37 Reddit posts.

A count looks wrong? Open an issue with the marker and the sentence. The evidence files are there so a pattern can be argued with.

How it was built

TypeScript throughout. RAID is 2.3 GB across ten parquet shards, and none of it is downloaded whole: the row-group statistics say which groups can hold the wanted rows, and only those are fetched over HTTP range requests, paced so the host does not have to refuse. RAID's copy on Hugging Face has no Reddit posts, so those come from RAID's own CSV file, read over two range requests, and the people's posts are never committed. The students' essays come from PERSUADE 2.0 and are never committed either; the essays the models wrote here are. The abstracts were dated once against arXiv's own records. Every Monday a GitHub Actions workflow rebuilds the corpora from their public sources, reruns the checks, re-measures every marker in each kind of writing and commits data/summary.json and one folder per kind of writing under data/genres/. On every push, CI typechecks, runs the 254 tests and rebuilds the page. The page is static, built with esbuild and served from GitHub Pages.

The first version was built in one day, the Reddit posts were added the next and the school essays the day after, with Claude Opus 5 writing the code through Claude Code and a human deciding what to measure, what to quote and what not to claim. It follows Will it run on a Mac? and Why didn't my job run?, which apply the same rule to shell flags and to GitHub Actions: publish what was measured, with the evidence next to it.

• Page: barbarkaragul-oss.github.io/is-it-really-an-ai-tell
• Repository, method and data: github.com/barbarkaragul-oss/is-it-really-an-ai-tell (MIT; the corpora keep their own licences)