Ask people how to spot AI writing and you'll hear the same list: "delve", the rule of three, passive voice, long and wordy sentences. We tested those beliefs against 600,000 texts and 119 measurable linguistic features. Some of the folklore holds up. A surprising amount of it doesn't.
The strongest signals turned out to be structural, not lexical. How a writer varies sentence length, how deep their sentences nest, and how much they lean on verbs and direct objects tell you far more than any list of "AI words". Several features that a model leans on heavily turned out to separate AI from human text barely at all on their own.
Here is what we found, and what it means for anyone who writes, grades, or builds detectors.
How we ran the experiment
We trained a gradient-boosted model (LightGBM) that sees only interpretable linguistic metrics, not the raw text. That restriction was the point. It forces every prediction to be explained by things like sentence length, word choice, and syntax.
The dataset includes more than 600,000 different texts. Each text was scored 0–100 by our model and by our production AI detector. The 119 features fall into a few families:
- Sentence structure: average, maximum and spread of sentence length; how deeply sentences nest
- Vocabulary: word length, syllables, difficult-word share, a curated list of "AI-style" words
- Lexical diversity: how varied the vocabulary is
- Syntax: the share of verbs, nouns, direct objects and conjunctions
- Punctuation and formatting: commas, parentheses, semicolons, whitespace, quotes
- Content: named entities, digits, proper nouns
The two detectors broadly agree on direction but not on every text. Their scores correlate at Spearman r = 0.54, and they give the same AI-or-human verdict at a 50-point threshold 73% of the time.
To find genuine signals, we focused on the texts where both detectors agreed. When two independent systems reach the same verdict, the label is much more trustworthy. On that subset we ranked every feature three ways:
- Correlation: does the AI score go up or down as the metric rises? It runs from −1 to +1. Around ±0.3 or more is a strong signal; under 0.1 is close to noise.
- Effect size: how cleanly the metric separates AI texts from human ones. Near 0 means no real difference.
- SHAP importance: how much the model relies on the metric. That is not the same as how well the metric separates the two groups.
The interesting stories live where these three disagree. A feature can be heavily used by the model yet barely separate the two groups on its own.
One caveat up front: "AI" and "human" here mean texts both detectors agreed on, not texts with confirmed authorship. Treat the findings as strong patterns, not ground truth.
What actually gives AI text away
The clearest signal of human writing is uneven rhythm. The clearest signal of AI writing is a steady diet of verbs and objects.
The human side of the chart is dominated by rhythm and structure. The AI side is dominated by syntax and vocabulary.
Human signal no. 1: sentences that vary wildly
The single strongest human marker is the spread of sentence lengths. The longest sentence in a text and the share of long sentences are close behind. People mix a three-word fragment with a forty-word sentence that wanders through a few clauses. AI keeps a steady, moderate length, sentence after sentence.
Human signal no. 2: deep, sprawling syntax
Humans write longer sentences on average, and those sentences nest more deeply. Clauses sit inside clauses, and asides open in parentheses. Parenthesis use alone is one of the strongest human signals. AI tends toward flatter, list-like constructions that are easy to parse.
Human signal no. 3: concrete, real-world detail
Human texts contain more named entities, digits, cardinal numbers and proper nouns. In other words, people mention specific people, places, dates and figures. AI writing stays abstract unless it is pushed toward specifics.
Human signal no. 4: white space
Human text has more paragraph breaks and blank lines. AI tends to produce dense, evenly sized blocks.
AI signal no. 1: verb-driven, action-framed syntax
The strongest AI markers are syntactic. Verb share, direct objects and progressive "-ing" forms all rise with the AI score. The result is prose built from subject–verb–object units with an ongoing-action framing: "the system is processing", "researchers are exploring".
AI signal no. 2: varied, polished vocabulary
AI spreads its vocabulary across many different words, using each only once or twice. Measures of vocabulary variety all point toward AI. Human writing, especially casual writing, repeats words more freely. AI also reaches for longer, rarer words, with more polysyllabic and "difficult" vocabulary.
The surprises: popular "tells" that barely matter
Several of the best-known signs of AI writing turned out to be weak. Some even pointed the wrong way.
The dashed lines mark the strongest signals from the previous chart. Every popular AI tell falls far short of them, and the two that clear 0.2 point toward human writing.
"AI words" are a weak signal
We built a curated list of AI-flavoured vocabulary, the kind of words that get memed online. The share of those words correlates only weakly with the AI score. The raw count is close to noise. The vocabulary does lean slightly toward AI, but on its own it is a poor detector. A model trained to scan for "delve" would miss most AI text and flag plenty of human writing.
The rule of three hardly registers
The "X, Y, and Z" pattern is often called a hallmark of AI style. In our data its link to the AI score is technically positive and practically negligible.
Passive voice points to humans, not AI
This was the biggest reversal. Passive voice is a human marker here. AI favours active, declarative sentences. The likely reason is that much of the human writing in our data is academic, business or technical. In those genres, phrases like "data was collected" and "results were analysed" are the norm.
AI is not the long-winded one
The stereotype of AI is verbose, sprawling prose. At the sentence level it is the opposite. Humans write the longer, deeper, messier sentences. AI's sentences are moderate in length and remarkably consistent.
Semicolons and first-person voice don't help
Semicolons lean slightly human, the opposite of their reputation as a sign of over-polished machine prose. First-person writing barely moves the score at all. An "I" in the text is not evidence of a human author.
Features the model loves but that barely separate anything
The most technical surprise came from comparing how much the model relies on a feature with how well that feature separates the two groups. Several features rank among the model's most heavily used, yet barely distinguish AI from human text when viewed alone:
The grey dots near the top are the real signals: they clearly separate AI from human text. The green dots at the bottom right are different. The model uses them a lot, but on their own they barely tell AI and human text apart.
So why does the model use them? Most likely they only help in combination with other features. The share of nouns, for example, says little by itself but becomes useful together with sentence length and verb share. Some of them, like uppercase letters and quotes, may just reflect quirks of our dataset. They need a closer look before anyone relies on them.
The takeaway: what a model relies on is not the same as what actually separates AI from human writing.
Where metric-based detection goes wrong, and what that means for you
The texts our metrics-only model got wrong show where this kind of detection struggles. Its mistakes split almost evenly between false alarms and misses.
False alarms: good formal writing looks "AI"
When the metrics-only model flagged human text as AI, those texts had high verb density, polysyllabic vocabulary, high lexical diversity and formal structure. That describes essays, reports, documentation and academic articles. Polished, careful human writing shares many surface traits with AI.
What those texts lacked were the human rhythm markers. They had fewer long sentences, less variation in sentence length and shallower syntax.
Misses: casual AI slips through
When the metrics-only model missed AI text, that text was written simply. It used shorter words, less varied vocabulary and far fewer stock AI phrases (the AI-vocabulary share dropped by 32%). That is conversational chatbot output, paraphrased or "humanised" content, and informal AI-generated articles.
For students and writers
If your formal writing has ever been flagged, this is probably why. Academic style is the style detectors find hardest to tell apart from AI. The features that most reliably mark human writing are about rhythm and specificity, not vocabulary:
- Vary your sentence length, from short and punchy to long and layered
- Use concrete details: names, numbers, dates, places
- Don't strip out asides, parentheticals or the passive voice where it fits
For educators
No single tell is reliable. A text full of "delve" can be human, and a text with none can be AI. A high detector score on formal academic writing deserves a second look, not an automatic verdict.
For detector builders
Check features three ways, not one. A feature that ranks high on SHAP but low on correlation and effect size is either an interaction effect or a dataset artifact, and you need to know which. Make sure your training data covers formal human writing and casual or humanised AI writing. Those are precisely the two blind spots that metric-based models fall into.
The bottom line
AI text is not betrayed by its vocabulary so much as by its evenness. It produces sentences of steady length, flat syntax, abstract content and dense blocks, built around verbs and their objects. Human writing is lumpier. It has fragments next to run-ons, nested clauses, parenthetical asides, real names and numbers, and more white space.
The famous tells mostly underdeliver. "AI words" and the rule of three are weak, and passive voice and long sentences point toward humans. The most useful mindset is to stop hunting for magic words and look at the shape of the writing instead.
