The AI writing tics that hurt engagement: A study

The net has sturdy opinions about what “AI-written” content material appears like, and even stronger ones about what’s supposedly unsuitable with it. Scroll any content material marketer’s LinkedIn feed, and also you’ll discover assured claims that em dashes and different AI “tells” sign unhealthy, automated writing.

The issue with these debates is that they usually confuse style with efficiency. What counts as “unhealthy writing” will all the time be subjective. But when the purpose for content material entrepreneurs is to speak clearly and compete within the info market, the sensible query needs to be: which LLM habits really flip readers off?

To search out out, we analyzed a big dataset of content material advertising and marketing pages to determine which AI writing “tics” we see most frequently known as out to grasp that are turning off readers — and those we could also be calling out for no purpose.

How we constructed our ‘AI tics’ examine

At this level, you’ve most likely all seen them, too:

• “In right now’s fast-paced digital panorama…”
• “It’s vital to notice that…”
• “Not solely… but additionally” (repeated over, and time and again…)
• “In conclusion” (even when nothing has been concluded)

The second you discover them, it’s arduous to not see them all over the place an LLM has helped produce copy. Many readers report hating these LLM patterns. However how precisely are they impacting person engagement?

To search out out, we gathered a listing of the most typical AI writing tells we and others have observed. These embody:

• “Not solely… but additionally” constructions: “Not solely does X do Y, nevertheless it additionally does Z.”
• Sentence begins with “then,” “this,” or” that”: “Then it is best to…” “Then the system…” “This reveals…” “This implies…” 
• Introductory filler: “On this article,” “We’ll discover,” and “Let’s have a look”. 
• “Conclusion” starters: “In conclusion,” or different AI equivalents of clearing your throat.
• Em dashes: The most infamous punctuation mark in right now’s content material advertising and marketing.

From there, we constructed a dataset of:

• 10 domains of various website dimension and month-to-month site visitors, in a wide selection of industries together with tech, ecommerce, healthcare, schooling, analytics, and extra
• 1,000+ content material advertising and marketing URLs, constructed from a mixture of workflows together with posts that had been both absolutely human-written, written collaboratively by people and AI, or utterly AI-generated.

Then we standardized our dataset by: 

• Aligning shorter posts and cornerstone content material by standardizing each writing tic as occurrences per 1,000 phrases. Since longer articles naturally comprise extra of, nicely, every thing, a 3,000-word information would in any other case look “worse” than a 600-word put up just because it has extra sentences.
• Excluding any web page beneath 500 phrases. Very brief pages don’t give sufficient room for stylistic patterns to emerge, and their engagement metrics are possible pushed extra by intent than by engagement alone.
• Prioritizing engagement fee as the first efficiency metric. Engagement fee finest captures a reader’s first actual choice: “Do I keep, or do I go away?” GA4 registers an engaged session as any lasting 10 or more seconds. Whereas 10 seconds might sound transient to evaluate whether or not a put up is AI, it’s lengthy sufficient for a person to skim an introduction, discover awkward or repetitive writing patterns, and scan headings to resolve whether or not the content material feels price persevering with.

Dig deeper: A smarter way to approach AI prompting

Why monitoring whole AI tics wasn’t sufficient

Our first intuition was to common the variety of AI tics per 1,000 phrases and evaluate the pages’ efficiency.

At a look, this appeared like a clear technique to separate human writing from AI-influenced writing. However the image shortly acquired difficult by one tic specifically — the notorious em sprint — which dominated the dataset and closely skewed the averages.

The difficulty pointed to a bigger drawback: AI tics are messy by definition. AI is educated on human writing. So if sure patterns present up ceaselessly, that doesn’t imply they’re uniquely “AI.” It could simply imply they’re frequent in English prose. 

To match, we ran the identical tic counter on two recognized controls: a novel I printed in 2021 (which I may assure was written with out ChatGPT, Grammarly, or different AI-assisted instruments). This scored a startlingly above-average 6.9 tics per 1,000 phrases.

Subsequent, we scored “Hamlet,” the well-known Shakespearean play, which scored an excellent larger ≈11.4 tics per 1,000 phrases. Shakespeare, it seems, is extra “AI-coded” than many AI-generated weblog posts.

In the end, we assessed that that is virtually solely because of the em sprint, which is prone to seem in droves in lots of human writers’ prose in addition to AI-produced copy.

With this in thoughts, we analyzed every “inform” individually, nonetheless standardizing per 1,000 phrases. The story grew to become a lot clearer — and way more helpful for writers attempting to resolve what’s really price avoiding.

Dig deeper: How to make your AI-generated content sound more human

The AI tics impacting efficiency

Not all posts are the identical, and many alternative elements influence the success or failure of any web page of content material advertising and marketing. That’s maybe why our information confirmed that the majority AI “tells” didn’t correlate strongly with efficiency or non-performance.

Something smaller than plus/minus .1 correlation is statistically insignificant. Nonetheless, there have been a handful price noting with a bigger influence than others.

‘Not solely’ and ‘not simply’ buildings could also be driving customers away

Phrases constructed round “not solely…” or “not simply…but additionally” stood out with larger-than-average detrimental correlations with engagement fee. Whereas these constructions, when used often, can add emphasis, the info reveals that frequent use is related to excessive person bounce charges. 

AI-assisted writers and editors ought to take be aware, as lots of the AI-generated posts we reviewed tripped over themselves with these constructions. In a single occasion, we discovered a single weblog put up that used “not solely” and “but additionally” 12 separate instances.

The strongest detrimental correlation in all the dataset was noticed in sentences starting with “Conclusion,” usually part headers previous a name to motion. The clearest AI stylistic crimson flag we discovered, posts with headers beginning with “Conclusion” had the biggest detrimental correlation  (≈ -0.118) with put up engagement fee.

Since this tic historically comes on the conclusion of a put up, it’s clear readers might shortly scroll down over the whole lot of those posts earlier than bouncing — or else that posts with these remaining headings are usually lower-quality on common.

Em dashes correlated barely positively

Em dashes had been by far the most typical stylistic tic within the dataset. Additionally they produced one of the shocking outcomes: a slight optimistic correlation with engagement fee.

Regardless of widespread on-line chatter treating em dashes as an “AI artifact,” this information suggests they’re not hurting efficiency, they usually might even align with higher engagement. (As somebody who genuinely likes em dashes — this was deeply validating.)

A believable clarification could also be that writers who use em dashes have a tendency to write down extra explanatory, nuanced sentences relatively than brief, flat declarations. These sorts of sentences usually seem in longer, extra considerate content material that many readers really have interaction with.

That mentioned, this doesn’t imply em dashes trigger engagement. An excessive amount of of factor remains to be an excessive amount of of factor. But it surely does problem the concept em dashes are the bugbear content material entrepreneurs make them out to be. 

Dig deeper: An AI-assisted content process that outperforms human-only copy

3 sensible takeaways for content material groups

Right here’s what content material entrepreneurs can act on right now.

1. Don’t over-optimize for AI detection

Google doesn’t subject SEO rankings like a monotonic punishment rating for “AI model.” Most phrases we checked out didn’t correlate with engagement in any respect.

Don’t rewrite content material simply because somebody declared a phrase “AI writing.” Write for reader usefulness and readability above all.

2. Be aware of the way you wrap up

Express conclusion blocks aren’t unhealthy — however generic, formulaic patterns are possible turning readers away.

Take into account mixing conclusions into evaluation, utilizing subtler transitions, or including new worth with headers, as a substitute of signposting apparent construction. 

3. Use the punctuation that is sensible 

In case your model requires em-dashes? On this dataset, they had been really related to higher reader engagement. Use them.

Don’t miss the forest for pretend plastic timber

AI is probably going right here to remain in content material workflows. However the points with “unhealthy” AI writing aren’t restricted to linguistic tics and punctuation. Whereas all of us have our stylistic opinions, we needs to be cautious about turning stylistic sizzling takes into editorial legislation. 

Write worthwhile writing. Take into consideration readers first. And don’t panic each time somebody on Twitter or LinkedIn decrees that “X phrase = AI.”