Methodology

How we measure the Dead — AI-generated content and bot activity across the web.

Overview

The Dead Internet Monitor tracks two distinct phenomena: the creation of AI-generated content (“AI Slop”) and the consumption of content by automated accounts (“AI Slurp”). We sample content from major platforms, classify it using large language models, and analyse author behaviour for bot-like patterns.

Our goal is not perfect accuracy — which remains elusive even for specialised detectors — but consistent, transparent measurement of trends over time.

+-----------------------+
|      COLLECTION       |  9 sources, staggered
+-----------+-----------+
            |
      +-----+-----+
      v           v
+----------+ +----------+
| CLASSIFY | |BOT DETECT|  parallel
|  (LLM)   | |(7-signal)|
+----+-----+ +----+-----+
     |            |
     +------+-----+
            v
+-----------------------+
|      AGGREGATION      |  Autopsy Matrix + DII
+-----------+-----------+
            v
+-----------------------+
|       DASHBOARD       |  deadinternetmonitor.com
+-----------------------+

Data Collection

Content is collected from seven sources on staggered schedules. The headline metric — the Dead quadrant percentage — is volume-weighted: each classified item counts once regardless of source. Collection volumes are not calibrated to real-world traffic share; a future improvement would weight sources by their actual share of internet traffic.

Reddit uses public RSS feeds (comments only, no API credentials required). The Guardian collects reader comments from articles across five editorial sections (Opinion, Politics, Technology, Science, World) via the public Discussion API. Steam collects user reviews for the top 50 games by popularity via the public App Reviews API — filtered to English, most-recent first, with review-bomb detection retained in the sample. Mastodon and Lobsters serve as control groups — decentralised or invite-only platforms with lower bot incentive.

Classification

Each item is classified by a large language model using a structured prompt (v3.0) that applies Bayesian calibration with platform-specific base rates derived from Ahrefs and Originality.ai research. This counters the documented tendency of LLM classifiers to default to “human” ( RAID 2024 found 10–15% false negative rates).

Models

Fallback triggers when primary confidence is below 0.5.

AI Indicators

The classifier looks for research-validated signals of AI generation:

• Overly balanced, hedging language (“It's worth noting”, “to be fair”)
• Formulaic structure and template phrases (“Let's dive in”, “In the realm of”)
• Comprehensive but shallow coverage of topics
• Synthetic empathy (“Great question!” without substance)
• Absence of personality, humour, or strong opinions
• Safety disclaimers on simple topics

Human Indicators

• Personal anecdotes and specific lived experiences
• Typos, slang, mid-thought corrections
• Strong opinions without both-sides framing
• Niche expertise with personal perspective
• Genuine emotional expression — frustration, sarcasm, humour
• Natural digressions and terse replies

Output

Each classification returns: a label (ai_generated, human_created, or uncertain), a confidence score (0.0–1.0), specific indicators observed, and a brief reasoning explanation.

Post-Processing

After the LLM returns its classification, a post-processing step corrects for the documented human-default bias. Items classified as “human” but carrying multiple AI indicators are reclassified as uncertain or AI-generated. Short content (<100 characters) is capped at 0.60 confidence.

+--------------+
| Content Item |
+------+-------+
       v
+--------------+   confidence
|Primary Model |---- >= 0.5 --> RESULT
| Gemini Flash |
+------+-------+
       | < 0.5
       v
+--------------+
|Fallback Model|---- >= 0.5 --> RESULT
| Claude Haiku |
+------+-------+
       | < 0.5
       v
  "uncertain"
       |
       v
+--------------+
|Post-Process  |  correct false negatives
| Recalibrate  |  using signal evidence
+------+-------+
       v
   FINAL LABEL
ai / human / uncertain

Classification coverage

Not every collected item can be classified. Content with fewer than 50 characters of body text — one-line replies, single-emoji posts, bare links, “this.” — carries too little signal for the classifier to separate human from machine, so it is markedskipped and excluded from the AI-rate denominator. (Items between 50 and 100 characters are classified but capped at 0.60 confidence, as noted above.) Skipped items are still collected and counted in volume totals; they are simply not scored. The headline AI rate is therefore a rate over classifiable content, not over everything collected.

Coverage — the share of collected items that carry enough text to classify — varies substantially by source. Short-form and review platforms skip the most; long-form and Q&A platforms skip the least. We publish the breakdown so the denominator behind each source’s rate is explicit. The table below is computed live from the trailing 30 days of collection counts and updates with each night’s data. Both total_collected and total_classified are published per source per day in the open aggregate export, so every coverage figure here is reproducible from the licensed data: coverage = total_classified / total_collected.

Bot Detection

Separate from content classification, we analyse author behaviour using a 7-signal weighted scoring system grounded in peer-reviewed research. Authors with 2+ collected items receive a bot score.

Scores above 0.7 are flagged as likely bots. Between 0.4–0.7 is suspicious. Below 0.4 is likely human.

The Autopsy

What type of Dead is the Internet?

The Autopsy Matrix crosses content origin with audience type. The content axis comes from our AI classification. The audience axis uses the global bot consumption rate — the proportion of web traffic that is automated.

AI Slop = AI-generated content · AI Slurp = bot consumption of content

• Alive — Human-written content reaching a human audience. The real internet.
• Zombified — Human-written content consumed by bots. Real voices talking to no one.
• Polluted — AI-generated content reaching human eyes. Slop you actually read.
• Dead — AI-generated content consumed by bots. Machines talking to machines.

The bot consumption rate (45%) is a cross-industry blend of four major bot traffic reports: Imperva 2025 (51%), Akamai 2025 (51%), Cloudflare Radar 2025 (~30%), and Barracuda 2023 (50%). This rate is applied uniformly across all sources as per-platform consumption data is not publicly available.

Dead Internet Index (Supplementary)

The primary headline metric is the Dead quadrant percentage from the Autopsy Matrix above — it directly answers “what percentage of the internet is dead?” in plain language.

The DII is a supplementary composite score (0–100) retained for internal analysis and historical continuity. It combines four weighted components, though it has known limitations: the Slop×Slurp component is mathematically identical to the Dead quadrant (redundant), bot engagement data is structurally sparse across several sources, and low-confidence human measures classifier uncertainty rather than deadness. The DII does not currently answer a distinct research question from the Autopsy.

When consumption data is available (Cloudflare Radar, robots.txt monitoring), a fifth component (0.20 weight) is added and the other weights adjust downward.

Open Data

DIM's aggregate measurements and per-item classifications are machine-readable. Researchers, journalists, and other monitors can download the data directly — no scraping, no API key, no registration. Use it.

Daily aggregate (recommended)

One row per day, with the headline AI rate, 30-day rolling averages, ex-AutoModerator + ex-orgsync breakdowns, Autopsy quadrant counts, and bot engagement — the same numbers that drive the dashboard. Pass?source=all for per-source rows, a specific source name (reddit, hackernews, etc.) for one source, or omit for the combined series.

• /api/export/aggregate — CSV of the combined daily series
• /api/export/aggregate?source=all — CSV with one row per source per day
• /api/export/aggregate?format=json — same data as JSON

Optional filters: start_date=YYYY-MM-DD, end_date=YYYY-MM-DD, source=<name>.

Per-item content (sample)

Up to 10,000 individual classified items at a time. Useful for replication, methodology review, or sub-population analysis. Each row is one collected post or comment with its classification, confidence, and bot score.

• /api/export — CSV of recent classified content
• /api/export?format=json — same data as JSON

Optional filters: source=<name>, classification=ai_generated|human_created|uncertain, min_confidence=0.0–1.0, content_class=<tag> (include only items with the named tag — e.g. automated_moderation, organizational_syndication), exclude_content_class=<tag> (exclude items with the named tag — mirrors the headline ai_percentage_ex_automod / ai_percentage_ex_orgsync breakdowns at the per-item level), start_date, end_date.

Other machine-readable surfaces

• /rss.xml — RSS feed of published Key Findings
• /api/stats — the live dashboard JSON (combined daily series + summary)
• /api/stats/sources — latest per-source AI rate
• /api/stats/communities — per-community DII and AI rate

Licensing

DIM's data and metadata are published under Creative Commons Attribution 4.0 International (CC BY 4.0). You can copy, redistribute, remix, and build on it for any purpose — including commercially — provided you give credit and link to the licence.

Suggested attribution:

Source: Dead Internet Monitor — deadinternetmonitor.com (CC BY 4.0)

Scope. CC BY 4.0 covers material authored by DIM — aggregate statistics, AI / human / uncertain classifications, classification confidence, bot scores, the Autopsy quadrant counts, the Key Findings collection, the RSS feed, the methodology, and DIM's commentary. It does not extend to original source-content text in the per-item exports (post titles, post bodies, comment bodies, author handles, source URLs collected from Reddit, Hacker News, Mastodon, Bluesky, YouTube, StackOverflow, Lobsters, Steam, and The Guardian); that text remains under the copyright of its original author and the originating platform's terms. Reusers of per-item exports must continue to honour those platforms' terms when redistributing source-content text. DIM's classifications, confidences, and bot scores attached to those items are CC BY 4.0.

Full licence text: creativecommons.org/licenses/by/4.0/legalcode. Repo: LICENSE-DATA. If you publish something based on this data, we'd like to see it: hello@deadinternetmonitor.com.

Updates — Newsletter

Methodology revisions, new sources, and notable findings are announced in the DIM newsletter — ad hoc, no more than once a month, no advertising. Subscribers are stored on Buttondown; email addresses are not used for any purpose other than sending these dispatches and are never sold or shared. See the privacy policy for full detail.

Limitations

• Classification is imperfect. Skilled writers can produce polished text that resembles AI output, and AI can mimic human writing. We target consistency over perfection.
• Sampling bias. We monitor a limited set of platforms. Findings may not generalise to the entire internet. We will continue to increase the number of sources and samples.
• Short content is harder. Posts under 100 characters are capped at 0.60 confidence; posts under 50 characters are excluded from classification entirely. Coverage varies by source — see Classification coverage.
• Temporal drift. As AI-generated content evolves, detection patterns must be updated. We version our prompts and models to track changes.
• Engagement data varies. Some sources (e.g. Reddit via RSS) do not provide vote scores, so bot engagement metrics are unavailable for those platforms. The bot engagement average only includes sources where engagement data exists.
• Text only. Currently, the monitor only measures text-based content. Future iterations will be multimodal.

Operational Reliability

Consistency is one of the three things this project is built on. Daily collection, daily classification, daily aggregation — an unbroken run since launch. The streak is itself a measurement: if the monitor stops measuring, the number stops meaning anything.

How the streak is monitored

Data collection runs on Google Cloud Scheduler, independent of any local machine. The autonomous editorial cycle that produces the daily memo runs alongside it. The pipeline itself has not missed a day. The editorial cycle has, occasionally.

Each cycle writes a heartbeat row to a Cloud SQL table when it completes. A separate Cloud Scheduler job (dim-cycle-alarm-daily) reads that row at 05:00 UTC each morning. If the most recent heartbeat is older than 26 hours, the job emails the operator. Silent on healthy days. One email per miss.

The alarm closes the detection gap, not the execution gap. A missed cycle still has to be caught up by hand. The point is that a miss becomes visible within the same morning rather than several days later via a gap in the timeline. The streak is a credibility metric; the alarm protects it.

What is monitored, what is not

Metadata

Classification Audit Trail

Each classification record stores:

• Model provider and model name
• Prompt version
• Token counts (input / output)
• Estimated cost (USD)
• Latency (ms)

Content Tags

The content_class field tags items by type without changing how they are classified. Tags are applied at collection time and enable breakdown views alongside the headline metrics.

New tags are added autonomously as the dataset grows. A content type qualifies for tagging when it meets all four criteria:

• Non-organic. The content is generated by a bot, platform system, or automated process — not by a human participant in the conversation.
• Deterministic. Membership is objective and rule-based (e.g. author name, system flag), not a judgement call.
• Material. The content type accounts for 10%+ of collected items in at least one source, observed across 3+ collection days.
• Additive only. Tagging must not change headline metrics. It adds a breakdown view, never removes data.

Discovery is automated: a weekly audit samples content from each source and uses an LLM to identify recurring non-organic patterns (bots, system messages, templated content). Candidates are then validated against the materiality threshold before being implemented.

Revision History

Current methodology: Rev 09 (2026-05-27).

The monitor evolves as we add sources, refine methodology, and respond to platform changes. Each revision is backfilled — we re-aggregate all historical data under the current methodology so the full timeline is consistent.

Benchmarks & References

How do our numbers compare to published research? This table maps DIM metrics against independent studies so you can judge for yourself.

Our AI rate (~11%) is conservative relative to web-page studies (40–74%) because we classify community discussion content — comments, posts, answers — where AI penetration is structurally lower than in published articles or SEO-driven pages. Our bot traffic blend (45%) sits below the latest Imperva/Cloudflare figures (51–57.5%) and may be revised upward as source-level consumption data becomes available.

The trend matters more than any single number. We are watching the Dead.