Does this AI app train on your chats? Every cell cited.

PrivacyMatrix compares 28 consumer AI assistant apps across 14 privacy questions. Each of the 392 cells stores the sentence from the vendor's own privacy policy, terms or help center that justifies it, the URL of that page, and the date the sentence was last found there. Every question is phrased so that "yes" is the better answer for your privacy. A free GitHub Action re-checks every quote weekly.

Published 2026-09-11 · Data verified 2026-09-11 · 314 quotes found at their source · 78 cells unknown where the documents are silent

The problem

"Does ChatGPT train on my conversations? Can I turn it off? If I delete a chat, when is it really gone? Does Meta AI use what I ask it to pick ads?" People ask these questions every day, and the answers they get are usually a months-old news article, a screenshot of a settings page, or a confident guess. The policies themselves are long, change quietly, and are almost never compared side by side with the actual sentences that matter.

PrivacyMatrix answers from the documents and shows its work. It is the second product built on the engine behind AgentMatrix, the cited feature matrix of AI coding agents: the same pipeline, pointed at privacy policies instead of developer documentation.

What is different

• Every value has a receipt. A cell is only shown as yes, partial or no when its quote was found, verbatim, at the linked URL. Formatting differences are tolerated (whitespace, capitalisation, punctuation, smart quotes), wording differences are not.
• Yes always means better for your privacy. "Chats excluded from training by default: yes" means the policy says your conversations are not used for training unless you opt in. The count of yes answers under each app name is a rough privacy score; the cells are the real answer.
• Silence is never read as an answer. If the documents do not address a question the cell is unknown, not no. Vendors are not accused of practices their documents do not describe, and they are not credited with protections their documents do not state.
• Quotes are re-checked every week, for free. A GitHub Action re-fetches every source page and searches for every quote. A quote that is not found opens an issue at once; if it is still missing a week later its cell is demoted to unknown. A rewritten policy can make the table go stale into "unknown"; it cannot leave a stale "yes" standing.
• The decision rule is written down. Each question has a rubric that says exactly what counts as yes, partial or no (for example: a self-service setting that stops training on new conversations is yes; an opt-out that exists only as a regional legal right is partial; training on by default with no documented way to stop it is no). A disagreement about a cell is a disagreement about the rubric, and the rubric is versioned in the repository.

What the matrix covers

Apps (28): ChatGPT, Claude, Gemini, Microsoft Copilot, Perplexity, Grok, Meta AI, DeepSeek, Vibe (formerly Le Chat, Mistral), Character.AI, Poe, Pi, You.com, Kimi, Qwen Chat, Duck.ai, Brave Leo, Lumo (Proton), HuggingChat, Replika, Apple Intelligence and Siri, Alexa+, Snapchat My AI, Kagi Assistant, Venice, Manus, Genspark, Z.ai.

Questions (14), in five groups:

What the data says (September 2026)

A few rows that stood out while building it. Every claim below is backed by the same evidence the matrix shows; open the live matrix and click the cell to read the quote.

What it does not tell you

• Documents, not behaviour. A cell says what the policy states, not what the vendor does. Nothing here audits practice.
• Regional variants. The matrix scores the global or US version of each policy. Meta, for example, serves different English variants by region, and EU users often have objection rights the US text does not describe. Differences are recorded in the notes of a cell, not in its value.
• Fetchability. Character.AI, Qwen Chat and Manus publish their policies only through JavaScript, so their cells cite the vendor's own script bundles; those URLs change on deploy and the weekly check will demote the cells when they do. OpenAI, Perplexity and xAI refuse requests from cloud IP ranges, so the weekly Action cannot re-check those 40 cells at all; each run lists them as unreachable and maintainers re-verify them from an ordinary connection.

How it works

Two loops, one free and one that costs API credits.

Every week, free

1. fetch every evidence URL in data/matrix.json (policy pages as text)
2. search each page for the cell's quote, tolerating formatting but not wording
3. quote found      -> verified_at = today
   quote missing    -> first time: cell kept and flagged, listed in an issue for a human
                       still missing a week later: value = "unknown"; old quote, URL and
                       value stay in the notes
   page unreachable -> cell untouched, reported
4. no value changed -> the refreshed dates are committed
   values changed   -> a pull request and an issue list the affected cells for a human

When a policy changes, with an API key

1. fetch every source page listed for the app in data/apps.json
2. Claude reads all of it in one request and returns one JSON entry per question:
   value, verbatim quote, source URL, notes, constrained by a schema
3. every returned quote is searched for in the fetched text; a quote that is not found
   demotes the cell to "unknown" (a fabricated or paraphrased quote does not survive)
4. if the new answer is "unknown" but the previous verified quote is still on its page,
   the previous cell is kept, so one bad answer never erases good data
5. the result is diffed against the previous matrix and opened as a pull request

The initial data was researched the same way, then audited twice: once per app against the rubric, and once across all 28 apps for consistency, so that two vendors describing the same situation get the same value. The mechanical check proves that a sentence exists on a page. Whether the sentence supports the value is a judgment made by the rubric, the model and the reviewers, in that order. Cells are open to challenge: report one with a link to the policy.

What it costs

Nothing to run. GitHub Actions and GitHub Pages are free for public repositories, and the weekly quote check needs no API key. Re-deriving cells with Claude is paid by whoever runs it, with their own key: at list prices roughly $0.30 to $2 per app with claude-fable-5-1, on the order of $20 for a full pass over 28 apps. The repository ships no key, and it does not need one to stay honest.

Contributing

Six more apps are on the wanted list as good first issues: Doubao, Yuanbao, Ernie Bot, T3 Chat, Nomi and Chai. Adding one means listing its policy URLs in data/apps.json, filling the 14 cells (with your own key, by hand, or leaving them unknown for someone else), and running npm run check, which fails if any quote is not found at its URL. Questions are open for proposals too; a good proposal is phrased so that yes is the more protective answer and comes with a rubric two people would apply the same way.

• Repository: github.com/barbarkaragul-oss/privacymatrix (MIT)
• Live matrix: barbarkaragul-oss.github.io/privacymatrix
• Raw data: matrix.json, with a CITATION.cff in the repo for citing it
• Sister project: AgentMatrix, the same engine applied to AI coding agent CLIs