Thnkrr — Moderation: The Decision Brief
What this is: the one-page version of Moderation Stack: Verified Research and Recommended Decisions — just the recommendations, the answers to your redline questions, and what we need from you. Every claim is verified in docs/20 with sources; the section pointers below go to the evidence. 2026-09-21.
The stack we recommend
| Layer | Pick | Evidence |
|---|---|---|
| Our own rules | Keep the built stage-1, hardened against disguised text (leetspeak, look-alike characters) | building now |
| Cheap pre-screen | OpenAI Moderation API — free, zero retention by default, used in education. Hive ($0.50/1k) is the bake-off alternative | §1 |
| General policy model | gpt-oss-safeguard-20b on Amazon Bedrock, retention locked off at the account level | §3 |
| Grooming gate | Thorn Safer Predict or Hive (Hive licenses the same Thorn model). Quote-only — two emails are drafted and ready | §2 |
| People | Our own queue, two named reviewers, SLAs anchored to Snap/Discord/Roblox published numbers | §6 |
| Images (avatars/banners) | Keep the built review-every-image-by-a-person pipeline; add an AWS-only AI pre-screen (Rekognition + a Bedrock vision pass) and free CSAM hash-matching | §10 |
Cost: everything above the grooming vendor ≈ $4.32/month at pilot volume, plus ≈$11 in the image layer's first month (then <$2/month) — all covered by the AWS credits. The grooming vendor is the only open price.
Answers to your redline questions
Self-hosting the general model — the premise doesn't hold, and you get the goal anyway. Amazon Bedrock serves gpt-oss-safeguard directly with zero retention, zero operator access, and a guarantee that the model's maker never sees inputs. Since we're already moving to AWS, Bedrock is the same processor already in the privacy notice — zero new third parties, which was the whole point of self-hosting. A self-hosted GPU is $478–587/month (≈110× Bedrock), sits >99% idle at pilot volume, and freezes all publishing when it crashes. One thing gates adoption: a week of our own latency benchmarking, since nobody has published numbers for this variant. (docs/20 §3)
Safer Predict — your claims check out, with three caveats. Conversation-level scoring, grooming/sextortion/child-access labels, and self-hosted deployment are all real and verified. Caveats: the "has-minor" label is from the beta write-up and unconfirmed today; languages are English + Spanish only (no Hindi/Hinglish — human review covers that gap); and the one public price anchor is $45,384/yr for the self-hosted enterprise bundle, so the Thorn-hosted API tier is the realistic path. Your hold on mentor messaging stands until Thorn or Hive is live. (docs/20 §2)
Anthropic's terms — usable, with conditions. Their policy explicitly allows minor-serving products under published guidelines (age verification, moderation, COPPA statement…). Two catches: zero-data-retention agreements are not available for their newest models without express authorization — a ZDR deal must name an older model — and even under ZDR, flagged content is kept up to 2 years, which must be disclosed in the guardian notice. There's no formal approval program, so the use case should be pre-cleared with their sales team in writing. Claude stays on the bake-off shortlist. Also settled: OpenAI usable with ZDR; Gemini and DeepInfra are out (under-18 bans). (docs/20 §4)
Photos and banners — they ship, and the checking layer is ready for approval. Avatars and banners are the only image uploads in the product, and the pipeline is already built and fail-closed: every image is reviewed by a person before it can appear — which exceeds the published practice of every mainstream platform except Yubo and Roblox. What we add on your sign-off: an AI pre-screen that narrows the human queue (Amazon Rekognition for nudity/violence/hate-symbols/drugs/weapons plus reading any text drawn into a banner — all inside AWS, no new third party, ≈$11 for the entire signup surge), and a CSAM layer via four free applications (Microsoft PhotoDNA, two hash databases, and NCMEC's hash list, which our existing image-hashing can match locally so images never leave our servers). One non-negotiable: an AWS account setting must be flipped before the first student image is processed — by default AWS may keep images to improve its AI, and the opt-out is what makes our "no training on student content" promise true for images. Verified answer on the legal risk: content moderation over a face-containing photo is not "facial analysis" or biometrics under the settled case law, provided the vendor never computes face geometry — Rekognition's moderation responses contain no face data, and we lock the face APIs off at the permissions level. Avatar rule: LinkedIn's model — "the photo must be you," enforced by the human queue, never by face matching. (docs/20 §10)
What we need from you
- Sign off the stack table above. Each row becomes a D- entry in docs/07 and the bake-off (docs/20 §7) starts.
- Send the two vendor emails — drafted, with the exact questions to ask, in docs/20 §2.4 (Thorn and Hive). The deciding question for Hive: can its API take a whole conversation, or single messages only? Both emails now also ask for image CSAM coverage in the same quote — one conversation per vendor covers text + image.
- Four names: the two reviewers; the NCMEC filer; the published India Grievance Officer. Note: under India's POCSO, failing to report a subordinate's finding is personal criminal exposure for the person in charge — the filer must be named in the runbook, not improvised. (docs/20 §5.2–5.3)
- Approve the image layer (the photos/banners answer above): the Rekognition + Bedrock pre-screen behind the existing human queue, and the four free CSAM hash-matching applications (PhotoDNA, IWF, Project Arachnid, NCMEC hash list) — all free to apply for today. (docs/20 §10.2)
What we build regardless (not waiting on any sign-off)
- Intimate-image removal flow on a 2-hour clock. India amended its rules in Feb 2026: takedown on complaint is now 2 hours, not 24 — one pipeline built to that clock satisfies the US 48-hour law automatically. Public, logged-out request page. (docs/20 §5.1)
- NCMEC registration this month, plus the reporting runbook. New finding: every case involving an Indian child must also be filed with local police — India's Supreme Court held that NCMEC alone forfeits our safe-harbour protection. (docs/20 §5.2–5.3)
- Published Grievance Officer with India's 24h/72h complaint clocks. (docs/20 §5.3)
- Crisis cards: US 988 (the LGBTQ+ "Press 3" option was discontinued — don't show it); India Tele-MANAS 14416 (KIRAN has been merged into it — don't list it). (docs/20 §5.4)
Still being tested, not assumed
The bake-off picks the pre-screen and confirms the policy model; the one-week latency benchmark gates Bedrock; the grooming-vendor conversation gates mentor messaging. Nothing above ships on a vendor's marketing claim. (docs/20 §7, §9)
Sign-off on the stack table triggers: D- entries in docs/07, the bake-off, the two vendor emails, and — for the image layer — the four hash-list applications and the AWS opt-out flip. The statutory builds are already unblocked.