Most open-source AI detectors can't hold a 0.5% false-positive rate [P]
热度趋势
趋势数据积累中
百分比基于当前可用热度信号，而非评论数或独立用户人数。
Hugging Face 相关模型动态已经出现，适合跟踪能力变化、生态影响和后续可用性。
一项针对开源AI检测器的研究发现，大多数检测器无法将误报率控制在0.5%以内。研究人员使用相同的协议对所有知名检测器进行了测试，结果显示不同模型的性能差异显著，例如“tropa-mini”的ROC-AUC为0.968，而“desklib/ai-text-detector-v1.01”为0.875。在测试的六个检测器中，有一个由研究作者开发，并以Apache-2.0许可开放权重发布，所有数据集和方法均可在Hugging Face上查阅。
We needed to know where the open-source AI-detection field actually stands, so we ran every notable open detector through the same protocol.
Setup:
- Public data only: Jabarian & Imas 2025 (NBER), Liang 2023 TOEFL essays, a 1,060-text frontier set (GPT-5.x, Claude Opus 5, Gemini 3.x), 5,000 pre-LLM (2018) FineWeb pages as human pool
- Every model gets its threshold set on the same 6,930 human docs to a matched 0.5% FPR
- Then measure recall per group: raw AI, humanizer-paraphrased AI, frontier models
What surprised us:
- 4 of 6 models effectively can't reach 0.5% FPR. MAGE scores >0.9999 on 26% of ordinary human web text. The old OpenAI RoBERTa detector lands at AUC 0.31 - worse than coin flip on modern generators.
- Humanizer-paraphrased text is where everything collapses: best model catches 42%, second best 4%.
- All models flag non-native essays at a higher rate than they flag native essays. This is a fundamental flaw in the entire class of models, not just one.
model ROC-AUC raw AI humanized AI frontier models tropa-mini 0.968 93.2 % 41.6 % 33.6 % desklib/ai-text-detector-v1.01 0.875 83.9 % 4.0 % 1.8 % SuperAnnotate/ai-detector 0.824 0.5 % 1.4 % 0.6 % Hello-SimpleAI/chatgpt-detector-roberta 0.571 0.8 % 0.4 % 0.2 % yaful/MAGE 0.507 —* —* —* roberta-large-openai-detector 0.313 0.0 % 0.1 % 0.0 % More information and data on HF
\ recall at matched 0.5% FPR; * MAGE can't reach 0.5% FPR at any threshold (flags 26% of human web text with score >0.9999)**
Disclosure: one of the six is ours - we run a hosted detector and released this one as open weights (Apache-2.0). All datasets and the methodology are in the model card, so you can rerun everything: huggingface.co/wasitaigeneratedcom/ai-text-detector-small