A dataset with 52 Text to image model evaluation [P]
热度趋势
百分比基于当前可用热度信号，而非评论数或独立用户人数。
Hugging Face 相关模型动态已经出现，适合跟踪能力变化、生态影响和后续可用性。
一个针对文本到图像（T2I）模型的新基准测试已经创建，包含192个具有挑战性的提示，旨在测试文本渲染、空间推理、人物真实感和否定等方面的能力。该基准测试评估了52个T2I模型，生成并分析了超过9,000张图像。一个视觉语言模型（VLM）根据预设的二元问题对输出进行判断。…
I created a simple text to image benchmark.
I curated 192 prompts that are difficult for T2I models in various ways: text rendering, spatial reasoning, human realism, negations, etc...
I then asked a VLM to judge every output against a pre-specified binary question with the ground truth baked in.
I'm publishing all the results including the images. (Most public T2I leaderboards don't publish the actual images and that's a shame IMO)
There is currently 52 model tested! more than 9k images have been generated and analysed!
Full methodology: https://imagebench.ai/methodology-v1
Hugging face dataset: https://huggingface.co/datasets/dh7/imagebench (it contains the prompts to reproduce the results AND the results)
Github: https://github.com/dh7/image-bench-ai
Gallery to inspect the results: https://imagebench.ai/gallery
Leaderboard: https://imagebench.ai/imagebench-v1
Limitations: it's text to image only, and VLM are not perfect as a judge.
Let me know what could be useful from there!