HomeAI · summaries
Storia in 2 fonti
OpenAI finds roughly 30 percent of popular AI coding test is broken
OpenAI reviewed SWE-Bench Pro, a widely used test for measuring AI models' programming skills, and found roughly 30 percent of its tasks are broken. The company is pulling its earlier endorsement of the benchmark.
Confronto fonti
2 prospettive sulla stessa storiaTimeline cronologica
- ·
the-decoder.com
OpenAI finds roughly 30 percent of popular AI coding test is broken
OpenAI reviewed SWE-Bench Pro, a widely used test for measuring AI models' programming skills, and found roughly 30 percent of its tasks are broken. The company is pulling its…
- ·
dev.to
OpenAI just found ~30% of SWE-Bench Pro is broken — and retracted their own recommendation
OpenAI pulled the plug on SWE-bench Verified earlier this year after finding contamination and design...