A researcher monitoring several AI-driven research projects has expressed skepticism about claims that autonomous R&D by language models is particularly sophisticated, pointing to systematic limitations in the models’ judgment and output quality.
According to the author’s observations, AI models frequently produce research that is methodologically incomplete in similar ways across projects. The author reviewed eight papers from an Apart hackathon that scored between 60% and 95% on Pangram and found they were often very similar to each other, with each overlooking straightforward analyses in comparable ways.
The author identifies one category where AI models show more promise: tasks with fast feedback loops and clear performance metrics. In one example, the author improved the state-of-the-art performance on “listing all legal Scrabble moves” by 10 times through iterative prompting—asking the model to try harder, aim for specific performance levels, and systematically try related methods from the literature. However, even in this case, two significant leaps came from human direction: when the author suggested measuring performance across full games rather than random positions, and proposing that the model design algorithms for different scenarios based on the number of blank tiles remaining.
The author also notes that language models—specifically citing Astra and Claude—tend to write research reports that are extremely difficult to understand. According to the author, interrogating the model with follow-up questions is often necessary to determine what the model actually did and what the key results are. The author speculates there may be a temptation to overlook unclear output quality and “just let it rip” rather than demand clarity.
One commenter responding to the observation noted that Opus 5.5 produces significantly clearer writing than Astra 6, suggesting variation in report quality across different models.
The author’s assessment suggests that while AI models may excel at specific optimization tasks with clear feedback mechanisms, they do not demonstrate the broad research ingenuity often attributed to them. The combination of methodological gaps, similar errors across independent projects, and communication difficulties indicates that autonomous AI research still requires substantial human oversight and direction.
Key facts
- Eight hackathon papers reviewed by the author scored 60-95% on Pangram but showed similar methodological gaps
- AI achieved 10x improvement on Scrabble move listing task, but required human guidance for key insights
- Language models produce research reports that are difficult to understand and often require follow-up questioning to clarify results
- The author expresses skepticism about claims of sophisticated autonomous AI R&D based on observed limitations in judgment and output quality