Jessica Hullman, a computer science professor at Northwestern University, contends that contemporary discussions about AI in science conflate two separate things: verification and knowledge. According to Hullman’s essay, the tension centers on whether science’s purpose is producing verifiable outputs—proofs, disease cures—or enabling human understanding.
The debate has concrete implications. Mathematician Terry Tao posed a scenario: would people accept an AI-discovered cancer treatment backed by formal correctness proofs and passed Phase 3 trials, but lacking human understanding of how it was derived? While patients facing stage 4 cancer might welcome any effective treatment, Hullman argues that the question reveals deeper concerns about accountability.
One example involves academic publishing. As Tom Dietterich noted, arXiv increasingly receives papers where human authors appear unable to understand their own claims. Critics question why accountability matters if verification succeeds. But Hullman suggests that when something goes wrong—a trial exploiting experimental weaknesses, or failed claims with no author to consult—human understanding becomes essential.
Hullman traces this problem to a “value-free ideal” originating in Cold War-era philosophy of science. This view holds that only epistemic criteria matter in evaluating evidence, excluding ethical and social values. This ideal shaped expectations that scientific truth could emerge purely through correct procedures, independent of human judgment or context.
The historical roots run deeper. In the late 1800s and early 1900s, scientists promoted the social contract: public funding and autonomy in exchange for societal benefits. To secure broad support and government investment, they framed truth-seeking as impersonal methodology, free from political or social influence. Philosophers of science embraced “epistemic values,” treating any influence from scientists’ intentions as a risk to minimize. This laid groundwork for today’s question: why should it matter who made a claim, so long as it’s correct?
According to Hullman, AI agents verifying results and conducting experiments autonomously represent the “long-awaited fulfillment” of this nineteenth-century vision. But modern philosophy of science, she notes, predominantly rejects the value-free ideal as unhelpful. Values inevitably enter when scientists weigh consequences in accepting or rejecting hypotheses. Even in mathematics—where correctness seems separable from understanding—the possible space of theorems is so vast that values guide which truths researchers pursue. What mathematicians want after AI verification, Hullman suggests, is insight the machine cannot provide.
Key facts
- Hullman distinguishes between verified correctness and actual scientific knowledge, arguing they are not equivalent.
- The debate hinges on whether understanding matters if AI can produce verifiable solutions to longstanding problems.
- Accountability becomes crucial when something goes wrong—without human understanding, there is no one to consult.
- The “value-free ideal” dating to the Cold War era shaped the belief that science requires only correct procedures, not human judgment.
- Modern philosophers of science reject the value-free ideal, recognizing that values influence which hypotheses scientists accept or reject.