Gary MarcusI am genuinely excited.
In my 2023 Senate testimony, in a 2023 essay here, and in my 2024 book Taming Silicon Valley, I strongly advocated for a system of preflight testing for AI models deployed at large scale, ideally mandatory, transparent and independent, modeled perhaps on the FDA’s analysis of costs and benefits for pharmaceuticals. When Senator Kennedy asked for my most urgent suggestion for AI policy, preflight testing was it.
At the time, back in 2023, the Senate seemed receptive. But nothing happened, and things changed markedly when Trump came to office. As recently as a few months ago, getting the US to implement preflight testing seemed hopeless, given a great hostility to AI regulation that was then widespread in Washington.
But the Mythos moment got The White House to consider and in fact implement preflight testing to some degree (though not with the transparency or independence or breadth or mandatoriness that I believe to be critical).
Now, I am thrilled today to report that Google DeepMind’s CEO Sir Demis Hassabis has just come out strongly and publicly in favor a version of preflight testing.
Hassabis adds that
Initially, Frontier Labs would voluntarily share models with the Standards Body for review up to 30 days before release. Once the assessment protocol is shown to be effective and robust, formalisation could quickly follow, meaning that Frontier Models would be required to pass it to be deployed in the US market. Labs would also work with the Standards Body to address any critical post-release vulnerabilities.
I am especially pleased by his reference to independent leading technical experts; it cannot be just the government and big tech companies that make these decisions, particularly because of the potential for conflict of interest in a time in which the US government is contemplating take a stake in AI companies.
You can read Hassabis’s full essay here. The FINRA model that he suggests is excellent, and the world will be a better place if it is implemented, with transparency and independence. I hope that his brave essay will be a turning point.