Dario Amodei's Preferred AI Watchdog Has Deep Ties to Anthropic, Effective Altruism

Anthropic CEO Dario Amodei’s call for outside monitors to police the AI industry has run into questions about how independent those monitors really are. The watchdog Amodei mentions, METR, is deeply connected to both Amodei’s Anthropic and the Effective Altruism movement.

Breitbart News reported that Amodei laid out his proposal in an essay titled “We Must Pace the Frontier,” published last Saturday morning. In it, he argued that leading AI companies should allow “embedded third-party evaluators” to maintain “employee-like access” so they can oversee safety efforts inside the labs. OpenAI’s Sam Altman

The nonprofit Amodei has in mind, the Model Evaluation and Threat Research group known as METR (pronounced meter), has deep and long-running ties to Anthropic and to Amodei himself according to reporting by the New York Post. METR spun off from the Alignment Research Center, an Effective Altruism-aligned tech incubator run by Paul Christiano.

Christiano and Amodei were housemates, coworkers and research collaborators in the 2010s at OpenAI, Anthropic’s biggest rival. Christiano later became one of the first five trustees of Anthropic’s Long-Term Benefit Trust.

The connections run through METR’s staff, too. Ajeya Cotra, a METR staffer, is married to Christiano and is a major organizer in the Effective Altruism world. METR’s founder and chief executive, Beth Barnes, was hired by Christiano and was involved in the movement in college.

The organization’s chief scientist, Hjalmar Wijk, and staffers Megan Kinniment and Pip Arnott all came out of the same research cluster at the University of Oxford, which the Post describes as a major feeder for Effective Altruism.

Wijk’s LinkedIn profile says he currently works on “trying to reduce worst-case risks from future models.” Kinniment previously worked at the Center on Long-Term Risk in London, where, according to her LinkedIn profile, she focused on how “malevolent actors could increase the likelihood of poor AI outcomes.” Jasmine Dhaliwal is listed as a policy staffer at the organization.

Amodei’s own ties to Effective Altruism run further still. He has pledged to give away 80 percent of his personal wealth, and his sister Daniela is married to Holden Karnofsky, who founded two major Effective Altruism nonprofits. Sam Bankman-Fried, the disgraced crypto executive whose empire collapsed amid fraud charges that sent him to prison, was one of Anthropic’s earliest investors and a prominent figure in the same movement.

METR and its affiliates have also drawn funding from Effective Altruism figures who are themselves Anthropic investors, including Democratic megadonor Dustin Moskovitz and Skype cofounder Jaan Tallinn.

In August, METR announced it had raised $71 million in new funding from outside investors over the previous six months. Public records show Tallinn’s Survival and Flourishing Fund has earmarked up to $752,000 for METR since its founding, including $324,000 in general cash and a $428,000 matching pledge.

Moskovitz’s Coefficient Giving, formerly known as Open Philanthropy before rebranding last year, gave $1,515,000 to the Alignment Research Center, METR’s incubator, in 2022. METR itself was originally known as ARC Evals before spinning off as an independent nonprofit and adopting its current name in 2023.

A METR spokesperson defended the group’s independence in a phone statement to the Post: “We don’t think any small group should have authority over what happens with AI, and our goal is to get information from inside these companies into the public domain – external investigators can help bring more information to light.”

The spokesperson said METR does not take compensation from AI labs for its work and does not accept donations from executives or employees of AI companies, adding that Moskovitz’s 2022 donation to the Alignment Research Center was firewalled and never used for METR’s operations. “Our funders have no say in the projects we work on and we do not accept any funding from AI companies or their employees,” the spokesperson said.

Not everyone is convinced. A source familiar with the White House’s thinking told the Post on Tuesday that “METR is essentially Anthropic in another form.” Perry Metzger, chairman of the Alliance for the Future, was blunter: “Saying you want to be audited by the ‘independent’ people at METR is a complete joke. It’s people who are funded by your buddies. It’s completely ridiculous.”

This viewpoint aligns with David Sacks, who responded to Amodei’s essay by posting, “Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff.”

Metzger compared the arrangement to bank oversight: “If you went out tomorrow and you’re a big bank and you had your audits done by an organization consisting entirely of your own former employees, which was funded by the same investors that are investing in you, do you know what the feds would do with you? It would be positively medieval.”

Breitbart News social media director Wynton Hall explained Effective Altruism and how it intersects with the AI industry in his New York Times bestseller Code Red: The Left, the Right, China, and the Race to Control AI:

What does all this mean, and what is the EA movement’s overarching goal? According to President Trump’s [former] AI czar, David Sacks, Republicans should understand that “hyperbolic and unproven claims” about potential AI job losses are “not an accident” but rather “part of an influence operation.” Toward what end? “The goal,” he says, is “to further ‘Global AI Governance,’ a massive power grab by the bureaucratic state and globalist institutions.” Global AI governance is the moniker used to describe the regulatory effort by globalist organizations and leftists to control virtually every dimension of AI deployment under the banner of “AI safety” for the world. He notes that the EA organizers behind global AI governance are “billionaires with a long history of funding left-wing causes and Trump hatred.” He says that “it’s fine to be concerned about a technology as transformational as AI, but if you repeat their claims uncritically, you may be falling for an AstroTurfed campaign by the ‘AI Existential Risk Industrial Complex.’”