California's AI Bouncer: No Unvetted Models Allowed at the Golden State Party

Governor Gavin Newsom just made it a lot harder for AI companies to sell to California—unless they can prove they're not about to unleash digital chaos on the state's agencies. Think of it as the velvet rope outside the hottest club in the West, except instead of checking IDs, they're checking for model bias and watermark compliance.

On Monday, Newsom signed an executive order requiring AI companies seeking state contracts to demonstrate policies that prevent misuse and protect privacy, security, and civil rights. Companies selling AI systems to California agencies will now need to show their homework before getting a check. No more selling the state's data to the highest bidder and hoping for the best—California wants to see the receipts first.

"California's always been the birthplace of innovation. But we also understand the flip side: in the wrong hands, innovation can be misused in ways that put people at risk," Newsom said in a statement. "California leads in AI, and we're going to use every tool we have to ensure companies protect people's rights, not exploit them or put them in harm's way. While others in Washington are designing policy and creating contracts in the shadow of misuse, we're focused on doing this the right way." Basically, while the feds figure out which way is up, California is out here playing GDPR parent.

The order directs the state's Government Operations Agency to develop procurement standards for AI vendors addressing illegal content generation, model bias, and risks to civil rights and freedom of speech. The California Department of Technology will develop recommendations for watermarking AI-generated images and manipulated video. Yes, that means your friendly neighborhood deepfake machine might need to come with a watermark so obvious your grandma could spot it.

This puts California directly in conflict with the Trump administration's effort to establish national AI standards and limit state-level regulation. Earlier this month, the Trump administration released a national artificial intelligence policy framework urging Congress to establish federal standards and reduce what officials describe as a patchwork of state AI regulations. It's the classic federal vs. state showdown, but with more GPUs and fewer cattle.

Kevin Frazier, an adjunct research fellow at the Cato Institute, said the dispute reflects a longstanding constitutional balance between state and federal authority. "Every technological breakthrough—from the steamboat to superintelligence—raises key questions about how to allocate regulatory authority between the states and the federal government,"