I represent a district whose constituents work for many of the companies building the models, compute, benchmarks, and data pipelines driving a lot of these conversations. This may come as a surprise to many, but they are also the people I hear from who are most concerned about the future of AI. That’s why I’ll be writing this series on my Substack, to discuss how we can legislate to meet the moment, and what I’m hearing from experts across industry, academia, and civil society. I want to start by talking about some progress we’ve already made on providing more transparency to consumers around AI-generated content, and how we got here.

In 2024, I passed the California AI Transparency Act - the first law in the nation to impose content transparency requirements on AI developers, and as of August 2nd, came into effect on the same day as the EU AI Act’s obligations under Article 50. The goal was straightforward: ensure that Californians understand where the content they view online comes from, so they can know what to trust. But there’s a lot of complexity in regulating an industry that is evolving almost daily.

AI has made remarkable advances to the benefit of businesses and consumers alike. We’ve also continued to see harmful deepfakes and AI-powered misinformation spread across the internet, undermining public institutions, news outlets, and democracy. This year, I’m authoring SB 1000 to adapt the law to reflect the technical realities that have come to the forefront in the years since the original law was signed. I’ve engaged with a wide set of subject matter experts across industry, civil society, and international authorities to ensure that the legislation we pass here in California helps push the world towards a more cohesive ecosystem of provenance. This includes meeting with those in the EU who are currently drafting the Code of Practice, as well as the French Ambassador for Digital Affairs, Clara Chappaz, around their thinking on content labeling.

How the law works

In practice, CAITA will mandate that providers of AI systems embed a common set of information into the metadata of content they generate - similar to a nutrition label that discloses:

1. The name of the developer,

2. Whether or not the content is generated or modified by artificial intelligence,

3. The name and version information of the AI system that made or altered the content,

4. The time and date of the content’s creation,

5. A unique identifier of the content, and

6. In 2029, whether the system that created the content is designed to function as an assistive technology for individuals with disabilities.

This information will travel with the content, and when posted to a large online platform like Instagram or YouTube, they’ll be required to disclose whether or not the content was generated or substantially altered by AI. The law also mandates that the tools used to detect the information have an accessible API so websites, online platforms, apps, or any other web-based software can call on the tool to check the origin of content.

Standardization - Benefits and Trade-offs

One of the major decisions we made this year was to include language in the bill that would encourage the adoption of technical standards. This was significant because - as I’ll outline below - content provenance technology is still developing and is by no means perfect. Many developers have adopted an open-spec standard called C2PA, created by an industry-led coalition which serves as a possible baseline for how provenance data can be embedded in content. It’s designed to be extensible so that developers can add measures to increase the resilience of that information. For the past year I’ve been in negotiations with the steering committee members of the standard in collaboration with the advocacy groups who originally helped draft the bill. Think of it like a certified mail system for digital content. When you send certified mail, the postal service stamps it with a verified record of who sent it, when, and where - and that record travels with the envelope. Under C2PA, every piece of content gets a machine-readable stamp that records its origin and any modifications, and that stamp is cryptographically tied to a verified account so it can’t be forged without detection - known as a “hard-binding.”

Let’s Get Technical

The standard is structured so that content generators (ChatGPT, Midjourney, or camera manufacturers like Nikon) apply to join and are issued certificates to sign the disclosures they embed in their content using cryptographic keys. Validators (Google SynthID, C2PA’s own verifier, or any other tool used to read C2PA disclosures) are also approved by the standards body, and verify those disclosures by checking the signer’s certificate against a registry of trusted certificate authorities.1 Those certificates expire after a set validity period for security purposes. If a signing key is compromised or stolen, the issuing certificate authority can revoke the certificate before it expires. For users, when metadata is lost in a piece of C2PA conformant content, the validator system that checks for the content’s information simply shows that the content’s origin can’t be verified, unless the content has what’s called a “soft-binding” - a type of disclosure that allows a validator tool to recover lost metadata.2 If the signed disclosures in the metadata are like stamps on a piece of mail, you can think of soft-bindings like invisible ink written on the envelope itself that only machines can read. These aren’t yet required by SB 1000 as these kinds of watermarks are still being refined, although some tools like Google’s SynthID have begun incorporating them more frequently. However, I think it’s important to acknowledge the shortcomings of this technology generally, particularly because of C2PA’s position as the leading standard.

First, consistency and scalability of techniques used to ensure provenance data is resilient to transformations is an ongoing challenge. Another problem we’re seeing even in the basic “hard-binding” disclosures (the stamp on the envelope) of C2PA is that the tools used to validate disclosures aren’t always consistent with one another, which can lead to the same piece of content being portrayed in two different ways across disclosure verification tools. Second, a major issue I have been pushing C2PA to address is that a validator tool can see a perfectly intact disclosure, confirm the certificate chain, and declare the disclosure “valid” without ever checking whether the certificate authority has since invalidated the key that made that disclosure. If a business is C2PA conformant, but its signing key becomes compromised, there is no requirement that a validator check whether those certificates have been revoked. Due to these limitations, we included language which would allow for new technologies that are interoperable with leading standards to comply with the law so that innovation can continue without fragmenting the ecosystem beyond what’s useful to accomplish the policy goals, and we don’t settle for the first solution presented to us.

Another significant addition we made this year was to allow people to embed personal information in the content disclosures required under the bill if they explicitly consent to doing so. It’s important to be clear that personal information can be embedded in all manner of content with C2PA - synthetic or otherwise. Many researchers and governments are looking for ways to leverage content provenance infrastructure for the purposes of digital rights management to help ensure that people have a say on whether the content they publish can be used for AI training, and explore ways to fairly compensate creators for licensed use. While that is out of the scope of my work this year, it is certainly something I’m considering for the future.

Visible Markings

When most people think of marking AI content, they envision visible marks on images. When we originally passed the California AI Transparency Act (SB 942) we included a provision that mandated companies provide users with the option to include them in image, video, and audio. However, SB 1000 removes that obligation, and here’s why. Perceptible watermarks are legible and frictionless, but that same ease of access has real trade-offs. First, there is not a single, monolithic mark that is universally applied to content. There would be nothing stopping people from creating their own labels which could confuse or mislead consumers. Additionally, visible marks are extremely easy to remove or misappropriate with little evidence that the content was tampered with. This is part of the reason why the EU has backed away from visible disclosures, requiring them only for specific kinds of content, in specific contexts. Finally, as awareness of the amount of AI content on the internet grows, people may come to assume that content without a perceptible watermark is inherently more credible.3 That assumption would be wrong.

Whether content is factually accurate and whether it was synthetically generated are largely unrelated questions. A photo can be real and misleading when put in the wrong context, and an AI-generated image can be benign. The real policy problem we’re trying to solve is misinformation, which requires two things: 1) Increased media literacy so people know to critically examine what they see online, and 2) Reliable information people can trust about where content comes from. The AI Transparency Act addresses the latter of these issues head-on. It pushes companies, device manufacturers, and media organizations to conform to standardized provenance approaches so that there is a consistent stream of information being embedded in the content we see online. That information will be required to be made available on large online platforms so people can easily assess where content comes from, and whether or not to trust it.

Subscribe now

The Open-Source/Open-Weight Question

One of the harder problems we’ve been wrestling with is what to do about open-weight or open-source models that anyone can download and run locally on their own hardware.4 While these models are often smaller (13B-30B parameters), the performance gap between them and those on the frontier is closing.5 This is important given the rise of AI-generated CSAM and NCII, much of which is created using these kinds of systems.6

This presents two problems. The first is that it’s difficult to trace content from these systems back to specific individuals - since the user prompt and model response are contained to their system. The second is that regulating the upstream provision of these models could harm the ecosystem of open systems, which have also been beneficial to academia, small businesses, and law-abiding users. While there are currently technical solutions being developed around baking provenance into the internals of these models, none are yet mature enough to make it mandatory by law. It’s a space I will continue to monitor as the bill goes into effect. Under SB 1000, open-source systems are required to embed provenance to the extent technically feasible, and if a licensor discovers someone is out of compliance, they have to notify the licensee that they’re breaking the law and request they modify the system to comply. If the licensee doesn’t alert the licensor of what action they’re taking, the licensor reports them to the Attorney General.

What comes next

Lawmakers can’t approach AI content labeling through a binary “AI or not AI” lens. The technical limitations and the potential for unintended consequences demand that we get the details right. Legislation must innovate along with technology. I am a firm believer in our government being capable of finding innovative solutions for complex problems. Achieving that goal requires nuance, and a willingness to adapt. The reality is that no technology is perfect, and we need to make sure that the laws we pass stand the test of time. The California AI Transparency Act will move us toward a more trustworthy information ecosystem, and I’m committed to being as nimble as the Silicon Valley startups I represent as each new wave of tech comes. California has led on complex issues before, and I’m confident we’ll lead here too.