Page 1


Date of Hearing: August 7, 2024

ASSEMBLY COMMITTEE ON APPROPRIATIONS

Buffy Wicks, Chair

SB 1047 (Wiener) – As Amended July 3, 2024


Policy Committee: Privacy and Consumer Protection Vote: 8 - 0

Judiciary 9 - 1

Urgency: No State Mandated Local Program: Yes Reimbursable: No

**SUMMARY:**

This bill establishes requirements the developer of an artificial intelligence (AI) model trained
using a specified level of computing power (“covered model”) must meet before training and
releasing the model, establishes a state entity to provide oversight and guidance regarding
covered model compliance, provides related whistleblower protections, grants enforcement
authority to the Attorney General and Labor Commissioner, and creates CalCompute, a public
cloud computing cluster.

Specifically, among other provisions, this bill:


1) Establishes risk assessment, safety, security, and testing requirements the developer of a

covered AI model must fulfill before training the covered model, using the covered model, or
making the covered model available for public or commercial use.

2) Requires, beginning January 1, 2028, the developer of a covered model to annually retain a

third-party auditor to perform an independent audit of compliance with the requirements of
the bill.

3) Establishes the Frontier Model Division (FMD) within the Government Operations Agency

(GovOps) to, among other responsibilities, annually review certifications from developers,
establish an accreditation process for third party auditors of covered models, publish
anonymized safety reports, and issue guidance, standards, and best practices necessary to
prevent unreasonable risks of covered models and covered model derivatives causing or
enabling critical harms.


4) Establishes the Board of Frontier Models (Board) within GovOps and makes the Board

responsible for directly supervising the FMD.

5) Requires the developer of a covered model to submit an annual certification to the FMD

certifying the developer’s compliance with the bill’s requirements and providing FMD with
specified information about its covered model. Beginning January 1, 2028, a developer must
also submit its most recent audit report to FMD as part of this annual certification.

6) Requires the developer of a covered model to report each AI safety incident affecting the

covered model to the FMD, as specified.

7) Requires the FMD, by January 1, 2027, and annually thereafter, to issue regulations to update


-----

Page 2

developments and applies to AI models that pose the greatest risk of causing or enabling
critical harms.

8) Requires a person who operates a computing cluster to implement written policies and

procedures when a customer utilizes computer resources sufficient to train a covered model.


a) Authorizes the Attorney General to bring a civil action to enforce any provision of the bill,

and authorizes the Labor Commissioner to enforce any provision that would constitute a
violation of the Labor Code, and specifies applicable penalties.

9) Provides whistleblower protections under the California Whistleblower Protection Act for

employees of the FMD and employees of developers of covered models.

10) Requires the California Department of Technology (CDT) to create a public cloud computing

cluster known as CalCompute to study the safe and secure deployment of large-scale AI
models and fostering equitable innovation.

**FISCAL EFFECT**


1) Costs (General Fund) to GovOps to establish and operate the FMD and the Board. GovOps

reports ongoing annual costs between $5 million and $10 million.

2) Costs (General Fund) to CDT to establish and operate CalCompute. In the first year of

implementation, CDT reports costs of $508,000 for two temporary positions, $6 million for
external consultants, and $1 million for GenAI talent practices to train state employees. In
subsequent years, CDT anticipates costs of $3.7 million for 16 additional positions, $3
million for external consultants, and $700,000 for GenAI talent practices. The bill authorizes
CDT to receive private donations, grants, and local funds to support CalCompute, which may
offset General Fund costs by an unknown amount.

CDT notes it is challenging to hire GenAI talent in the public sector since salaries are often
two or three times higher in the private sector, and private sector employers can offer
bonuses, equity, and stock options. As a result, CDT reports, its fiscal estimates for this bill
include significant funding for external consultants, and actual costs for consultants may be
higher if the state does not provide sufficient funding for training of state employees.
GovOps will likely also face these challenges and additional cost pressures.


3) Costs (General Fund, Labor and Enforcement Compliance Fund) of an unknown but

potentially significant amount to the Department of Justice (DOJ) and Labor Commissioner
to enforce violations of the bill. Costs to DOJ may be in the high hundreds of thousands to
low millions of dollars annually. Actual costs will depend on the number of violations, the
number of actions filed, and the amount of workload associated with each action.

4) Costs (Trial Court Trust Fund, General Fund) to the courts to adjudicate violations of the bill

and whistleblower cases. Actual costs will depend on the number of violations, the number
of actions filed, and the amount of court time needed to resolve each case. Although courts
are not funded on the basis of workload, increased pressure on the Trial Court Trust Fund
may create a need for increased funding for courts from the General Fund. The fiscal year
2024-25 budget provides $37.3 million ongoing General Fund to backfill declining revenue
to the Trial Court Trust Fund


-----

Page 3


5) Minor and absorbable costs to Judicial Council to review and process the model jury

instructions recommended by the FMD, as required by the bill.

According to the Legislative Analyst’s Office, the General Fund faces a structural deficit in the
tens of billions of dollars over the next several fiscal years.

**COMMENTS:**


1) **Purpose. This bill is sponsored by the Center for AI Safety Action Fund, Economic Security**

California Action, and Encode Justice. According to the author:

Large-scale artificial intelligence has the potential to produce an
incredible range of benefits for Californians and our economy—from
advances in medicine and climate science to improved wildfire
forecasting and clean power development. It also gives us an
opportunity to apply hard lessons learned over the last decade, as
we’ve seen the consequences of allowing the unchecked growth of
new technology without evaluating, understanding, or mitigating the
risks. SB 1047 does just that, by developing responsible, appropriate
guardrails around development of the largest, most powerful AI
systems, to ensure they are used to improve Californians’ lives,
without compromising safety or security.

2) **Background. As the development of AI models has progressed, scholars, policymakers,**

philosophers, and others have raised concern about the lack of regulation for the most
powerful emerging AI models. As discussed thoroughly in the policy committee analyses of
this bill, many people inside and outside the industry have encouraged governments to put an
anticipatory regulatory framework in place now, despite the considerable uncertainty about
the future of AI modeling, to mitigate the risk of future harm resulting from AI models.

This bill seeks to fill this void in California by implementing risk assessment, safety,
analysis, cybersecurity, and testing requirements with which developers of the most powerful
AI models must comply. Under the bill, the FMD and its governing Board within GovOps
are responsible for monitoring compliance with the bill’s requirements, among many other
duties outlined in the bill. The Attorney General may file a civil action to enforce any
violation of the bill’s provisions, and the Labor Commissioner may enforce any provision of
the bill that is also a violation of the Labor Code. The bill contains an initial definition of a
“covered model” that determines which models must comply with the bill’s requirements in
the short term. The bill requires the FMD to issue regulations by January 2027 to provide an
updated definition of “covered model,” and requires the FMD to update the definition
annually thereafter to ensure it captures the most powerful AI models that carry the greatest
risk of inflicting harm.

This bill also establishes whistleblower protections for employees of the FMD and
employees of developers with covered models, with the goal of protecting employees who
report violations and other unsafe practices of covered developers. Finally, the bill
establishes CalCompute, a public computing cluster, within CDT.


-----

