State bank examiners get a playbook for inspecting AI

• Key insight: The CSBS framework covers generative and agentic AI, the technology the Federal Reserve, OCC and FDIC placed outside the scope of their revised model risk guidance in April.
• What's at stake: State agencies supervise 3,355 of the country's 4,233 FDIC-insured banks and savings institutions, so most U.S. banks may face the new questions at their next state exam.
• Forward look: Each state agency decides on its own whether to fold the framework into its supervisory program, and CSBS publishes no count of which ones have.

Overview bullets generated by AI with editorial review.

On Wednesday, a coalition of state regulators released a procedure examiners may use to scrutinize the use of artificial intelligence inside state-chartered banks.

The Conference of State Bank Supervisors, which represents financial regulators in all 50 states, the District of Columbia and four U.S. territories, published what it calls the AI Supervisory Framework. It covers state-chartered banks and the nonbank financial companies that states license.

It includes five documents: a core guide for examiners, a 28-page work program setting out the procedures examiners follow, a supplement covering nonbanks, a worksheet for rating individual uses of AI by risk and a one-page list of the source materials CSBS used to develop the framework.

For bankers, the new framework is a preview of their next exam. CSBS published it so institutions can assess their own AI programs and "prepare for examinations," according to a Wednesday press release announcing the framework.

State regulators supervise 3,355 (79%) of the country's 4,233 banks and savings institutions insured by the Federal Deposit Insurance Corp., according to FDIC records as of June 30.

The new framework from CSBS comes a few months after the Federal Reserve, the Office of the Comptroller of the Currency and the FDIC replaced their 15-year-old model risk guidance, the rules for how banks build, test and oversee the models behind lending, pricing and risk decisions.

That federal guidance, issued in April, had a giant hole in it.

"Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance," the agencies wrote in a footnote. (Agentic AI means systems that take actions on their own, with little human direction.)

On Wednesday, CSBS addressed that hole directly.

Because some existing model risk resources "may not fully address generative AI, agentic AI, or similar capabilities," examiners may use the CSBS framework to identify "AI-specific considerations that may warrant additional review," according to its core examiner guide.

The framework is strictly a discretionary tool; each state agency decides how much of it to adopt, and it "does not create new legal obligations or supervisory requirements," according to the core guide.

"While any new technology can present risks, the CSBS AI Supervisory Framework provides a principles-based approach to governance intended to help financial institutions explore and implement AI with additional confidence," said Brandon Milhorn, the CSBS president and CEO, in the press release.

What an examiner asks first

The core guide lists eight questions for an examiner to ask any bank:

1. Does the bank use AI at all?
2. Has it identified where?
3. Does AI touch any customers or shape decisions?
4. Does the bank rely on vendors or outside platforms for AI?
5. Has the bank looked for AI embedded in vendor products it already uses?
6. Does the bank use generative AI?
7. Does the bank sort its AI uses by risk?
8. Does customer or other sensitive information pass through any AI?

An unclear or negative answer to the first question "may warrant confirmation against the institution's vendor inventory, software inventory, approved tools, and recent product or platform changes," the core guide said.
The extra checking matters because AI use "may be missed if it is embedded in software or treated as a feature rather than as a distinct system or use case," according to the work program.

The core guide says examiners may request documents such as AI policies, inventories, board reporting and samples of AI-generated material that customers see (including "chatbot transcripts").

Examiners may also ask for contract terms covering "the use, retention, sharing, or training of AI systems on customer, consumer, confidential, or institutional data," per the guide.

The questions about AI that acts on its own

Where an institution runs agentic AI, CSBS suggests examiners review how the bank defines the actions the agentic system may take, the checkpoints where a human intervenes in the system's actions and decisions, logging, reversibility and the ability to restrict or halt the system, according to the work program.

That last item (the ability to halt a system) is one of two areas where bankers report the least readiness.

In a survey earlier this year conducted by software and information services firm Wolters Kluwer, nearly three quarters (72%) of the 230 bankers who responded named either model kill-switch protocols or regulatory reporting of AI failures as the risk area for which they were least prepared.

A worksheet banks fill in themselves

CSBS calls its worksheet for scoring the risks of AI "an optional industry tool" on the framework's landing page. Banks complete one worksheet for each of their AI uses and retain it "as support for examination requests," according to the form.

For each AI use case a bank has, the worksheet assigns an overall risk tier between 1 and 3. Four factors contribute to the scoring: how much the AI use case affects consumers, how much human oversight it gets, how much harm an error or outage could cause and how sensitive the data is.

Tier 3, which is the highest-risk one, involves "direct consumer outcomes, sensitive personal data, limited human review, significant operational reliance, or material potential harm from errors or outages," the worksheet said.

Sensitive data includes protected-class information and biometric data as well as "data elements that could serve as proxies for protected characteristics," according to the worksheet.

Controls accumulate as risk rises. Tier 1 asks for an inventory entry naming the use case's business owner and a written policy on acceptable AI use. Tier 2 calls for documenting how the AI reaches any outputs a customer sees.

A Tier 3 use case carries all of that plus independent model validation by a qualified party and incident response procedures written specifically for AI.

The same questions, whatever the bank's size

CSBS did not assign dollar figures, error rates, consumer counts or any other absolute measure to any of the risk tiers. The organization instead wants everything to scale to an institution's "size, complexity, risk profile, and use of artificial intelligence," according to the core guide.

The same eight questions, document request list and worksheet apply to every institution the states charter. Meanwhile, the federal model risk guidance from April said it is "expected to be most relevant to banking organizations with over $30 billion in total assets."

Among state-chartered institutions, roughly 99% hold $30 billion or less in assets, according to an American Banker analysis of midyear call reports.

Most state-chartered banks, in other words, sit below the line the federal guidance sets.

By contrast, the CSBS framework released Wednesday applies to state-chartered institutions of all sizes, from Roxboro Savings Bank, a two-branch North Carolina thrift with $360.7 million, to Goldman Sachs Bank USA, which holds $758.8 billion in assets.