Accountability and Governance Are the Foundations of Trust in AI

On June 9, the Paladin Global Institute released “The Governance Layer: A Primer for Tech and Cyber Policy,” the second installment in its work on the AI Tech Stack. The primer, produced under the leadership of Institute President Kemba Walden and Senior Director Devin Lynch, proposes a governance structure built from protocols, principles, policy, and law, and it offers recommendations for both the internal governance systems organizations build and the external institutions that oversee them. This document arrives at the right moment and asks the right question. The question is not whether AI should be governed. The question is how governance can be structured so that individuals and enterprises can trust services that include AI as a component.

The answer I have advocated for many years is accountability. In 2015, Paula Bruening and I released “Rethinking Privacy: Fair Information Practice Principles Reinterpreted.” We argued there that the accountability principle shifts the burden of policing the data marketplace away from the individual and onto the organization. Under that model, the accountability principle places responsibility for governance programs on organizations, requiring them to assess the risks their data practices pose to individuals and to society, mitigate those risks, and stand ready to demonstrate the effectiveness of their internal processes when a regulator asks. We designed that argument for personal data. It maps directly onto AI, where the inferences are more powerful and the consequences of failure are harder for any individual to detect or correct on their own.

Notice and choice will not carry this weight. Weak notice and choice regimes place too much burden on individuals to protect themselves from those who monetize and weaponize data. The same is true of AI, and with greater force. No consumer can read a model card and evaluate whether a foundation model was trained responsibly. No small business can audit the supply chain behind the AI features embedded in its accounting software. Trust must instead rest on demonstrable governance, checked by institutions with the authority and resources to verify the work. Until people can rely on regulators to enforce robust protections, we will be left with an environment that discourages trust in the innovative uses of data and AI that could help society address problems like the delivery of effective health care.

A second lesson from my prior work runs through the Paladin paper. I have long rejected the proposition that we must decrease privacy if we want to increase security. It takes data to protect data. Defending devices, networks, and personal information requires analysis of the information flowing through those systems, and cloud-based threat intelligence improves when it draws on the collective experience of many organizations. AI governance presents the same dynamic. Incident analysis, runtime oversight, and continuous monitoring all require data about how systems behave once deployed. Privacy, security, and safety reinforce one another when governance is designed with all three in mind, and the paper’s call to move from static rules to dynamic oversight rests on exactly this insight. 

The data broker industry shows what happens when accountability is absent. In 2019, I wrote that data brokers had built an unregulated secondary market for buying and selling personal data, with profiles used to target victims of domestic violence, police, judges, and victims of crime, while foreign adversaries worked with brokers to manipulate citizens and destabilize the country. The Federal Trade Commission, despite its best intentions, lacked the legal tools and the resources to respond. If AI-enabled services are allowed to develop in the same vacuum, the harms will compound because AI systems can derive sensitive inferences about people at a scale the brokers never reached and may be powered in part by training data coming from data brokers. The governance layer Paladin describes is how we avoid repeating that mistake.

The paper’s specific recommendations deserve support. Congress should legislate for understanding first and then regulate with evidence, mapping AI systems and risks before drawing jurisdictional boundaries or preempting state law. The Center for AI Standards and Innovation at the Department of Commerce should be codified and funded as the federal government’s central technical authority on AI systems, with an incident review function modeled on the National Transportation Safety Board. Aviation became trustworthy not because aircraft never fail, but because every failure produces public findings that improve the entire system. AI deserves the same institutional learning. Oversight should align to the AI tech stack and the model lifecycle, from data and models through infrastructure and applications, with testing before deployment and monitoring after deployment. 

A national data security and privacy baseline remains the unfinished business beneath all of this. Experts from industry, the advocacy community, and think tanks have agreed that the time for federal privacy legislation arrived long ago, and the FTC needs more technical and legal staff along with stronger authority. Congress appears unable to deliver. AI raises the stakes. Some of the data that trains and flows through AI systems is the same personal data a federal privacy law would protect. In the absence of U.S. federal leadership, the task of protecting individuals falls to states, municipalities and public policy stakeholders to recommend harmonized approaches that states can move toward. Those harmonized approaches should include industry best practices and international standards. In Rethinking Privacy, we framed sound information practices as bridges across regions, cultures, technologies, platforms, and individuals. Interoperable AI standards, aligned with allies and especially with the European Union, should be a central focus.

Professor David Hoffman, Deep Tech Program Coordinator Merritt Cahoon, and Research Assistant Jojo Hong were reviewers on The AI Tech Stack: A Primer for Tech and Cyber Policy. Paladin Capital Group is a founding partner, sponsor, and funder of Cybersecurity Leadership Program and CISO Executive Certificate Program. David Hoffman serves on the advisory board.