What is ‘ethical AI’ and how can companies achieve it?

Sustainability & Regulation
04/06/2023 Article

The rush to deploy powerful new generative AI technologies, such as ChatGPT, has raised alarms about potential harm and misuse. The law’s glacial response to such threats has prompted demands that the companies developing these technologies implement AI “ethically.”

But what, exactly, does that mean?

The straightforward answer would be to align a business’s operations with one or more of the dozens of sets of AI ethics principles that governments, multistakeholder groups and academics have produced. But that is easier said than done.

We and our colleagues spent two years interviewing and surveying AI ethics professionals across a range of sectors to try to understand how they sought to achieve ethical AI – and what they might be missing. We learned that pursuing AI ethics on the ground is less about mapping ethical principles onto corporate actions than it is about implementing management structures and processes that enable an organization to spot and mitigate threats.

This is likely to be disappointing news for organizations looking for unambiguous guidance that avoids gray areas, and for consumers hoping for clear and protective standards. But it points to a better understanding of how companies can pursue ethical AI. 


profile picture of Dennis Hirsch

Dennis Hirsch

The Ohio State University

Professor of Law; Director, Program on Data and Governance

profile picture of Piers Norris Turner

Piers Norris Turner

The Ohio State University

Associate Professor & PPE Coordinator; Director, Center for Ethics and Human Values

Grappling with ethical uncertainties

Our study, which is the basis for a forthcoming book, centered on those responsible for managing AI ethics issues at major companies that use AI. From late 2017 to early 2019, we interviewed 23 such managers. Their titles ranged from privacy officer and privacy counsel to one that was new at the time but increasingly common today: data ethics officer. Our conversations with these AI ethics managers produced four main takeaways.

First, along with its many benefits, business use of AI poses substantial risks, and the companies know it. AI ethics managers expressed concerns about privacy, manipulation, bias, opacity, inequality and labor displacement. In one well-known example, Amazon developed an AI tool to sort résumés and trained it to find candidates similar to those it had hired in the past. Male dominance in the tech industry meant that most of Amazon’s employees were men. The tool accordingly learned to reject female candidates. Unable to fix the problem, Amazon ultimately had to scrap the project.

Generative AI raises additional worries about misinformation and hate speech at large scale and misappropriation of intellectual property.

Second, companies that pursue ethical AI do so largely for strategic reasons. They want to sustain trust among customers, business partners and employees. And they want to preempt, or prepare for, emerging regulations. The Facebook-Cambridge Analytica scandal, in which Cambridge Analytica used Facebook user data, shared without consent, to infer the users’ psychological types and target them with manipulative political ads, showed that the unethical use of advanced analytics can eviscerate a company’s reputation or even, as in the case of Cambridge Analytica itself, bring it down. The companies we spoke to wanted instead to be viewed as responsible stewards of people’s data.

The challenge that AI ethics managers faced was figuring out how best to achieve “ethical AI.” They looked first to AI ethics principles, particularly those rooted in bioethics or human rights principles, but found them insufficient. It was not just that there are many competing sets of principles. It was that justice, fairness, beneficence, autonomy and other such principles are contested and subject to interpretation and can conflict with one another.

This led to our third takeaway: Managers needed more than high-level AI principles to decide what to do in specific situations. One AI ethics manager described trying to translate human rights principles into a set of questions that developers could ask themselves to produce more ethical AI software systems. “We stopped after 34 pages of questions,” the manager said.

Fourth, professionals grappling with ethical uncertainties turned to organizational structures and procedures to arrive at judgments about what to do. Some of these were clearly inadequate. But others, while still largely in development, were more helpful, such as:
Hiring an AI ethics officer to build and oversee the program.
• Establishing an internal AI ethics committee to weigh and decide hard issues.
• Crafting data ethics checklists and requiring front-line data scientists to fill them out.
• Reaching out to academics, former regulators and advocates for alternative perspectives.
• Conducting algorithmic impact assessments of the type already in use in environmental and privacy governance.

Ethics as responsible decision-making

The key idea that emerged from our study is this: Companies seeking to use AI ethically should not expect to discover a simple set of principles that delivers correct answers from an all-knowing, God’s-eye perspective. Instead, they should focus on the very human task of trying to make responsible decisions in a world of finite understanding and changing circumstances, even if some decisions end up being imperfect.

In the absence of explicit legal requirements, companies, like individuals, can only do their best to make themselves aware of how AI affects people and the environment and to stay abreast of public concerns and the latest research and expert ideas. They can also seek input from a large and diverse set of stakeholders and seriously engage with high-level ethical principles.

This simple idea changes the conversation in important ways. It encourages AI ethics professionals to focus their energies less on identifying and applying AI principles – though they remain part of the story – and more on adopting decision-making structures and processes to ensure that they consider the impacts, viewpoints and public expectations that should inform their business decisions.

Ultimately, we believe laws and regulations will need to provide substantive benchmarks for organizations to aim for. But the structures and processes of responsible decision-making are a place to start and should, over time, help to build the knowledge needed to craft protective and workable substantive legal standards.

Indeed, the emerging law and policy of AI focuses on process. New York City passed a law requiring companies to audit their AI systems for harmful bias before using these systems to make hiring decisions. Members of Congress have introduced bills that would require businesses to conduct algorithmic impact assessments before using AI for lending, employment, insurance and other such consequential decisions. These laws emphasize processes that address in advance AI’s many threats.

Some of the developers of generative AI have taken a very different approach. Sam Altman, the CEO of OpenAI, initially explained that, in releasing ChatGPT to the public, the company sought to give the chatbot “enough exposure to the real world that you find some of the misuse cases you wouldn’t have thought of so that you can build better tools.” To us, that is not responsible AI. It is treating human beings as guinea pigs in a risky experiment.

Altman’s call at a May 2023 Senate hearing for government regulation of AI shows greater awareness of the problem. But we believe he goes too far in shifting to government the responsibilities that the developers of generative AI must also bear. Maintaining public trust, and avoiding harm to society, will require companies more fully to face up to their responsibilities.

Enter ChatGPT

That brings us to a new world of ChatGPT-powered trading algorithms and similar programs. They could take the problem of too many traders on the same side of a deal and make it even worse.

In general, humans, left to their own devices, will tend to make a diverse range of decisions. But if everyone’s deriving their decisions from a similar artificial intelligence, this can limit the diversity of opinion.

Consider an extreme, nonfinancial situation in which everyone depends on ChatGPT to decide on the best computer to buy. Consumers are already very prone to herding behavior, in which they tend to buy the same products and models. For example, reviews on Yelp, Amazon and so on motivate consumers to pick among a few top choices.

Since decisions made by the generative AI-powered chatbot are based on past training data, there would be a similarity in the decisions suggested by the chatbot. It is highly likely that ChatGPT would suggest the same brand and model to everyone. This might take herding to a whole new level and could lead to shortages in certain products and service as well as severe price spikes.

This becomes more problematic when the AI making the decisions is informed by biased and incorrect information. AI algorithms can reinforce existing biases when systems are trained on biased, old or limited data sets. And ChatGPT and similar tools have been criticized for making factual errors.

In addition, since market crashes are relatively rare, there isn’t much data on them. Since generative AIs depend on data training to learn, their lack of knowledge about them could make them more likely to happen.

For now, at least, it seems most banks won’t be allowing their employees to take advantage of ChatGPT and similar tools. Citigroup, Bank of America, Goldman Sachs and several other lenders have already banned their use on trading-room floors, citing privacy concerns.

But I strongly believe banks will eventually embrace generative AI, once they resolve concerns they have with it. The potential gains are too significant to pass up – and there’s a risk of being left behind by rivals.

But the risks to financial markets, the global economy and everyone are also great, so I hope they tread carefully.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The Conversation

Register to be kept informed of our next publications

Related news & insights

Digital Reinvention
27/09/2023 Video

What’s happening in the banking industry and where banks need to go in the future

Jim Marous presents the main findings of the report 'Innovation in Retail Banking 2023' by Qorus and Infosys Finacle. According...

SME Banking
26/09/2023 Interview

Unveiling the future of Agribanking: Desjardins

Sylvain Morel, Vice-President, Agriculture and Agri-Food Market Development and Business Relations at Desjardins, shares insights into the agribanking landscape in...

Sustainability & Regulation
26/09/2023 Video

How BPCE Group and NZBA will achieve the carbon neutrality pathway

Maxime Druais, Head of ESG at BPCE, outlines the objectives of the Net-Zero Banking Alliance (NZBA) and the various steps...

Sustainability & Regulation
25/09/2023 Interview

Unveiling efficient organizational models for CSR and sustainability: Mobilize Financial Services

Mallika Mathur Lheritier, Group Chief Transformation & Sustainability Officer at Mobilize Financial Services, offers an illuminating glimpse into their effective...

25/09/2023 Interview

Mercedes-Benz's integration of ChatGPT: Revolutionizing in-car experience and exploring new avenues

Magnus Östberg, Chief Software Officer at Mercedes-Benz AG, delves into the role of ChatGPT and the exciting possibilities it brings...

Sustainability & Regulation
25/09/2023 Video

How BNP Paribas built a climate stress testing platform to provide a relevant and transparent quantification of climate risk factors

Marc Irubetagoyena, Head of Stress Testing at BNP Paribas, has developed a climate stress testing platform with the objective of...

25/09/2023 News

KfW IPEX-Bank joins forces with Crédit Agricole CIB and La Banque Postale to strengthen NEoT Green Mobility's multicurrency financing structure

This collaboration has led to an increase in NGM's existing multicurrency financing structure to the equivalent of EUR 145 million,...

SME Banking
21/09/2023 Video

How Santander UK supports small businesses through third-party partnerships

Susan Davies, Head of Business Banking at Santander UK, presents Santander UK's Beyond Banking hub, an ecosystem of products and...