The Ethics of Responsible Innovation: Why Transparency Matters

In today’s world, data-driven systems make important decisions in industries. While these systems can provide many benefits, they can also foster mistrust by obfuscating decision-making pathways. Therefore, data transparency in the system is crucial for responsible innovation.

Transparency requires clear, explainable communication. Since transparency helps people understand how decisions are made, organizations should communicate the purposes and results of models, identify potential biases, and promote user trust. In addition to promoting trust, data transparency in the system may also become legally required. This legal requirement means that organizations must be able to communicate what, why and how their data management systems work.

Transparency in practice

Credit lending algorithms need to be able to explain to customers why their application was accepted or rejected. Failure to provide a reason for decision harms vulnerable populations and can lead to discriminatory outcomes.

Transparent credit lending practices help organizations spot potential problems and fix problems quickly.

Also, physicians use data-driven systems to inform clinical decisions and need to understand why the system is recommending specific treatments for their patients. Through transparency in the healthcare process, doctors can provide their patients with the highest standards of care and patients can make truly informed decisions about their well-being.

Although the context for these two data-driven systems differs significantly, the methods used to achieve transparency are similar.

Read on to learn why transparency is a critical principle of responsible innovation and how organizations can achieve it.

How organizations can achieve transparency

Organizations can use strategies such as model cards, data pipelines, model definition, and natural language processing (NLP) to achieve transparency. These strategies help users understand which use cases are appropriate for the model and which are more worrisome. Understanding the source and context of the data used to train the model helps you understand any constraints built into the system and any assumptions introduced during the data chain. Creating transparency mechanisms also provides users with an understanding of possible negative impacts of decisions made by the system.

To make your decisions transparent, consider asking the following questions.

• Can the responses from a data-driven tool be interpreted and explained by expert people within the organization?
• Will it be clear that people are interacting with the data management system?

• Which testing meets the expectations of auditing standards [FAT-AI, FEAT, ISO, etc.]?

Know the source and understand the data line

A data pipeline tracks data as it moves through an organization’s data ecosystem and is essential to achieving transparency. A data pipeline provides a transparent view of how an organization stores, processes and analyzes data. Data pipeline also helps organizations understand the suitability of their system for different uses. Organizations can better understand which data assets are relevant and appropriate for different uses by documenting inputs, transformations, and outputs. For example, an organization can use data pipeline to predict which data source is best suited to predict customer demand, allowing them to optimize data collection and storage processes.

By providing this transparent view of how data is stored, processed and analyzed, organizations can assess the potential impact of data management systems.

Why model explainability is vital to transparency

Another important part of transparency is the explainability of the model or the need to disclose the basis of the systems and solutions. Model Interpretability helps users interpret the results of AI models using techniques such as Partial Dependence (PD) plots, Individual Conditional Expectation (ICE) plots, Local Interpretable Model Agnostic Explanation (LIME) and HyperSHAP. These (and other) techniques let users know how a decision was made in the system.

An explanation helps us understand the reason for the decision made by the model. For example, if you ever use a navigation app while driving to a restaurant, the model’s explanation would allow the app to explain why it recommended a route on the freeway instead of taking another route on surface streets. Or what if a simple image classification algorithm classifies a husky as a wolf? The model’s explanation helps us understand that the model has learned that the snow in the background is the same as the wolf in the picture.

Narrating complex insights using NLP

The complexity of models can make it difficult for participants to understand how decisions are made and what factors are considered.

To address this, organizations can use NLP as a powerful tool in explaining patterns. NLP can generate text-based explanations for data-driven decisions in decision-making. Organizations should consider developing data model descriptions written in simple business language that can be understood by any audience through natural language learning. This means that stakeholders and members of the public can interpret and understand the results of these systems without having to work in data science or computer programming.

For example, imagine a healthcare organization using a data management system to determine which treatments to recommend to patients with a particular condition. NLP can generate textual explanations that reflect factors considered in the recommendation, such as the patient’s medical history, the latest research on the condition, and even the success rate of various treatments. This will allow patients to understand why a particular treatment has been recommended and be confident in their decision. By making this decision-making process more visible, organizations can identify any biases or assumptions that affect outcomes and work to address them.

Ultimately, when it comes to developing data-driven systems, transparency is critical to responsible innovation. And to promote transparency, communication is key. Organizations can ensure that their data-driven decision systems are transparent by implementing measures such as model cards, model explainability, data pipeline, and natural language insight. By doing so, organizations can build trust in these systems and ensure they are used responsibly and efficiently.

Read more stories about justice and accountability from SAS bloggers

Vrushali Sawant and Christy Boyd contributed to this article