Dear Sakib Mostafa ,

Interpretability enables transparent AI models to be readily understood by users of all experience levels. Explainable AI applied to black box models means that data scientists and technical developers can provide an explanation as to why models behave the way they do -- and can pass the interpretation down to users.

Explainable AI (XAI) is gaining a lot of attention these day, and there is a lot of talk about model interpretability.

https://kaspergroesludvigsen.medium.com/explainable-ai-vs-interpretable-ai-b6df8e57ac70

Regards,

Shafagat

Hi Sakib Mafat

I like this question because I too find a lot of people do use the terms interchangeable.

So my understanding is the following.

Interpretable AI are AI models that human can understand why the results are the way they are. But this does not mean that they are always explainable. Explainable AI is understanding the how the results are inferenced or predicted. Alot of deep learning models are black-boxed (BB) are not explainable, although there are methods to find some explainability through use of libraries like SHAP or LIME that produces inference models on top of BB models for some explainability. There a lot of Data mining models that are good at explainability, giving results of probabilities or statistic for inference explaining the how. For example a decision trees that can explain the node split.

Transparent AI is about the documentation of the model, which includes aspects of the data used for creating the model and reporting model robustness metrics. Hence there are framework and checklist like the 2015 TRIPOD were created to aid reporting models.

Explainability and interpretability terms are often used interchangeably. So let's have more details on these:

Explainability methods summarise the explanations of AI or machine learning models' behaviour and describe the system's internal reasoning and dynamics.[1] "The more explainable a model, the deeper the understanding that humans achieve in terms of the internal procedures that take place while the model is training or making decisions". [2]

Interpretability is often described as the degree to which a person can comprehend the reason for an ML model's decision.[3] One popular definition of interoperability, is the one of Doshi-Velez and Kim, defined as “the ability to explain or to present in understandable terms to a human”[4]

Transparent AI is about detailed reporting and documentation of model development and validations processes e.g. the TRIPOD statement [5].

References:

1. Gilpin, L.H., et al. Explaining explanations: An overview of interpretability of machine learning. in 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA). 2018. IEEE.

2. Linardatos, P., V. Papastefanopoulos, and S. Kotsiantis, Explainable AI: A Review of Machine Learning Interpretability Methods. Entropy, 2021. 23(1): p. 18.

3. Miller, T., Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 2019. 267: p. 1-38.

4. Doshi-Velez, F. and B. Kim, Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608, 2017.

5. Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD). Circulation. 2015;131(2):211-219. doi:10.1161/CIRCULATIONAHA.114.014508.

Some useful and finer distinctions in the usage of these terms, are making their way into broader discourse.

For example, I have found Aparna Dhinakaran's descriptiom of "Explainability" into Global, Cohort and Local Explainability useful:

https://towardsdatascience.com/a-look-into-global-cohort-and-local-model-explainability-973bd449969f

The three terms are often used interchangably. But one can distinguish between AI that is directly interpretable or transparent (without additional methods) and explainable AI that requires additional techniques to make sense of.

Explainable AI vs Interpretable AI

---------------------------------------------------

If humans can directly make sense of a machine’s reasoning and

actions without additional explanations, we speak of interpretable AI [1].

If humans need explanations to understand AI, we speak of explainable AI [1]. Deep learning is generally non-interpretable due its complexity but can be explained with the help of a number techniques that create explanations a human can make sense of, e.g. by visualizing concepts of a neural network [3].

Interpretable AI vs Transparent AI

----------------------------------------------------

Transparent AI is seen as AI that can be directly and easily understood by humans without additional work. Mostly, these are simple models like decision trees. So transparent AI can be said to be interpretable. From a technical perspective also deep learning is transparent - you know all computation the model performs, but humans cannot make sense of the millions of numbers that are being processed.

As an alternative one might also look at the original meaning of the words:

To “explain” means “to make known, to make plain or understandable”. The related term “interpret” can be defined as “to explain or tell the meaning of”.[2] So interpret means "I tell you what it is about". To explain means "I process it (using some method), so that it becomes understandable".

PS: I would disagree that among researchers AI is called "transparent" if it is well-documented.

[1] Meske, C., Bunde, E., Schneider, J., & Gersch, M. (2022). Explainable artificial intelligence: objectives, stakeholders, and future research opportunities. Information Systems Management, 39(1), 53-63.

[2] Schneider, J., & Handali, J. (2019). Personalized explanation in machine learning: A conceptualization. arXiv preprint arXiv:1901.00770. (Published in European Conference on Information Systems)

[3] Schneider, J., & Vlachos, M. (2022). Explaining classifiers by constructing familiar concepts. Machine Learning, 1-34.

Article Explainable Artificial Intelligence: Understanding, Visualiz...

https://ieeexplore.ieee.org/abstract/document/8466590

Those terms are bounded to the fact people are confused about what machine learning is. Machine learning is just a subset of AI, beyond ML there's also for instance symbolic AI, ergo rules, and then the knowledge is already expressed and it's transparent. However for data science it's a bit tricky. If from the output of a ML model rules can be elicited, then one can say there's interpretability or explainability for human to comprehend what the algorithm has done.

I think these are much-debated concepts in deep learning. I really appreciate everyone sharing resources and clarifying the terms from their perspectives. I also think there is a lack of proper terminology in this field, and researchers using the terms interchangeably isn't helping.

Interpretive portability allows transparent AI models to be easily understood by users of all levels of experience. Interpretable AI applied to black-box models means that data scientists and technical developers can provide an explanation for why the models behave the way they do, and they can pass the explanation on to users.

Explainable AI, also known as XAI, refers to the ability of an AI system to provide a human-understandable explanation of its decision-making process. This means that the system is able to provide a rational justification for why it made a particular decision, in a way that is understandable to a human user.

Interpretable AI, on the other hand, refers to the ability of an AI system to provide a human-understandable representation of its internal workings. This means that the system is able to provide information about how it processes data and makes decisions, in a way that is understandable to a human user.

Transparent AI, also known as accountable AI, refers to the ability of an AI system to provide information about its decision-making process in a way that allows for independent verification and accountability. This means that the system is able to provide sufficient information about its decisions and how they were made, in a way that allows for external auditing and evaluation.

Overall, explainable AI, interpretable AI, and transparent AI all focus on improving the human-understandability of AI systems. However, they differ in the specific type of information that they provide and the purposes for which that information is used.

Sakib Mostafa

Explainable AI, Interpretable AI, and Transparent AI are related but distinct concepts that aim to improve the human-understandability and trustworthiness of AI systems. They can be defined as follows:

- Explainable AI (XAI) is the ability of an AI system to provide a human-understandable explanation of its decision-making process. This means that the system can provide a rational justification for why it made a particular decision, in a way that is understandable to a human user¹².

- Interpretable AI (IAI) is the ability of an AI system to provide a human-understandable representation of its internal workings. This means that the system can provide information about how it processes data and makes decisions, in a way that is understandable to a human user¹².

- Transparent AI (TAI) is the ability of an AI system to provide information about its decision-making process in a way that allows for independent verification and accountability. This means that the system can provide sufficient information about its decisions and how they were made, in a way that allows for external auditing and evaluation¹³.

The main difference between these concepts is the type and level of information that they provide and the purposes for which that information is used. For example:

- XAI focuses on providing explanations for specific decisions or outcomes, such as why a certain image was classified as a cat or why a certain loan application was rejected. XAI can help users understand the rationale and logic behind the system's decisions, as well as identify potential errors or biases¹².

- IAI focuses on providing insights into the general functioning and behavior of the system, such as what features or variables are important, how they are related, or what rules or patterns are learned. IAI can help users understand the overall structure and dynamics of the system, as well as modify or improve its performance¹².

- TAI focuses on providing evidence and documentation for the system's decisions and actions, such as what data was used, what methods were applied, or what criteria were followed. TAI can help users verify and validate the system's decisions, as well as ensure its compliance and accountability¹³.

These concepts are not mutually exclusive, but rather complementary and interrelated. For example, a transparent AI system can enable explainable and interpretable AI by providing access to relevant information. An interpretable AI system can facilitate explainable AI by providing intuitive and meaningful representations. An explainable AI system can support transparent AI by providing clear and concise explanations. Therefore, a desirable goal is to achieve a combination of these concepts in an AI system, depending on the context and needs of the users.

(1) What is the difference between Explainable AI, Interpretable AI, and .... https://www.researchgate.net/post/What_is_the_difference_between_Explainable_AI_Interpretable_AI_and_Transparent_AI.

(2) Explainable vs Interpretable AI | MLearning.ai - Medium. https://medium.com/mlearning-ai/explainable-vs-interpretable-ai-92a311fa410c.

(3) Everything You Want to Know About Transparent and Explainable AI. https://www.smartkarrot.com/resources/blog/transparent-explainable-ai/.