« SaaSのセキュリティは重要となりますが、アプリが多いので大変ですよね。。。 | Main | 米国上院の情報委員会が2016年大統領選におけるロシアの影響を調べた報告書(第5巻)を公開していますね。。。 »

2020.08.21

NISTはAIに自己説明を求める?(説明可能な人工知能の4原則)

こんにちは、丸山満彦です。

NISTが説明可能な人工知能の4原則のドラフトを公表し意見を募集していますね。。。

NIST

・2020.08.18 (News) NIST Asks A.I. to Explain Itself

Technical agency proposes four fundamental principles for judging how well AI decisions can be explained.

・[PDF] Draft NISTIR 8312 Four Principles of Explainable Artificial Intelligence 


Abstract

We introduce four principles for explainable artificial intelligence (AI) that comprise the fundamental properties for explainable AI systems. They were developed to encompass he multidisciplinary nature of explainable AI, including the fields of computer science, engineering, and psychology. Because one size fits all explanations do not exist, different users will require different types of explanations. We present five categories of explanation and summarize theories of explainable AI. We give an overview of the algorithms in the field that cover the major classes of explainable algorithms. As a baseline comparison, we assess how well explanations provided by people follow our four principles. This assessment provides insights to the challenges of designing explainable AI systems.


 

4つの原則とは

  1. Explanation: 
    AI systems should deliver accompanying evidence or reasons for all outputs.

  2. Meaningful: 
    Systems should provide explanations that are understandable to individual users.

  3. Explanation Accuracy: 
    The explanation should correctly reflect the system’s process for generating the output.

  4. Knowledge Limits: 
    The system only operates under conditions for which it was designed or when the system reaches a sufficient level of confidence in its output.

ざっというと

  1. 説明:
    AIシステムは、全ての出力に付随する証拠や理由を提供する必要がある。
  2. 有意味:
    システムは、個々の利用者が利用できる説明を提供する必要がある。

  3. 説明の正確性:
    説明は、出力を生成するためのシステムプロセスを正しく反映している。

  4. 知識の限界:
    システムは、それが設計された条件下でのみ又は、システムがその出力が十分に信頼できる状況になった場合にのみ動作する。

という感じでしょうかね。。。

目次です。


Table of Contents

1 Introduction

2 Four Principles of Explainable AI
 2.1 Explanation
 2.2 Meaningful
 2.3 Explanation Accuracy
 2.4 Knowledge Limits

3 Types of Explanations

4 Overview of principles in the literature

5 Overview of Explainable AI Algorithms
 5.1 Self-Explainable Models
 5.2 Global Explainable AI Algorithms
 5.3 Per-Decision Explainable AI Algorithms
 5.4 Adversarial Attacks on Explainability

6 Humans as a Comparison Group for Explainable AI
 6.1 Explanation
 6.2 Meaningful
 6.3 Explanation Accuracy
 6.4 Knowledge Limits

7 Discussion and Conclusions

References


 

1. Introduction

With recent advances in artificial intelligence (AI), AI systems have become components of high-stakes decision processes. The nature of these decisions has spurred a drive to create algorithms, methods, and techniques to accompany outputs from AI systems with expla nations. This drive is motivated in part by laws and regulations which state that decisions, including those from automated systems, provide information about the logic behind those decisions1 and the desire to create trustworthy AI [30, 76, 89].

Based on these calls for explainable systems [40], it can be assumed that the failure to articulate the rationale for an answer can affect the level of trust users will grant that system.

Suspicions that the system is biased or unfair can raise concerns about harm to oneself and to society [102]. This may slow societal acceptance and adoption of the technology, as members of the general public oftentimes place the burden of meeting societal goals on manufacturers and programmers themselves [27, 102]. Therefore, in terms of societal acceptance and trust, developers of AI systems may need to consider that multiple attributes of an AI system can influence public perception of the system.

Explainable AI is one of several properties that characterize trust in AI systems [83, 92].

Other properties include resiliency, reliability, bias, and accountability. Usually, these terms are not defined in isolation, but as a part or set of principles or pillars. The definitions vary by author, and they focus on the norms that society expects AI systems to follow. For this paper, we state four principles encompassing the core concepts of explainable AI. These are informed by research from the fields of computer science, engineering, and psychology.

In considering aspects across these fields, this report provides a set of contributions. First, we articulate the four principles of explainable AI. From a computer science perspective, we place existing explainable AI algorithms and systems into the context of these four prin ciples. From a psychological perspective, we investigate how well people’s explanations follow our four principles. This provides a baseline comparison for progress in explainable AI.

Although these principles may affect the methods in which algorithms operate to meet explainable AI goals, the focus of the concepts is not algorithmic methods or computations themselves. Rather, we outline a set of principles that organize and review existing work in explainable AI and guide future research directions for the field. These principles support the foundation of policy considerations, safety, acceptance by society, and other aspects of AI technology.

.....

7. Discussion and Conclusions

We introduced four principles to encapsulate the fundamental elements for explainable AI systems. The principles provide a framework with which to address different components of an explainable system. These four principles are that the system produce an explanation, that the explanation be meaningful to humans, that the explanation reflects the system’s processes accurately, and that the system expresses its knowledge limits. There are differ681 ent approaches and sophies for developing and evaluating explainable AI. Computer science approaches tackle the problem of explainable AI from a variety of computational and graphical techniques and perspectives, which may lead to promising breakthroughs. A blossoming field puts humans at the forefront when considering the effectiveness of AI ex685 planations and the human ors behind their effectiveness. Our four principles provide a multidisciplinary framework with which to explore this type of human-machine interaction.

The practical needs of the system will influence how these principles are addressed (or dismissed). With these needs in mind, the community will ultimately adapt and apply the four principles to capture a wide scope of applications. One example of adapting to meet practical requirements is illustrated by the trade-off between explanation detail and time constraints. These constraints highlight that certain scenarios require a brief, meaningful explanation to take priority over an accurate, detailed explanation. For example, emergency weather alerts need to be meaningful to the public but can lack an accurate explanation of how the system arrived at its conclusion. Other scenarios may require more detailed explanations but restrict meaningfulness to a specific user group; e.g., when auditing a model.

The focus of explainable AI has been to advance the capability of the systems to pro duce a quality explanation. Here, we addressed whether humans can meet the same set of principles we set forth for AI. We showed that humans demonstrate only limited ability to meet the principles outlined here. This provides a benchmark with which to compare AI systems. In reflection of societal expectations, recent regulations have imposed a degree of accountability on AI systems via the requirement for explainable AI [1]. As advances are made in explainable AI, we may find that certain parts of AI systems are better able to meet societal expectations and goals compared to humans. By understanding the explainability of both the AI system and the human in the human-machine collaboration, this opens the door to pursue implementations which incorporate the strengths of each, potentially improving explainability beyond the capability of either the human or AI system in isolation.  

In this paper, we focused on a limited set of human factors related to explainable decisions. Much is to be learned and studied regarding the interaction between humans and  explainable machines. Although beyond the scope of the current paper, in considering the  interface between AI and humans, understanding general principles that drive human reasoning and decision making may prove to be highly informative for the field of explainable  AI [23]. For humans, there are general tendencies for preferring simpler and more general  explanations [58]. However, as described earlier, there are individual differences in which  explanations are considered high quality. The context of the decision and the type of decision being made can influence this as well. Humans do not make decisions in isolation  of other factors [45]. Without conscious awareness, people incorporate irrelevant information into a variety of decisions such as first impressions, personality trait judgments,  and jury decisions [21, 29, 90, 91]. Even when provided identical information, the context, a person’s biases, and the way in which information is presented influences decisions  [4, 15, 17, 23, 36, 43, 68, 94]. Considering these human factors within the context of  explainable AI has only just begun.  

To succeed in explainable AI, the community needs to study the interface between humans and AI systems. Human-machine collaborations have shown to be highly effective  in terms of accuracy [67]. There may be similar breakthroughs for AI explainability in  human-machine collaborations. The principles defined here provide guidance and a philosophy for driving explainable AI toward a safer world by giving users a deeper understanding into a system’s output. Meaningful and accurate explanations empower users to apply this information to adapt their behavior and/or appeal decisions. For developers and auditors, explanations equips them with the ability to improve, maintain, and deploy systems as appropriate. Explainable AI contributes to the safe operation and trust of multiple facets of complex AI systems. The common framework and definitions under the four principles facilitate the evolution of explainable AI methods necessary for complex, real-world systems.


 

|

« SaaSのセキュリティは重要となりますが、アプリが多いので大変ですよね。。。 | Main | 米国上院の情報委員会が2016年大統領選におけるロシアの影響を調べた報告書(第5巻)を公開していますね。。。 »

Comments

Post a comment



(Not displayed with comment.)




« SaaSのセキュリティは重要となりますが、アプリが多いので大変ですよね。。。 | Main | 米国上院の情報委員会が2016年大統領選におけるロシアの影響を調べた報告書(第5巻)を公開していますね。。。 »