第1页 / 共57页
第2页 / 共57页
第3页 / 共57页
第4页 / 共57页
第5页 / 共57页
第6页 / 共57页
第7页 / 共57页
第8页 / 共57页
机器推理可解释性（综述论文）.pdf
1 Introduction
1.1 Motivation
1.2 Contributions
1.3 Omissions
2 Explainability
2.1 Purpose of Explanations
2.2 High-level Questions
2.3 Categorization for Explanations
2.3.1 Attributive Explanations
2.3.2 Contrastive Explanations
2.3.3 Actionable Explanations
3 MR Branches and Explanations
3.1 Classical Logic-Based Reasoning
3.1.1 Axiom Pinpointing
3.1.2 Constraint Programming (CP)
3.1.3 Satisfiability (SAT) and Beyond
3.1.4 Automated Theorem Proving (ATP) and Proof Assistants
3.1.5 Abduction
3.2 Non-classical Logic-Based Reasoning (Logic Programming)
3.2.1 Abductive Logic Programming (ALP)
3.2.2 Inductive Logic Programming (ILP)
3.2.3 Answer Set Programming (ASP)
3.3 Argumentation
3.4 Decision Theory
3.5 Planning
3.6 Multi-Agent Systems
3.7 Reinforcement Learning
3.7.1 Constrained RL
3.7.2 Multi-Agent RL (MARL)
3.8 Causal Approaches
4 Remarks
5 Conclusions
0
2
0
2
p
e
S
1
]
I
A
.
s
c
[
1
v
8
1
4
0
0
.
9
0
0
2
:
v
i
X
r
a
Machine Reasoning Explainability
Kristijonas ˇCyras∗, Ramamurthy Badrinath, Swarup Kumar Mohalik,
Anusha Mujumdar, Alexandros Nikou, Alessandro Previti,
Vaishnavi Sundararajan, Aneta Vulgarakis Feljan
Ericsson Research
September 2, 2020
Abstract
As a field of AI, Machine Reasoning (MR) uses largely symbolic means to formalize and em-
ulate abstract reasoning. Studies in early MR have notably started inquiries into Explainable AI
(XAI) – arguably one of the biggest concerns today for the AI community. Work on explainable
MR as well as on MR approaches to explainability in other areas of AI has continued ever since. It
is especially potent in modern MR branches, such as argumentation, constraint and logic program-
ming, planning. We hereby aim to provide a selective overview of MR explainability techniques
and studies in hopes that insights from this long track of research will complement well the current
XAI landscape. This document reports our work in-progress on MR explainability.
1
Introduction
Machine Reasoning (MR) is a field of AI that complements the field of Machine Learning (ML)
by aiming to computationally mimic abstract thinking. This is done by way of uniting known (yet
possibly incomplete) information with background knowledge and making inferences regarding un-
known or uncertain information. MR has outgrown Knowledge Representation and Reasoning (KR,
see e.g. [27]) and now encompasses various symbolic and hybrid AI approaches to automated rea-
soning. Central to MR are two components: a knowledge base (see e.g. [61]; common in Axiom
Pinpointing, Automated Theorem Proving (ATP) and Proof Assistants, Non-classical Logic-Based
Reasoning (Logic Programming), Argumentation) or a model of the problem (see e.g. [89]; com-
mon in Constraint Programming (CP), Planning, Decision Theory, Reinforcement Learning), which
∗Corresponding author. Email: kristijonas.cyras@ericsson.com, ORCiD: 0000-0002-4353-8121
1
K. ˇCyras et al.
Report on Explainability in Machine Reasoning
formally represents knowledge and relationships among problem components in symbolic, machine-
processable form; and a general-purpose inference engine or solving mechanism, which allows to
manipulate those symbols and perform semantic reasoning.1
The field of Explainable AI (XAI, see e.g. [1, 13, 20, 54, 130, 146, 149, 155, 169, 177, 196])
encompasses endeavors to make AI systems intelligible to their users, be they humans or machines.
XAI comprises research in AI as well as interdisciplinary research at the intersections of AI and
subjects ranging from Human-Computer Interaction (HCI), see e.g. [149], to social sciences, see e.g.
[33, 144]. Explainability of AI is often seen as a crucial driver for the real-world deployment of
trustworthy modern AI systems.
According to e.g. Hansen and Rieger in [100], explainability was one of the main distinctions
between the 1st (dominated by KR and rule-based systems) and the 2nd (expert systems and statis-
tical learning) waves of AI, with expert systems addressing the problems of explainability and ML
approaches treated as black boxes. With the ongoing 3rd wave of AI, ML explainability has received
a great surge of interest [13, 54, 149]. By contrast therefore, it seems that a revived interest in MR
explainability is only just picking up pace (e.g. ECAI 2020 Spotlight tutorial on Argumentative Ex-
planations in AI2 and KR 2020 Workshop on Explainable Logic-Based Knowledge Representation3).
However, explainability in MR dates over four decades, see e.g. [100, 110, 151, 155, 196]. Explain-
ability in MR can be roughly outlined thus.
1st generation expert systems provide only so-called (reasoning) trace explanations, showing in-
ference rules that led to a decision. A major problem with trace explanations is the lack of “informa-
tion with respect to the system’s general goals and resolution strategy”[151, p. 174]. 2nd generation
expert systems instead provide so-called strategic explanations, revealing “why information is gath-
ered in a certain order, why one knowledge piece is invoked before others and how reasoning steps
contribute to high-level goals”[151, p. 174]. Going further, so-called deep explanations separating the
domain model from the structural knowledge have been sought, where “the system has to try to figure
out what the user knows or doesn’t know, and try to answer the question taking that into account.”[204,
p. 73] Progress in MR explainability notwithstanding, it can been argued (see e.g. [137, 151, 169])
that to date, explainability in MR particularly and perhaps in AI at large is still insufficient in aspects
such as justification, criticism, and cooperation. These aspects, among others, are of concern in the
modern MR explainability scene (around year 2000 onwards), whereby novel approaches to explain-
ability in various branches of MR have been making appearances. We review some of them here and
spell out the explainability questions addressed therein.
1See, however, e.g. [25] for an alternative view of MR stemming from a sub-symbolic/connectionist perspective.
2https://www.doc.ic.ac.uk/ afr114/ecaitutorial/
3https://lat.inf.tu-dresden.de/XLoKR20/
2
Ericsson Research
K. ˇCyras et al.
1.1 Motivation
Report on Explainability in Machine Reasoning
The following summarizes our motivations and assumptions in this work.
1. We appreciate that MR is not yet a commonplace AI term, unlike e.g. ML or KR. We recognize
that MR has evolved from KR and comprises other, mostly symbolic, forms of reasoning in AI.
However, we here do not attempt to characterize MR, let alone cover all of its branches. Rather,
we focus on the MR branches that most prominently exhibit approaches to explainability. Our
overview of MR explainability is therefore bounded in scope. Nonetheless, it is dictated by our
(invariably limited) professional understanding of the most relevant (e.g. historically important,
well established and widely applicable, or trending) MR contributions to XAI.
2. Accordingly, we acknowledge that explainability in AI has been studied for a while and has
in time evolved in terms of (a) areas of AI and (b) desiderata. Yet, we maintain that the foun-
dational contributions and the lessons learnt (as well as forgotten) from the research on ex-
plainability in KR and expert systems are still very much relevant to both MR explainability in
particular, and XAI at large. Specifically:
(a) Adaptations of long established MR explainability techniques (Classical Logic-Based
Reasoning and Non-classical Logic-Based Reasoning (Logic Programming); see Sections
3.1 and 3.2 respectively) have found their ways into newer MR areas (for instance, Axiom
Pinpointing in Description Logics and Planning; see Sections 3.1.1 and 3.5 respectively).
(b) Relatively newer MR branches, such as Answer Set Programming (ASP, Section 3.2.3)
and Argumentation (Section 3.3), necessitate and inform newer forms of explainability.
In particular, previously established techniques of mostly logical inference attribution do
not suffice anymore, see e.g. [82, 123, 151, 204]. On the one hand, some modern MR
approaches, such as ASP and Constraint Programming (CP, Section 3.1.2), currently pro-
vide techniques that can effectively be considered interpretable but whose workings are
nevertheless difficult to explain. On the other hand, XAI desiderata now include aiming
for dialogical/conversational explanations, interactivity, actionability as well as causality.
These rediscovered concerns are being addressed by modern MR approaches to explain-
ability. Thus, various branches of MR are concerned with explainability anew.
3. We maintain that modern MR systems can hardly be called explainable just by virtue of being
symbolic4, in contrast to early expert and intelligent systems often being referred to thus [1].
However, we speculate and contend that MR presumably offers more immediate intelligibility
than e.g. ML. Perhaps this is what lends MR explainability approaches to be applied to both MR
itself and other areas of research, such as ML (e.g. [3, 15, 51, 106, 185]), decision support and
recommender systems (e.g. [45, 80, 158, 172]), planning (e.g. [38, 44, 69, 74, 81]), scheduling
4Where symbolic entities carry intrinsic semantical meaning and are perhaps more readily interpretable and intelligible
than the algebraic-symbolic entities in sub-symbolic/connectionist AI.
3
Ericsson Research
K. ˇCyras et al.
Report on Explainability in Machine Reasoning
(e.g. [58]), legal informatics (e.g. [17, 57]), scientific debates (e.g. [183]). We speculate that
it is also potentially the theoretical guarantees often ensured by MR methods that make MR
explainability appealing, cf. e.g. [105, 187].
4. Most recent overviews of general XAI appear to focus mostly on ML and somewhat ignore
MR, e.g. [1, 13, 96, 146, 149, 177], apart from potentially briefly discussing early expert sys-
tems. (See however [158] for a systematic review of explanations in decision support and rec-
ommender systems, where MR constitutes majority of the referenced approaches; reviews of
explainability in e.g. ASP [78] and Planning [44] are also welcome examples of not-so-common
area-specific MR overviews.) We feel that having a broader view of the XAI agenda is crucial
in general and that an overview of MR explainability is due for at least the following reasons.
(a) Explainable MR constitutes a rich body of research whose works span many branches of
MR with long histories. A consolidated, even if limited, overview will provide guidance
to exploring and building on this research.
(b) MR explainability techniques are also used for e.g. explainable ML and an overview will
help AI researchers to see a bigger picture of XAI as well as to promote innovation and
collaboration.
(c) XAI has arguably started with explainable MR and some of the same conceptual MR
explainability techniques can be (and are being) applied to achieve current XAI goals.
An MR explainability overview may allow researchers to rediscover known problems and
solutions (potentially saving from reinventing things) and to give credit where it is due.
1.2 Contributions
Our contributions in this report are as follows.
1. Building on conceptual works on XAI as well as XAI overviews, we propose a loose catego-
rization of MR explainability approaches in terms of broad families of explanations, namely
attributive, contrastive and actionable.
2. We provide an overview of some prominent approaches to MR explainability, differentiating by
branches of MR and indicating the families that such explanations roughly belong to.
3. We indicate what kinds of questions the explanations aim to answer.
This is not a systematic review, but rather an overview of conceptual techniques that MR brings
to XAI. We do not claim to be exhaustive or even completely representative of the various MR ap-
proaches to explainability, let alone to characterize what counts as a branch of MR. Rather, we hope
to enable the reader to see a bigger picture of XAI, focusing specifically on what we believe amounts
to MR explainability.
4
Ericsson Research
K. ˇCyras et al.
1.3 Omissions
Report on Explainability in Machine Reasoning
In this report, we omit the following aspects of XAI.
• We leave out the following research areas that could potentially be considered related to MR.
1. Rule-based classification/prediction, e.g. [209], which comprises of largely ML-focused
approaches for associative/predictive rule generation and/or extraction. Overviews of such
rule-based ML explainability can be found in e.g. [1, 9, 13, 96].
2. Neuro-symbolic computing and graph neural networks, see e.g. [129] for an overview,
where the use of classical logics alongside ML techniques has recently been argued as
a factor enabling explainability. Other works use classical logic for approximation and
human-readable representation of ML workings. For instance, the authors in [47, 48]
use neural networks to learn relationships between ML classifier outputs and interpret
those relationships using Boolean variables that represent neuron activations as well as
predicates that represent the classification outputs. These then yield human-readable first-
order logic descriptions of the learned relationships. Neuro-symbolic computing com-
prises heavily of ML-focused approaches that we deem beyond the scope of this report on
MR explainability.
3. Probabilistic reasoning, which is a field (of Mathematics as well as Computer Science)
in itself, and spans AI at large. We meet several explanation-oriented approaches when
discussing various MR branches, but we do not separately consider Bayesian Networks or
Probabilistic Graphical Models among MR branches with a focus on explainability.
4. Game theory, which is a field of Mathematics that studies strategic interaction between
rational decision makers. In AI, game theory provides foundations to reasoning with pref-
erences, multi-agent systems (MAS) and mechanism design among others, possibly with
the benefit of enabling explainability. SHAP (Shapley additive explanations) [140] is a
well-known example of application of game theory to explainability in ML as well as MR
(e.g. [126]). We do not think that game-theoretic approaches constitute a branch of MR,
or at least not one with a particular focus to explainability. We do, however, review some
related approaches to explainability from Decision Theory.
5. Robotics, which is an interdisciplinary branch of Engineering and Computer Science. In
terms of explainability, the cornerstone notion there is explainable agency [130], which
amounts to an autonomous agent being able to do the following:
(a) explain decisions made during plan generation;
(b) report which actions it executed;
(c) explain how actual events diverged from a plan and how it adapted in response;
(d) communicate its decisions and reasons.
In [10] Anjomshoae et al. review explainable agency approaches where goal-driven agents
5
Ericsson Research
K. ˇCyras et al.
Report on Explainability in Machine Reasoning
and robots purport to explain their actions to a human user. We encounter some approaches
to explainability of autonomous agents which belong to specific branches of MR such as
Planning. Generally, however, explainable agency considers human-AI interaction aspects
which we do not cover in this report.
• Medium of explanations, i.e. whether explanations are textual, graphical, etc., see e.g. [93,
128, 149, 189]. However important, these are largely human-AI interaction aspects, beyond the
scope of this report.
• Evaluation of explanations is beyond the scope of this report too. We believe that, on the one
hand, well-established computational metrics for systematic comparison and evaluation of ex-
planations are generally lacking [97],5 which we believe is especially true with respect to MR
approaches to explainability. On the other hand, we believe that research on evaluation of,
particularly, MR explainability approaches via human user studies is complicated and not yet
mature either, see e.g. [149].
We hope these omissions do not lessen our contribution of a conceptual overview of MR explain-
ability.
2 Explainability
We briefly discuss here the main purposes of explanations in AI systems and present a categorization
of explanations. We propose that, intuitively, the main purpose of explanations in XAI is to enable the
user of an AI system to not only understand (to a certain extent) the system, but also to do something
with the explanation. We suggest this entails that the explanations answer some sort of questions about
the AI system dealing with the problem at hand. The kinds of answers provided by the explanations
will help us to loosely categorize them, enabled by approaches to explainability in MR.
2.1 Purpose of Explanations
At large, the purpose of explainability of an AI system can be seen to be two-fold, to quote from [110,
p. 160]:
The system either provides knowledge and explanations necessary for the user to carry
out his or her task, or alternatively, the system carries out some action and then explains
the need and reason for the action the system itself has taken to the user.
Borrowing from [13, p. 85, our emphasis], we stipulate what an explainable AI system entails:
Given an audience, an explainable Artificial Intelligence is one that produces details or
reasons to make its functioning clear or easy to understand.
5Though the literature on computational measures of explainability in ML is expanding, see e.g. [37, 149].
6
Ericsson Research
K. ˇCyras et al.
Report on Explainability in Machine Reasoning
In other words, an explainable AI system has to first produce details and reasons underlying its
functioning – call this an explanation. It is then upon the explainee (i.e. the recipient or user of the
explanation, also called the audience) to take in the explanation. Thus, we are interested in the purpose
of an explanation (see e.g. [149, 158, 176, 194]) from the explainee point-of-view:
“What will I (i.e. the agent, human or artificial) do with the explanation?”
We do not attempt a representative list of the purposes of explanations, but nevertheless give some
examples. For instance, an expert in the domain in which an AI system is applied may want to do any
one of the following:
a) Understand how the system functions in general, in mundane situations, whence they expect
explanations to pertain to certain known aspects of the domain;
b) Learn how the system outputs aspects of the domain that are unexpected to the expert;
c) Confirm and compare the system’s behavior in both expected and unexpected situations;
d) Act on the produced knowledge, whence they expect guidance towards desirable results.
Different purposes of explanations are well reflected by what are called XAI design goals in
[149], that amount to “identifying the purpose for explanation and choosing what to explain for the
targeted end-user and dedicated application.” See also e.g. [158, 194] for purposes of, particularly,
human-centric explanations in AI. Note, however, that we do not limit ourselves to human users of
AI systems. Instead, we allow AI (or otherwise intelligent machine) agents themselves to require
explanations from AI systems. Considering the progress in developing intelligent machines and au-
tonomous systems, it seems natural to foresee situations where an AI agent probes another for e.g. a
justification of the latter’s (intended) actions in order to achieve or negotiate common goals.
We do acknowledge that the intended purpose is part of the explanation’s context, among other
factors such as the explainer, the explainee and the communication medium [194]. The context, specif-
ically the explainee, may inform or correlate with the purpose of explanations, especially in cases of
human audiences [13, 149, 198, 214]. But that need not generally be the case: if the explainees are
people, then it is “people’s goals or intended uses of the technology, rather than their social role”
[214] that shape the explainability desiderata. Note that there can be both AI-to-human and AI-to-AI
explainability, but we consider the purpose of explanations, rather than the nature of the explainee, to
be primary.
Finally, we recognize that various usability desiderata [189], including actionability (see e.g. [59,
123, 189]), are key to the purpose of explainability. We thus maintain that irrespective of the purpose
of explanations, for the explainee to consume and act upon, i.e. “to do something with” an explanation,
the explanation has to answer some question(s) about the AI system and its functioning. We discuss
next how explanations in MR (and potentially in AI at large) can be categorized according to the
questions they aim to answer.
7
Ericsson Research
K. ˇCyras et al.
Report on Explainability in Machine Reasoning
2.2 High-level Questions
Intuitively, answering what-, why-, when-, how-types of questions about an AI system’s functioning
falls under the purview of explainability [81, 137, 149, 155]. At a high-level, we are interested in the
following questions:
Q1 Given a representation of the problem and given a query, e.g. a decision taken or an observation,
what are the reasons underlying the inference pertaining to the query?
Q2 Given a representation of the problem and its solution or an answer to a query, why is something
else not a solution or an answer?
Q3 Given different information or query, e.g. a different decision or observation, would the solu-
tion/answer change too, and how? (Purely based on explanation, without recomputing!)
Q4 Given a representation of the problem and its current solution, and given a representation of a
desired outcome, what decisions or actions can be taken to improve the current solution and to
achieve an outcome as close as possible to the desired one?
These questions need not be limited to MR, but may well apply to e.g. ML too.
Questions of type Q1 pertain to inputs to the system (see e.g. [137, p. 14]), definitions and meaning
of internal representations (see e.g. [155, p. 387], [128, p. 110], [32, p. 164]) and attribution of those
to the outputs (see e.g. [149]). Such questions aim at soliciting “insight about the information that is
utilized and the rules, processes or steps that are used by the system to reach the recommendation or
the outcome it has generated for a particular case.”[93, p.1483] Answers to Q1 type of questions thus
“inform users of the current (or previous) system state” and how the system “derived its output value
from the current (or previous) input values”[137, p. 14].
Questions of type Q2 and Q3 pertain to reasons against the current outcome and in favor of alter-
native outcomes as well as to alternative inputs and parameters (see e.g. [32, 81, 128, 137, 149, 155]).
Such questions solicit characterization of “the reasons for differences between a model prediction and
the user’s expected outcome.”[149] Answers to Q2 and Q3 types of questions thus “inform users why
an alternative output value was not produced given the current input values” and “could provide users
with enough information to achieve the alternative output value”[32, p. 167].
Questions of type Q4 pertain to changes to inputs, modelling or the problem itself that would
lead to user-desired outputs (see e.g. [137, 149]). Such questions aim at soliciting “hypothetical ad-
justments to the input or model that would result in a different output”[149]. Answers to Q4 type of
questions thus provide guidelines that help the user to achieve a (more) desired outcome.
We next propose a loose categorization of explanations in MR (and potentially AI at large) that
places explanations in families of types of explanations aiming to answer the high-level questions.
8
Ericsson Research
相关推荐
2023年江西萍乡中考道德与法治真题及答案.doc
2012年重庆南川中考生物真题及答案.doc
2013年江西师范大学地理学综合及文艺理论基础考研真题.doc
2020年四川甘孜小升初语文真题及答案I卷.doc
2020年注册岩土工程师专业基础考试真题及答案.doc
2023-2024学年福建省厦门市九年级上学期数学月考试题及答案.doc
2021-2022学年辽宁省沈阳市大东区九年级上学期语文期末试题及答案.doc
2022-2023学年北京东城区初三第一学期物理期末试卷及答案.doc
2018上半年江西教师资格初中地理学科知识与教学能力真题及答案.doc
2012年河北国家公务员申论考试真题及答案-省级.doc
2020-2021学年江苏省扬州市江都区邵樊片九年级上学期数学第一次质量检测试题及答案.doc
2022下半年黑龙江教师资格证中学综合素质真题及答案.doc
