University of Twente Student Theses

Login

Trust in automated decision making : how user's trust and perceived understanding is influenced by the quality of automatically generated explanations

Papenmeier, A. (2019) Trust in automated decision making : how user's trust and perceived understanding is influenced by the quality of automatically generated explanations.

This is the latest version of this item.

[img] PDF
3MB
Abstract:Machine learning systems have become popular in fields such as marketing, financing, or data mining. While they are highly accurate, complex machine learning systems pose challenges for engineers and users. Their inherent complexity makes it impossible to easily judge their fairness and the correctness of statistically learned relations between variables and classes. Explainable AI aims to solve this challenge by modelling explanations alongside with the classifiers. With increased transparency, engineers and users are empowered to understand the classifier's behaviour. Other positive effects are user trust and acceptance. Inappropriate trust, however, can be harmful: In safety-critical domains such as terrorism detection or physical human-robot interaction, users should not be fooled by persuasive, yet untruthful explanations. We therefore conduct a user study in which we investigate the effects of truthfulness and accuracy on user trust. Our findings show that accuracy is more important for user trust than transparency. Adding an explanation for a classification result can potentially harm trust, e.g. when adding nonsensical explanations. We also found that users cannot be tricked into having trust for a bad classifier with meaningful explanations. Furthermore, we found a mismatch between observed (implicit) and self-reported (explicit) trust.
Item Type:Essay (Master)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science
Programme:Interaction Technology MSc (60030)
Link to this item:https://purl.utwente.nl/essays/77415
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page