University of Twente Student Theses
Assessing User Satisfaction with Information Retrieval Chatbots
Wilmer, S.N. (2021) Assessing User Satisfaction with Information Retrieval Chatbots.
|
PDF
1MB |
| Abstract: | This study aims to develop an instrument that measures user satisfaction with chatbots, as such instruments are currently solely existing for vocal interaction experiences. The current study follows from previous research that established a preliminary 42-item questionnaire (USQ), aimed at measuring user satisfaction with chatbots. The additional value of this study lies in the revision and compression of the USQ. For this purpose, 48 participants interact with five chatbots and fill out the USQ after each interaction. A Principal Component Analysis is conducted, which leads to the establishment of a 24-item, four-factor USQ. Next, it is examined whether the 24-item USQ consists of sufficient internal consistency. Reliability assessment of the 24-item USQ indicates that the overall USQ score, as well as the individual factors, consist of sufficient reliability. Also, the 24-item USQ’s (concurrent) validity is examined, which is sufficient as well. This validity assessment is based on comparing scores on the 24-item USQ and its individual factors, with scores of an already existing questionnaire; the SUISQ-MR. This study additionally focuses on possible confounding effects of chatbot-familiarity and geekism, which are examined with the support of non-linear regression analyses. Solely chatbot-familiarity has a significant upward-opening parabolic influence on the scores of the 24-item USQ. Concludingly, this study confirms the results of previous research. Yet, it also provides additional insight in possible future research directions, advancing the ultimate goal; deploying a final revised version of the USQ that measures user satisfaction with chatbots. |
| Item Type: | Essay (Master) |
| Faculty: | BMS: Behavioural, Management and Social Sciences |
| Programme: | Psychology MSc (66604) |
| Link to this item: | https://purl.utwente.nl/essays/85847 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page