University of Twente Student Theses
The reliability of mystery shopper reports : the effects of disconfirmed expectancies and exposure to misinformation
Tang, Sijbrand van der (2014) The reliability of mystery shopper reports : the effects of disconfirmed expectancies and exposure to misinformation.
|
PDF
338kB |
| Abstract: | OBJECTIVES: An important obstacle impeding the reliability of mystery shopper reports is researcher cognition bias, as mystery shoppers are the research instrument. This study investigates to what extend mystery shopping reports are reliable by investigating effects of two forms of researcher cognition bias: disconfirmed expectancies and exposure to misinformation. METHOD: Using the method of mystery shopping, 63 mystery shoppers were divided over four conditions in a 2 * 2 field experiment (no disconfirmed expectancies vs. disconfirmed expectancies and no misinformation vs. misinformation). Instructed with a (manipulated) checklist, mystery shoppers were instructed to remember and report exact prices of five products from a local supermarket. Also, nineteen mystery shoppers participated in follow up interviews. Using an evolutionary interview structure, the goal of the interview was to offer explanation and meaning to findings of mystery shop visits. The follow up interviews proved a useful method for exposing experienced difficulties, such as forgotten or misremembered products. RESULTS: Out of 315 products observed, 217 times mystery shoppers reported correct, which represents an average of 3.33 out of five correct reports per mystery shopper. Mystery shoppers who were disconfirmed in their expectancies showed a halo-effect which negatively influenced correct reporting rates of surrounding items on checklists. The effects of exposure to misinformation were limited to manipulated products and did not show forms of a halo-effect. Mystery shoppers who were not confronted with a researcher cognition bias reported a significant higher number of correct reports (4.24). One third of mystery shoppers spoke up to the research leader about experienced difficulties. Follow up interviews showed that a lack of self-confidence in own observations and a lack of intrinsic motivation are reasons for mystery shoppers to deliberately withhold information about experienced difficulties. CONCLUSIONS: Results suggest that effects of researcher cognition bias have a significant negative effect on the reliability of mystery shopper reports. Inaccuracy of checklists resulted in a rise of incorrect reports, also for existing products. Moreover, two third of mystery shoppers did not speak up about experienced difficulties. Deliberately withholding information about experienced difficulties impedes the reliability of mystery shopper reports. |
| Item Type: | Essay (Master) |
| Faculty: | BMS: Behavioural, Management and Social Sciences |
| Subject: | 05 communication studies |
| Programme: | Communication Studies MSc (60713) |
| Link to this item: | https://purl.utwente.nl/essays/65021 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page