Promoting Algorithm Adoption in the Presence of Human Experts: A Field Study
41 Pages Posted: 27 May 2020 Last revised: 18 Mar 2021
Date Written: March 17, 2021
Promoting the adoption of algorithm-based service in the presence of human-based service is a challenge faced by an increasing number of service providers. We partner with a mobile platform that offers two types of free consulting services to support high school students in China to make college application decisions: a volunteer consulting service provided by college student volunteers and an algorithm consulting tool that provides personalized recommendations. Analyses of clickstream activities suggest that organic adoption of algorithm consulting is low, and students who use volunteer consulting rarely try out algorithm consulting. We then conduct a randomized field experiment to examine how sending facilitating information in the form of call-to-action messages affects algorithm adoption. We design three different messages: a straightforward reminder, a negatively framed message that highlights the inadequacy of volunteer consulting, and a positively framed message that focuses on the unique advantages of algorithm consulting. We show that all three messages significantly enhance algorithm adoption, with the reminder message having the strongest average effect. Moreover, message effectiveness varies across students with different experiences with volunteer consulting. Specifically, the effect of reminder message is stronger for students with more volunteer searches; the positive message is more effective for students who chat with more volunteers; and the negative message has a greater effect for students whose activities are suggestive of unsatisfactory experiences with volunteer consulting. Our work advances the understanding of algorithm adoption in the presence of human experts and demonstrates the effectiveness of call-to-action interventions to promote adoption.
Keywords: algorithms, human experts, service adoption, call-to-action, field experiment, mobile analytics
Suggested Citation: Suggested Citation