Keywords

1 Introduction

Online product reviews are becoming very crucial in supporting buying decisions of consumers. These reviews are also called electronic word of mouth because they are written online by customers who bought the product and used it. For an online review to be influential on a buying decision, it should be helpful to consumers. An online review is more helpful to consumers if it is written by customers than the experts [1]. However, in products with more technical outcomes consumers will likely seek information from experts [2]. To get reviews, consumers use search engines, e-commerce websites, blogs and vlogs for both online and in-store shopping.

Social media is also a great source for obtaining product reviews. It has an extensive collection of reviews of every type [3]. Moreover, social media has an impact on the buying behavior of consumers [4]. People may not want to post reviews on the Internet; however, they do so unconsciously when they share information about their latest purchase with their friends on social media platforms such as Facebook and Twitter. Social media posts also include the experiences consumers have had with the product, which can be less biased and more in-depth. More and more people are using social media for sharing their everyday experiences, making this source valuable for getting reviews. Consumers will use social media reviews because social media gives social position and a virtual community to consumers, which are the two motives for consumers to read reviews online [5]. Also, one of the primary motives for consumers posting product reviews online is because there is a social benefit in doing so [6]. Posting reviews on social media will give consumers maximum social benefit. Consumers who want to get reviews are either not aware of this or don’t use social media because reviews are difficult to search for and are not available in a consolidated form.

There is no single platform on which consumers can get reviews from multiple channels. Specially for in-store shopping, when reviews are not available for products it is inconvenient to look for reviews online. Consumers are required to lookup different websites for getting reviews while standing in front of the product for a long time to make a buying decision. We are trying to consolidate reviews from multiple social media channels and present it in an augmented reality fashion.

Augmented reality is changing the way we see the world. Augmented reality is a technology which superimposes virtual objects in the real world and supplements reality [7]. Using augmented reality, information can be augmented on everyday objects and places to be viewed in an intuitive way using special applications. The interaction of the user with the real world and their perception of reality can be enhanced by the use of augmented reality [7]. We are using augmented reality in our research for displaying consolidated reviews from multiple social media channels to increase engagement and give a sense to consumers that they are directly interacting with the product.

In this paper, we are proposing an app that will scan the product packaging to grab keywords regarding the product using optical character recognition. It will then search for posts and comments regarding those keywords on various social media platforms and detect sentiments in the posts and comments. A report of reviews on social media will be generated and augmented over the product. To use the app, consumers will need to just scan the product using the application and reviews from different channels will be augmented on the product itself.

2 Theoretical Background

2.1 Product Reviews

Multiple studies have been done on knowing the effects of product reviews on the sales of the product [8,9,10,11]. Prior research showed evidence that customer reviews affect consumer buying behavior [8], which ultimately impacts sales. Researchers found that the volume of reviews has a greater influence on sales of experience products, while valance of reviews has influence on sales of search products [10]. Zhu and Zhang [9] found that when there is scarcity of information regarding a product, its reviews become important. Hu et al. [11] found that consumers pay attention to both quantitative and qualitative aspects of review. They also found that the impact of online reviews decreases with the increase in product age. There will be an overall effect but no unfavorable effect of positive and negative news.

There is some research that has been done on what makes a review helpful [1, 12, 13]. According to Pan and Zhang [12] characteristics of a product review, type of product and characteristics of a reviewer influences the helpfulness of a review. According to Li et al. [1] review helpfulness is a combination of perceived source credibility, perceived content diagnosticity and perceived vicarious expression. According to Liu et al. [13] the most important factors that affect the helpfulness of reviews are reviewer expertise, writing style and timeliness. Moreover, research suggests that consumers consider those reviews helpful which include both subjective and objective elements [14].

It is also important to know why consumers read and also post reviews. Hennig-Thurau and Walsh [5] identified five motives that entice consumers to lookup reviews online. Those motives are (1) getting information regarding a purchase, (2) establishing a social position, (3) being a member of a virtual community, (4) doing it in exchange of money and (5) learning to consume products. Along with this, consumers choose the sources that most cost effectively convey the information needed [15] and online reviews reduce the time to get information. Hennig-Thura et al. [6] also identified primary motives for posting reviews online, which are getting social benefits, getting paid, having concern for others, and self-enhancement.

2.2 Product Reviews on Social Media and Its Analysis

Wang and Chang [16] conducted a Facebook experiment to understand the influence of social ties and product related risk on purchase intentions when consumers see product review on social media. They found that reviews from a strong tie source (i.e. friend or family) have significant effect on consumers when product is high risk (i.e. expensive) and result in higher perceived diagnosticity of the information in the review. Another study [17] found that product reviews by peers on social media are positively associated with product attitude, which in return is positively associated with purchase intention.

Since Twitter is a large and popular microblogging platform, it is popular among researchers to study social media product reviews. Researchers consider tweets as “relatively new type of electronic word of mouth” [4]. Jansen et al. [18] found that 19% of tweets among all tweets in some way mention product or product brand. Also, out of those tweets that mentioned product brands, 20% contains sentiments or opinion regarding that brand, service or product and remaining 80% are seeking information regarding that brand. Promising results in research on Twitter shows that other social media platforms can reveal more benefits. Therefore, in our study we are focusing on Facebook and Amazon along with Twitter.

Analysis of online product reviews is done by sentiment detection. There are many research studies on different frameworks, model and approach for detecting and classifying sentiments. Some research investigates automatic sentiment analysis [19, 20], while others focus on speed of the analysis [21] and simplification [22]. Ghose and Ipeirotis [23] were able to identify helpfulness and the economic impact of the reviews using their model. They suggest to display helpful reviews first because they will be more useful. In another research, Ghose and Ipeirotis [14] designed two review ranking systems for consumers and manufacturers based on helpfulness and expected effect on sales, respectively. Liu et al. [24] came up with a framework for not only analyzing the reviews but also comparing reviews of competing products. Shaikh and Lobo [3] created a system to estimate sales performance by using online reviews. Kim and Hovy [25] developed a framework to automatically identify pros and cons in the reviews. Research was also performed on predicting utility of reviews [26]. All these analysis models and frameworks may fail if reviews are sarcastic. They may trigger a false negative. For this, Tsur et al. [27] presented a novel algorithm which can detect sarcastic sentences in product reviews.

2.3 Augmented Reality

Any system which combines the real world with the virtual world, provides possibility of real time interaction and works in a three-dimensional space is an augmented reality system [7]. Augmented reality systems evolved from see through head mounted display to handheld smartphones. Olsson and Salo [28], in their research discussed that there are two types of AR applications, one is an AR browser which shows content based on geolocation and the other is based on image recognition to detect markers and display content. They found more than expected positive experience of publicly available AR applications at the time of research. Their research shows that augmented reality enables users to do things better than before by helping them get a new perspective and obtain relevant information that is hard to find. They got mixed results on user interface and usability aspects of augmented reality but identified curiosity and novelty as main factors that influence users to install AR applications. Another study [29], although in different context also found that various underlying technological components affect the user experience of augmented reality and identified novelty as factor that facilitates positive user experience and emotions.

Augmented reality sometimes struggles with usability and user-friendliness aspects of the user experience. Research with augmented reality in the field of education shows that usability is a challenge for augmented reality which makes it difficult for students to use [30]. Kim et al. [31] studied failure of augmented reality applications in a Korean market. They suggest that while developing an app more focus should be on usefulness factors rather than enjoyment factors. They also found that information quality has great influence on perceived usefulness and enjoyment of an application. For improving the usability of augmented reality applications, Ko et al. [32] developed usability principles by identifying 22 usability problems. Duplicated expressions of information, providing limited information and unfamiliar icons were top three usability problems among all. For these problems, in their usability guidelines they suggested to display information actively using only a camera, providing additional information and designing universally understandable icons.

We also found research on an app similar to our proposed app, which runs in augmented reality by Balduini et al. [33] called BOTTARI. They used augmented reality to display recommendations related to point of interests around the geolocation of users using Twitter stream and static descriptions. Although, both apps revolve around augmented reality and social media recommendations, our app focuses on product reviews compared to point of interests. Moreover, the content of BOTTARI is based on geolocation while content of our app is based on image recognition and marker detection. The main difference is that their research was focused on how they designed BOTTARI and practical results they gained from it, while our research is focused on knowing the effect of social media product reviews on buying behavior and user experience aspects of augmented reality by conducting a research study.

2.4 Hypothesis

From our literature review it is evident that social media has potential to be used as a source of product reviews when presented in a consolidated form. We believe that consumers can make better decisions with the help of social media reviews and be more confident in their decision. Therefore, we hypothesize that

  • H1. There is an overall positive effect of social media product reviews on consumer buying behavior.

  • H2. Social media product reviews are useful to consumers.

Also, augmented reality is a promising technology to present information in an intuitive way as the literature indicates. We believe that displaying social media reviews in augmented reality will increase ease of use, engagement and overall experience with the app. Therefore, we hypothesize that

  • H3. Augmented reality increases overall experience of the app.

  • H4. Augmented reality increases usability of the app.

To test our hypothesis, we designed a controlled experiment to gauge consumer buying behavior and to see at what extent augmented reality helps in increasing experience compared to conventional information display methods. We will investigate current difficulty level of finding reviews and its impact on consumers in making their purchase decision. The following section discusses the methodology used in conducting the experiment.

3 Methodology

We conducted a between-subject research study to find the impact of social media reviews on the buying decision of consumers for a product when reviews are presented in augmented reality fashion in comparison to reviews presented on a static page. We will discuss more about it in this section.

3.1 App Design and Prototype

In our app design, we selected to show dummy product reviews from Facebook and Twitter in a consolidated form and dummy product ratings from Amazon. The app screen is divided into three parts, reviews from Facebook, reviews from Twitter and ratings from Amazon. Reviews from Facebook and Twitter include trends that shows percentages of positive and negative posts along with top positive comment and top negative comment. Ratings from Amazon include the number of people who have rated the product and their ratings. Dummy reviews and ratings were designed in a way to give a look and feel of actual social media accounts and posts.

We created three dummy product boxes of Bose portable bluetooth speakers to serve as products for purchase and placed them on a display shelf. These include SoundLink Color II, SoundLink Revolve and SoundLink Mini II. Using the same app design, we created two prototypes of our proposed app for each product. One is a static page containing reviews that will appear after scanning a QR code on a price tag and the other will augment reviews directly on the product after scanning it. Ratings and reviews for every product are different but both prototypes for each product contains identical information. For simplicity, we will call the prototype with reviews displayed on a static page as QR app and the prototype with reviews displayed in augmented reality as AR app in this research paper.

3.2 Participants, Method and Task

We recruited 20 participants for the study who are all students of a small university in the northeast United States. Participants were randomly divided into two groups. The first group was assigned AR app while second group was assigned QR app. We named the first group AR and second group QR. Therefore, one participant will work with only one of the prototypes making it between-subject study. We are conducting between-subject study to remove any biases that may come across if one participant will try both prototypes.

Before the task, participants were asked to answer pre-task questions regarding the difficulty in making purchasing decisions, decision factors, where they lookup reviews, frequency of looking up reviews and difficulty in finding reviews. After pre-task questions, participants were told a scenario for buying a portable Bluetooth speaker and were given a mobile device, which was the iPhone 6 plus with prototype app open for getting reviews of all the products placed on the display shelf. The AR group was asked to scan the product boxes directly while the QR group was asked to scan the QR code on the price tag. Participants were asked to come up with a product they will buy based on the reviews and rating available on the app. Participants were explicitly told that price should not be a factor in their buying decision.

After the task participants were asked interview questions regarding their buying decisions and factors considered in making their decision. Afterwards, survey questions were asked to rate confidence in purchase decision, usefulness of the app, ease of use, future use of the app, chance of recommendation to friend and overall experience with the app. At last, demographic questions regarding hours spent on social media, primary reason to use social media, online and in-store shopping frequency for electronics products, age and gender were asked. At the end of the survey, an open-ended question regarding their experience and suggestions for the app was asked.

4 Results

The sample had an age range of 19–27 years with an average age of 21.7 years. There were 8 females and 12 males. The participants reported an average of 2.34 hours per day spent on social media sites. The primary reasons participants listed for using social media is to know what their friends are up to, to interact with friends, for group messaging and to view new trends.

When analyzing the shopping behavior of the sample, we found that participants tend to purchase products online more compared to in-store for electronic goods. Out of 20, there were 11 participants who shop online between 7 times/year to more than 10 times/year and only 3 participants who shop in-store. While buying a product, about half of the time on average the participants have trouble deciding between two or more products. They lookup reviews of products about half of the time on average before and while purchasing a product.

It is slightly easy for them on average to find reviews about any product. To decide between different alternatives, they consider reviews as the top deciding factor (65% of participants) followed by price (40% of participants). Other deciding factors include features, quality and design. Amazon is the top choice (65% of participants) to lookup review, followed by looking up reviews directly on the website from where they are buying (55% of participants). Other places for looking up reviews include Google, YouTube and Reddit.

After performing the task, almost half of the participants in both groups selected SoundLink Revolve as the product they will buy. The prototype for SoundLink Revolve had more positive reviews compared to other two products. However, regardless of which speaker they selected, all of them considered positive and negative reviews, Amazon ratings and volume of ratings as factors for making their buying decision. Our aim was not to check which product they will buy, but to see if they are able to make a confident decision by looking at social media reviews.

Ratings of the prototype with reviews displayed in augmented reality were different compared to ratings of prototype with reviews displayed on a static page, but the difference is not statistically significant. Participants in both groups agree that they are confident in their purchase decision with 6.1 and 5.9 ratings in AR and QR groups respectively on average out of 7, shown in Fig. 1.

Fig. 1.
figure 1

Mean confidence rating of AR app compared to QR app, on scale of 1–7

Full size image

Participants in AR group agree that social media reviews were useful in making their buying decision with 5.5 rating out of 7 while the QR group somewhat agree that the reviews were useful with 5.4 rating out of 7 on average, shown in Fig. 2.

Fig. 2.
figure 2

Mean usefulness rating of AR app compared to QR app, on scale of 1–7

Full size image

Participants in AR group agree that AR app was easy to use with 5.9 rating out of 7 while QR group strongly agree that QR app was easy to use with 6.6 rating out of 7 on average, shown in Fig. 3. Participants in both groups somewhat agree that they will use this app in future to get help in their purchase decision with 5.2 and 5.4 rating in AR and QR groups respectively on average out of 7, respectively. Participants rate their overall experience with the AR app to be 4.2 in the AR group and with the QR app to be 4.1 in the QR group on average out of 5, shown in Fig. 4.

Fig. 3.
figure 3

Mean ease of use rating of AR app compared to QR app, on scale of 1–7

Full size image
Fig. 4.
figure 4

Mean overall experience rating of AR app compared to QR app, on scale of 1–5

Full size image

Participants stated that, the overall experience of participants with the app was positive. For the QR app, participants liked the design, color coding of positive and negative comments and ease of use. They also found it intuitive. For the AR app, some participants found it easy to use while others found it difficult to keep steady and hard to read text from the AR. Suggestions for the QR app include adding a filter for reviews, adding more and better quality of reviews, adding reviews from Amazon and clickable links with detailed reviews. Suggestions for the AR app include side by side comparisons of reviews for multiple products, feature for making augmentation still after scanning, adding more reviews and, displaying reviews based on personal preference.

5 Discussion

From the results, it is clear that product reviews play an important role in the buying decision of the consumers. This is consistent with prior research [8]. Consumers can sometimes have a hard time making purchasing decisions and depend on product reviews to help them. With more and more consumers buying products online, it is also getting easier for them to get reviews since all major ecommerce websites have a product review section under the description of each product to help customers in making a satisfactory buying decision. Amazon also has extensive product inventory with large amounts of verified reviews, which makes it a favorite place of consumers to lookup honest product reviews. However, consumers don’t consider social media as a platform to lookup reviews for products. In our results only one participant considered social media as a source for reviews. This is because, the primary reason our participants use social media is to socialize with their friends and family. They don’t know that social media has a power to give opinions about products they are interested in. Moreover, it is hard to search reviews on social media and consolidate it, which makes consumers stay away from social media for using it as a place for looking up reviews.

When provided with social media reviews in a consolidated form using either version of our app, participants were successfully able to use it to help them in making a buying decision. The consolidated form categorizes review into positive and negative sentiments only, which helps participants make a buying decision with minimal ambiguity. Both versions of the app showed positive ratings, in terms of confidence in buying decision, usefulness of app, ease of use and overall experience. Strong intent to use both versions of app in making future purchases decisions suggests a desire and interest in social media reviews. From the above discussion, it is clear that social media reviews help consumers to make satisfactory buying decisions. Likewise, in an open ended question about overall experience with the app, participants found social media reviews helpful and useful to look at while making their purchase decision. This result is also consistent with previous research [17]. Therefore, our hypotheses that there is an overall positive effect of social media product reviews on buying behavior of consumers (H1) and social media product reviews are useful to consumers (H2) are true.

In the case of augmented reality, there is a more positive trend in AR app compared to QR app but there are no huge differences in ratings between the two and the difference is not statistically significant. One important thing to note here is the content of both prototypes for each product. They contain exactly the same information. The only difference is in the method of display (i.e. augmented reality and static page). Even a small difference can show us the inclination of the participants towards any one of the prototype.

Let’s start with confidence in using the app. With exactly the same information in both app prototypes, participants are more confident in using AR app compared to QR app. This can be due to the advance and futuristic nature of the augmented reality app, which generated more confidence in purchase decision compared to QR app. Participants also consider AR app slightly more useful. Again, this can be due to their perception of augmented reality to be innovative and intuitive. The overall experience rating is slightly more for AR app than QR app. Since with augmented reality app, participants are experiencing something new for the very first time it is possible for them to give more rating to AR app compared to normal QR app. While the differences are not significant, the result show a supporting trend for our hypothesis that augmented reality increases overall experience of the app (H3).

Reasons for difference between ratings of AR app and QR app being not significant could be because of the small sample size. Since our study is formative we decided to go with small number of sample size. Also, our sample is composed of college students who are constantly exposed to new technologies like augmented reality. Therefore, participants might have considered augmented reality app normal and rated it almost the same as QR app.

However, when it comes to ease of use there is considerable difference between both prototypes. As augmented reality is known for positively transforming the user experience [28], it was assumed that it would be easier to use than QR app. But for participants, QR app was easier to use compared to AR app. Also, the participants are little more likely to use QR app in future than AR app. Therefore, our hypothesis that augmented reality increases usability of the app (H4) was not supported. Past research also shows that augmented reality struggles with usability [28, 30]. Answers to these differences lie in the responses to open ended questions. Participants found AR app tiring to use. For the continuous augmentation of reviews, it is required to hold phone still at the same position for the duration of the task. That was annoying for some participants. Moreover, dynamic nature of augmented reality resulted in non-steady augmentations which made reviews hard to read for some participants.

As for these challenges with the AR app, participants also provided solutions when asked for suggestions for the app. To make the app more stable and eliminate frustration in holding phone still, we can implement an augment and capture feature. This feature will make the screen still after information will augment and users don’t need to hold the phone still anymore. This will also solve the problem of non-steady augmentation and increase the ease of use of the app.

6 Limitations and Future Research

In our research, we are proposing to analyze product reviews and display them based on sentiment analysis i.e. top positive and top negative comments from Facebook and Twitter. However, much better systems and mechanisms are available for the analysis of reviews based on expected helpfulness [14] and scoring based on utility of review [26]. Our analysis also doesn’t include analysis of sarcastic product reviews [27], which may trigger false positive in analyzing sentiment of reviews. As Wang and Chang [16] showed, reviews from strong ties have a positive effect on consumers, therefore in future we can give preference to reviews posted by friends of the consumer.

We received many suggestions for the app from the participants in the study. In the future, we will implement those suggestions to make our app better. As shown in our results, augmented reality didn’t show much difference and in fact has comparatively low ease of use rating, which is an aspect we need to improve on. Increasing the number of subjects in future research will also help in getting more generalizable results. We can use different categories of products instead of electronics, specially products for which consumers don’t see reviews before buying. Improvements like side by side comparison feature along with augment and capture can be incorporated in a future version of the app to improve the usability of the app.

7 Conclusion

In this study, we proposed our novel app which combines product reviews, social media and augmented reality to have an impact on buying decision of consumers. The research study showed that our app has promising results and great potential to influence the buying intention of consumers. Social media reviews are found to be helpful in making decisions by consumers and augmented reality also increased overall experience of the app. Moreover, our results are also consistent with the previous research. By performing improvements in the app suggested by participants of the study, we will be able to overcome the usability issues of augmented reality encountered by participants while using the app.