Keywords

1 Introduction

The proliferation of immersive computing, pervasive computing, ubiquitous computing, and ambient intelligence has brought attention to a myriad of new questions related to the dynamics of memory under the influence of technology—particularly for the shifting interplay between internal and external memory [1,2,3]. Although humans have tried to offload memory tasks well before the advent of modern digital technology [4], the unique affordances introduced into our affairs through new technological environments has forced consideration of the ways in which technology-mediated cognitive operations differ from previous forms of cognitive operations, and how these differences impact past and future scholarship. The purpose of this paper is to explore the implications of cognitive offloading in our current technological environment. To accomplish this task, we will first characterize memory and memory control to accommodate the intricate role of human memory in the context of the global information infrastructure. Then, we will explore the ways in which strategically offloading information in particular contexts of technology use can potentially expand, constrain, and alter the functioning of memory and cognition. We end with an example from our own empirical research that illustrates the influence of technology features on our perceptions of knowledge.

2 A Characterization of Memory and Memory Control

Beyond cognitive processes that reside within the confines of internal memory storage exists several extramnemonic processes that produce behavior that we typically understand as within the realm of memory. We store the phone numbers of our friends, classmates, and roommates on our smartphones, although we decide that it may be useful to store our partner’s number internally. Yet, if someone asks us if we know our roommates number, we are naturally inclined to say that we do if the information is accessible to us. In such a perspective, memory is not only related to storage capacity, but also extramnemonic skills that exists in the larger cognitive context of servicing intellectual and behavioral goals.

Memory capacity reflects the dynamic ways in which individuals strategically select information for encoding to enhance memory performance, while reducing cognitive demands on memory [5]. Accordingly, the accurate assessment of one’s own memory performance is a crucial ability to memory and memory control. If an individual can accurately monitor learning, she can make key decisions that will service her intellectual and behavior goals, such as how to recall information and when to terminate practice. If judgments of learning are inaccurate, the allocation of subsequent study time will suffer. Still, how people think about and monitor their own learning is highly imperfect [6]. Take as an example—while traveling to London by train, Alan Baddeley [7] noticed a familiar face, and decided to attempt to resolve how he knew the man. First, he recalled two associations related to the man—the name Sebastian and something with children. This led to several other relevant (and irrelevant) associations until he successfully recalled that the two were in the same baby-sitting group. Now, imagine the same scenario, but this time after seeing the familiar man on the train, Baddeley noticed that he was pushing a child in a stroller, which allowed him to immediately remember the baby-sitting group. Based on these two scenarios, Baddeley would likely feel more confident that he will be able to remember the man in the second scenario because he was able to identify him immediately. However, because this inference relies on contextual cues, we cannot be sure that it is diagnostic of future learning. In fact, the scenario in which Baddeley spent more time searching for the correct answer might lead to better long-term retention, despite having taken longer to retrieve. In the same way, because learners use monitoring to efficiently obtain their learning goals, better monitoring accuracy is expected to be related to more effective learning and higher levels of retention.

To illustrate this perspective, Benjamin (2007) considers the memory behaviors of lifeloggers—individuals who engage in comprehensive external encoding using technologies such as video recorders, computers, mobile devices, and so forth. There are several advantages to lifelogging. First, because storage capacity on a hard drive is practically limitless, decisions do not need to be made about what and how to encode. Second, because external digital sources are increasingly accessible, information is more easily “recalled” from memory stores. Third, because outsourcing information reduces cognitive demands on memory, time and resources are freed up for other activities [5]. In Alan Baddeley’s case, outsourcing the question of how he knew the man on the train to a digital device may have been optimal given that the information is, and will continue to be, one Facebook search away. But, what about the information that must be committed to memory, such as information relevant to creativity, problem-solving, expertise, and other complex domains of knowledge? Without the skills in interacting with memory, how can the lifelogger accommodate these cognitive tasks? Here we must consider one critical advantage of strategic mental encoding—higher-order cognition guides memory behavior, but also memory guides higher-order cognition [5]. Therefore, the ability to strategically control memory access and flexibly use the outputs of memory processes to serve specific tasks may be as important as the ability to store information in a place that is accessible.

Control of memory is particularly important in consideration of the current and future potential for offloading aspects of memory in new technological environments. With the arrival of ubiquitous computing and ambient intelligence, our personal devices have become an essential component to our own memory and knowledge [1, 2]. In fact, many memory processes are now accomplished with the help of digital technology (e.g., remembering birthdays, findings directions). Offloading responsibility for information is optimal in many cases, such as when accuracy is paramount, or when offloading unneeded information may reduce interference of new information [5]. In spite of these positive impacts, such as expanding the capacity of human cognition and improving the efficiency of information searching, we should be cautious in assuming that all features of technology that reduce the cognitive effort of interaction and improve performance will necessarily benefit long-term retention and transfer of information [8]. This said, our ability to adaptively integrate internal with external processes, and our ability to monitor the decision to do so, represents a defining feature of what it means to be a successful cognitive agent in a complex environment.

3 Cognitive Offloading

When initially thinking about cognitive offloading, one might find it easier to think of familiar experiences connected to the term, such as storing important contacts on a phone, using a navigation app to finding directions, or archiving e-mails for later use. In its most basic sense, it is the idea that people can offload some of their cognitive functions onto technology, thereby extended the performance capacity of their human faculty [9]. Generally, cognitive offloading is understood as associated with common cognitive technology, such as computers and smartphones. At first glance, instances of offloading cognition onto cognitive technology may seem clear. The issue, however, becomes more evident once we consider how cognitive offloading may manifest in new media environments, and also media environments of the future. The arrival of wearable computers, the Internet of Things (IoT), and virtual and augmented realities are dramatically changing human-computer and computer-mediated interactions. Through improved modality, agency, interactivity, and navigability [10], these technologies offer users greater levels of “presence” and an illusion of non-mediation [11]. The need for having a physical and designated technological device for information retrieval, display and exchange is diminishing. Some tech analysts have gone so far as to predict the “death” of smartphones in the next ten years [12]. These new environments force us to reexamine fundamental constructs such as what technological affordances constitute as within the realm of cognitive offloading, and what are the subsequent consequences on human memory and perception. To address this issue, recent work has expanded the purview of cognitive offloading to include, in a very general sense, actions that offload cognitive demands onto-the-body and into-the world:

We tilt our heads while trying to perceive ambiguous images, we gesture while imagining spatial transformations, and we rely on smartphones and search engines to store and retrieve information. In other words, we often think using our bodies and the external world [13].

In each of these examples, an action is performed in a way that accommodates an ongoing cognitive act so as to reduce cognitive demands on memory and cognition. In this sense, cognitive offloading encompasses actions that offload cognition onto-the-body (e.g., gestures, physical movement) and into-the-wild (e.g., writing things down, setting reminders). Therefore, we settle on a definition of cognitive offloading as, “the use of physical action to alter the information processing requirements of a task so as to reduce cognitive demand” [13]. Nonetheless, new media environments have expanded the range of actions people can take to reduce the burden on memory and understanding this expansion of possibilities requires attention to what it is new media spaces afford to people. The complex and systematic ways that media technology influence different possibilities for action by the user raises important questions related to the ways that cognitive offloading manifest in new media spaces, and their potential consequences on human perception and behavior.

4 Cognitive Offloading in a Complex Media Space

The proliferation of immersive computing and ambient intelligence has brought attention to the variety of experiences made possible by features and affordances of emerging technology, and potential outcomes on human perception and behavior. Although the same cognitive tendency may lead people to offload information in a variety of technology-driven contexts, the outcomes of offloading this information—and accessing it in the future—may differ according to features of the environment and context of use. These emergent technologies expand human activities beyond the realm of physical reality or even create entirely new human experiences. Technology-mediated interactions have gone from serial and codified message exchanges to fully immersive experiences enriched with social cues and machine intelligence; and, as the boundaries between the virtual and physical spaces blur, a technology-mediated environment has emerged. In the next section, we will discuss particular human-technology dynamics that may have benefits for cognitive offloading, and also consequences for accurate monitoring of learning. The approach of this discussion will be to characterize cognitive offloading in the context of various media experiences, and to discuss potential consequences of this behavior. This will be explored through consideration of three prominent technological environments— immersive virtual environments, ambient intelligent environments, and ubiquitous and pervasive computing.

4.1 Immersive (Virtual) Environments

Immersive virtual environments (IVEs) alter our perceptions of ourselves and our surroundings. It does this by replacing sensory information with technologically synthetic content to manufacture experiences that feel real, even though they are mediated [14]. Wickens (1992) mentions several notable features of IVE technology which may create a greater sense of presence, such as three-dimensional viewing, dynamic displays, and enhanced sensory information (among other features). These structural features of IVE technology (e.g., virtual reality), have potential to reduce the cognitive effort required to navigate through and interpret information in the system. For instance, data overload in visual and auditory domains pose challenges to operators in a wide range of workplaces, such as aviation, medicine, or process control [15]. This said, multimodal interfaces may facilitate strategic offloading by allowing information to be distributed across channels in a more task-appropriate manner [16, 17]. Although this could be an effective way of reducing the cognitive load on a taxed memory system, we should be careful to assume that all features of IVEs will improve performance. This is especially true for features of IVE technology that eliminate desirable difficulties in the environment that may be necessary to promote flexible outputs and transfer of knowledge [8]. For instance, evidence suggests that guiding trainees through the correct landing path using flight simulations can produce error-free performance in immersive environments, yet produces poorer transfer to landing skills once augmentation is removed [18]. These findings suggest that realism itself will not invariably improve memory.

4.2 Ambient Intelligent Environments

Ambient Intelligent (AmI) environments refer to technology-mediated spaces that are sensitive and responsive to our requirements and desires. AmI environments incorporate aspects of context-aware computing, disappearing computers, and pervasive/ubiquitous computing to proactively support people in their daily lives [19]. It is an inconspicuous technological environment that is perceptive to the particular characteristics of human behavior and is capable of reciprocating with an intelligent response. There is a clear advantage of offloading cognition in AmI environments— not only do AmI environment retain the wealth of knowledge that is typical of common cognitive technology (e.g., the internet), but also, the ability for these systems to adapt to users needs and desires certainly reduces the burden on memory, and in some sense, removes the decision to offload all together. Nonetheless, the question of how this may affect memory behavior requires consideration. In a speculative vein, this type of “embedded” offloading may have consequences that are conceptually similar to those described by the “Google Effect” [1]. The Google effect is a phenomenon first described by Sparrow, Liu, and Wegner (2011) as the tendency to forget information that is perceived as easily accessible through Internet search engine such as Google. In the original study on the Google effect, the authors demonstrated that participants who typed to-be-remembered information into a computer that they expected would save the information remembered less than individuals who typed to-be-remembered information into a computer and did not think it would be saved (memory was assessed in both cases without the memory aid) [1]. In line with these findings, Storm, Stone, & Benjamin (2017) noted that using the internet to access information makes people more likely to use the internet to access new information, and less likely to rely on their own memory [20]. This said, while the vision of interacting with smart objects every day offers a great range of fascinating extensions to human performance, seamless reliance on external processes to satisfy cognitive demands may have negative consequences for intellectual and behavior goals that depend on mechanisms of information aggregation that are (presently) singular to human memory systems.

4.3 Ubiquitous/Pervasive Computing

Ubiquitous and pervasive computing is a concept where computing is made to appear anytime and everywhere. In these environments, information is always present or reachable and almost always delivers the information that is desired by the user. In its ideal form, technology of this type move with us through the world to build pervasive, yet inconspicuous systems for offloading tasks. People cannot possibly know everything. Therefore, the benefits of having information at our fingertips are obvious. In fact, searching the web may be even faster than searching internal memory; whereas efforts to recall information internally can be time-consuming—and often fruitless—search engines return search results instantly, often even faster than these questions can be asked. This being said, an informed user should be able to take advantage of this boundless access to information by utilizing effective encoding strategies that maximizes performance on intellectual and behavior goals, while minimizing cognitive demand. However, the evidence on users’ abilities to effectively control encoding strategies is mixed, and therefore the benefits of these systems are questionable. For example, Henkel (2014) examined whether the act of photographing objects influences what is remembered about them. On a tour of an art museum, participants viewed 30 objects—15 of which were photographed and 15 observed. Their findings reveal that participants remembered fewer objects and fewer details about the objects remembered if they had been photographed [21]. Although these findings highlight potential consequences associated with having technology available anytime and everywhere, the counterargument is clear—if technology is made available anytime and everywhere, what does it matter if information is not stored internally? Although externally-stored memory has the advantage of retaining information with reliable precision, it does not hold the capacity for self-organization. On the other hand, internal (human) memory systems have an exceptional capacity for self-organization and reorganization, which explains why creativity and expertise derive from well-organized internal memory systems and not digital memory [5].

5 An Example: Influence of Technological Ownership and Modality on Perceptions of Knowledge

As mentioned previously, the ability to adaptively integrate internal with external processes, and the ability to monitor the decision to do so, represents a defining feature of what it means to be a successful cognitive agent in our complex media environment. Still, how people think about and monitor their own learning is highly imperfect [6], and this lack of cognitive control is exacerbating by the unique actions afforded in particular human-technology dynamics. We now proceed to empirically explore our theoretical proposition through a case of one human-technology interaction: using technology to find answers to common declarative knowledge questions. These are the sorts of questions that we might come across in daily life. For example, you might be having a beer with friends after work when someone asks, “What do you think is the best-selling beer in the United States?” When this situation arises, you have one of two options—you can use your own internal knowledge to give your friend your best guess, or you can Google it. The results presented here come from a series of studies where we seek to determine how the act of searching for the answer internally versus outsourcing the query to Google influences your perceptions of your own knowledge, and also how particular features of the device you use to google the answer moderate the effect. For this discussion, we focus only on the moderators. For the full report, please contact the authors.

5.1 Research Questions

The purpose of this experiment was to test whether offloading cognition onto cognitive technology influences self-perceptions of knowledge differently when the cognitive technology is personally owned (versus not owned) and when information is accessed on a mobile device (versus a stationary device). Comparing different levels of familiarity with an external source and their relative influence on metacognition has been explored in various contexts. Similar manipulations have been studied in the context of close human dyads [22] and online access points (Google vs. Lycos) [23]. Thus, we directly examined the influence of ownership (own device: offloading cognition onto a personally owned device; control device: offloading cognition onto an unfamiliar lab device) on inflated cognitive evaluations. Accordingly, we predicted that retrieving answer to trivia questions from a personal device would result in inflated cognitive evaluations compared to retrieving answers from a control device. The modality of the device we use to access information may also carry cues relevant to knowledge judgments. Because mobile external digital sources are increasingly accessible, information is more easily “recalled” from memory stores. Thus, we also directly examined the influence of modality (smartphone: offloading cognition onto a smartphone; laptop: offloading cognition onto a laptop computer) on inflated cognitive evaluations. We predicted that retrieving answer to trivia questions from a smartphone would result in inflated cognitive evaluations compared to retrieving answers from a laptop.

5.2 Procedure

We aimed to recruit about 30 participants per condition (total 120 participants) based on the minimum suggested power (80%) used to detect differences between groups [24, 25]. The final sample contained 115 undergraduate students (94 female, M age = 19.63) at University of Illinois at Urbana-Champaign participated in a between-subjects design. Participants were randomly assigned to one of four independent groups divided by two independent factors: ownership (own device vs. control device) and modality (smartphone vs. laptop). This said, participants were instructed to use either their own smartphone (n = 27), their own laptop (n = 29), a control (lab) smartphone (n = 33), or a control (lab) laptop (n = 26) to find all their answers to a ten-item trivia quiz. Trivia quiz items were selected based on pre-tested fairness ratings [23]. Responses from the pre-test indicate that participants found the items to be fair, but not particularly obvious (e.g., “What is the densest planet in our solar system?”). After completing the trivia quiz, participants completed the dependent measures and then were debriefed before leaving.

5.3 Dependent Measures

Response Accuracy.

Responses to the ten-item trivia quiz were scored such that participants received one point for each correct response. Responses were counted as “correct” if they very closely or exactly match the correct answers (slight misspellings or conceptual matches will count as “correct”). The judgments were made by a research assistant blind to condition and experimental hypotheses.

Cognitive Evaluations.

Immediately after completing the trivia quiz, participants completed the Cognitive Self-Esteem Scale (CSE) [23]. This 14-item scale measures participants’ beliefs about their cognitive abilities. The CSE scale contains three sub-components that assess confidence in the ability to think (e.g., “I am smart”), remember (e.g., “I am proud of my memory”), and locate information (e.g., “I have a knack for tracking down information”). Responses were coded on a 7-point scale (1 = strongly disagree to 7 = strongly agree), such that higher ratings would indicate higher levels of CSE. The CSE scale demonstrated good reliability (α = .93).

5.4 Results

Response Accuracy.

We conducted a one-way ANOVA to evaluate the relationship between response accuracy and cognitive evaluations. Results revealed a significant effect of condition on response accuracy, F(1, 114) = 5.03, p < .05. Follow-up tests were conducted to evaluate pairwise differences among the means. Participants who used their own mobile device scored significantly lower on the trivia quiz (M = 8.52, SD = 1.05) compared to participants who used their own laptop (M = 9.34, SD = .90) and participants who used a control laptop (M = 9.23, SD = .71) (ps < .05). All other comparisons were not significant (ps > .05). Because the interpretation of this finding is ambiguous and extraneous to our investigation, we hesitate to draw conclusions on this result. We will, however, take note of this finding with respect to our main prediction regarding the effect of ownership and modality on cognitive evaluations.

Cognitive Evaluations.

We conducted a two-way ANOVA to evaluate the effect of ownership (owned versus control) and modality (smartphone versus laptop) on CSE ratings. As predicted, results indicate a significant main effect of ownership, such that participants had higher overall CSE scores when they used their own device (M = 5.21 SD = .74) compared to participants who used a control device (M = 4.93, SD = .90) to complete the experiment, F(1, 115) = 4.54, p < .05, \( \upeta_{\text{p}}^{2} \) = .039. Also, results indicate a significant main effect of modality, such that participants had higher overall CSE scores when they used a smartphone (M = 5.30, SD = .86) compared to participants who use a laptop (M = 4.82, SD = .74) to complete the experiment, F(1, 115) = 11.54, p < .05, \( \upeta_{\text{p}}^{2} \) = .094 (see Fig. 1). The interaction effect of ownership and modality on CSE was not significant (p > .05). This implies that the particular features of a device used to offload cognition, such as ownership and modality, may influence cognitive evaluations. Furthermore, cognitive evaluations are not related to actual performance.

Fig. 1.
figure 1

Effect of ownership and modality manipulations on cognitive self-esteem (CSE) mean scores (N = 115). Error bars represent standard error.

Full size image

5.5 Discussion

Media technology provide unique opportunities for offloading cognition in order to extend the capacity of our cognitive capabilities. Although the availability of these actions have wide-ranging implications, many of which are beneficial and valuable, they bring with it novel consequences. This example illustrates a particular instance of this dynamic human-technology interaction. Although participants correctly retrieved 9 out of 10 trivia questions on average, cognitive evaluations following the task reveal that features of a technological device used to offload cognition play an important role in influencing attributions of knowledge. More specifically, participants who used their own device to find answers reported inflated cognitive evaluations compared to participants who used a control device, and participants who used a smartphone reported inflated cognitive evaluations compared to participants who used a laptop. Our findings are consistent with the notion that individuals misattribute outcomes and characteristics of technology to the self while judging their own knowledge, which have potential consequences on strategic control of memory decisions, such as when to strategically encode information. For instance, a student who uses Google to study for an upcoming exam by “confirming” definitions he thinks he “mostly” understands may be surprised when he is not able to recall the information from memory during the exam. Likewise, a student who uses a navigation app to drive home to visit family may be caught off guard when they are unable to articulate the directions to a friend.

6 Closing Remarks

Taken together, the claims forwarded through this discussion are not intended to promote a technologically deterministic stance toward the positive or negative consequences of technology use. Instead, our discussion is meant to illuminate unique contexts of cognitive offloading made possible by a new, complex media space; and hopefully, lead our readers toward asking new questions—not about whether memory should be extended, but rather—about how to offer new answers to old questions given that memory has been extended. From the preceding discussion, we can see how and why the use of cognitive technology to facilitate information retrieval has become a pervasive habit of human behavior. Our growing digital memory repositories bring with them several potential advantages, such as when accuracy is crucial, or when the Internet is available and its use is contextually appropriate. Given that we will likely become increasingly connected to our digital memory, one may even argue that technological advancements will eventually obsolesce the need for strategic memory encoding. However, we should be cautious to assume that all features of technology that reduce cognitive demands, and even improve performance, will necessarily service our intellectual and behavior goals. After all, memory is more than just remembering [5]. So, a person immersed in a technologically rich environment may be more likely to remember a class essay deadline than a person who chooses to rely on memory alone. But, they may both have trouble deciding what to write about—both have “memories” inundated with trivial facts and details. Yet, the person who strategically encoded information is able to seamlessly navigate a self-organizing internal memory system to piece together the contents of the essay, the person who offloaded their knowledge in word documents, lecture images, and recordings is still sifting through their external “memory” for content relevant to the task at hand.

The broad assertion forwarded by this research is that new technological environments do not necessarily supplant human activity but rather changes it, often in unintended and unanticipated ways, and as a result poses new coordination demands on the user [26]. The unique affordances introduced into our affairs through interacting with new, complex media environments has forced consideration of the ways in which technology-mediated memory behavior deviate from previous memory practice, and how these changes beg reconsideration of antecedents and consequences of memory and memory control. This said, research is needed to explore the ways in which technology expands, constrains, and alters the functioning of memory and cognition in order to offer new answers to old questions, such as how to determine what information is most important to encode for some intellectual or behavioral goal, how to optimize retrieval practice to enhance that selective learning, and how to train people in this process.