Keywords

1 Introduction

1.1 Present Situation

Checking the account balance or making a transfer via the Internet - for 40 million people in Germany online-banking is now a matter of course and a indispensable part of everydays life [1]. Registered banking transactions are made conveniently from anywhere daily through various devices such as computer, laptop, tablet or smartphone. Besides various advantages, such as spatial and temporal independence of branch banks, online-banking also salvages some dangers. Illegal actions, such as unauthorized access to bank accounts, cause immense damage. The German Bundeskriminalamt estimated the amount of loss through phishing attacks in online-banking on 27.9 million euros in 2014, representing an increasing annual percentage change of 11.5 % [2]. Even though approximately halving the number to 3,440 cases in 2012 through the use of various protective measures as for instance the mTAN method (delivery of a transaction authentication number, which is necessary for making a transfer, via SMS), the number of cases till 2014 have more than doubled with 6,984 cases in total [1]. This fact indicates that the perpetrators have technically adapted to the changed framework conditions and developed new, even better malware to bypass the transaction process considered safe [2]. Especially after the introduction of authorization mechanisms, which require a positive action of the account beneficiary, the so-called two-factor authentication, other non-technical attack routes are chosen to get to the personal login data and transaction numbers (TAN) of bank customers and used for the perpetrators purposes. These types of attacks are summarized under the concept of Social Engineering, which in wider sense describes the manipulation of a person at predetermined purposes. The term Social Engineering initially sketches not directly a criminal influence of another person, only the urged change in action of a individual for own purposes [3].

1.2 Social Engineering and Current Transaction Techniques

However, Social Engineering in conjunction with online-banking describes selective damaging assaults on the trust of users, e.g. with a manipulated, confidential appearing login page, which prompts customers in an unauthorized way to enter their TAN [3]. Since no personal contact with the user is necessary, assaults on sensitive information via various electronic channels can be carried out. The scenarios can be distinguished as followes [4]:

  • Phishing emails (query of personal identification data such as PINFootnote 1 or TAN)

  • Spam emails (attachments with links to malicious software ridden pages)

  • Manipulation of well visited sites (installation of malicious software on randomly visited pages)

  • Falsified web page appearance (simulation of a variation of the original page to get transaction data)

To carry out successfully attacks one has to attain the users`confidence. For this, various factors of social interaction between people can be considered. Therefore, perpetrators lay primarily focus on similarities (e.g. native language), sympathy (e.g. personalized approach), context (e.g. layout and logo) or feigned authorities (e.g. imitating official news) [4]. By means of these various factors, attackers attempt to collect informations about a person via the virtual path in order to influence their victim into divulging personal customer data. On the opposite side IT specialists are working on various authentication methods, which impede perpetrators using the captured customer data without active user actions. In this context, however, three concomitant problem areas occur, which are closely interwoven. On the one hand, verification systems are buildt to ensure, that the customer has to be proactive to guarantee, that not only a computer performs a transaction with stolen login keys. Near by, it is of paramount importance, that the customer checks the transaction data in two places: once when inserting the transaction data in the online-banking transaction form and repeatedly on the TAN-indicating device before entering the TAN for transaction completion. As we have already seen in a previous user study [5], this fact is closely associated with poor basic skills in relation to a safe use of personal identification information, a secure established connection, various types of online-banking fraud cases and a few other important aspects regarding online-banking. On the other side, since no studies dealing with mTAN or Sm@rtTAN authentication techniques (see next paragraph for explanation) have been documented yet, the unknown problem of acceptance and usability of using the transaction processes should be investigated. Therefore, as part of a research project funded by the German Federal Ministry of Education and Research we concentrate on how fraud protection for online banking can be improved. Thus, we focus on enquiring the usability of the two banking transaction techniques, which are predominantly used in Germany at the moment [6]: First, a transaction authentication number (TAN) transmission via SMS (the so-called mTAN technique) and second via flickering barcode on the computer screen (the so-called Sm@rtTAN technique using a TAN generator along with a banking card for generating the TAN). Both browser-based techniques are used as a form of single use one-time password to authorize financial transactions [7].

1.3 Aspired Goal

As there is an ongoing challenge between system developers and hackers, we see the need to consult the private customer as a resource to increase the online-banking security. The underlying intention focuses thereby on developing primal configuration guidelines to enhance the usability of the two authentication techniques. Furthermore, it shall be investigated how users (re-)act while executing a financial transaction to identify specific ways how to enable customers to increase their online-banking security. To investigate this issue the users behavior was observed in two user-centered studies, which were conducted in a laboratory setting. Therefore, the usability of these transaction techniques and matters of detecting different cases of fraudulent assaults were analyzed in two explorative approaches.

2 Methods for Both Experiments

2.1 Materials

Since it was an experiment, that should reflect as closely as possible the everyday use of the transaction process, a virtual database environment was created with the help of which transactions could be carried out in real form. The database environment was created by our federated project partners, modeled after the online banking environment of Raiffeinsenbanken and Volksbanken. In an account with personalized login (user name and password, same for all respondents) all services of a real bank could be handled, but the actions carried out ran only within the virtual environment - there were no real jobs executed. To execute a transfer, two transaction methods have been made available for the participants: the mTAN method and the Sm@rtTAN process. To perform a transaction via mTAN process participants have been handed over a mobile phone (Nokia 6233) on which the SMS with the transaction data and the TAN have been sent. To perform a transfer with the Sm@rtTAN process participants have been handed over a TAN generator with optical sensor for the flicker code and a debit card, which was only active within the virtual environment. To compare the two transaction processes several questionnaires were applied. To investigate the user acceptance, questions similar to the acceptance categories according to Pousttchi, Selk and Turovsky [8] were used. The user-friendliness of the transaction process has been interrogated with an questionnaire on technology acceptance [9]. In addition, the assessment of suitability for use was based on the international standard DIN EN ISONORM 9241/110-S (short version) by Prümper [10]. In order to assess, whether more technology-affine or technology-averted subjects participated in the study, a technology affinity questionnaire [11] was used as control variable. The interrogation of the demographic data of the participants included information on age, gender, educational attainment and occupation. In addition, the participants were asked about which Internet-enabled devices they currently use for online-banking and what authentication method they use for it. To this, answers were specified with the possibility of multiple answers. Additionally for conducting Experiment 2, we used three different potential assaults on the virtual database environment by means of three developed add-ons (developed by our federated partners). The add-ons based on Social Engineering included a written request for referral back to committee, which should not to be complied to, an additional, but unlawful interrogation of the mobile phone number after logging into the account, and a change in the recipient data in the background of the ongoing transaction. The last manipulation could only be recognized by participants through verifying the inserted transaction data (IBANFootnote 2 and transfer amount) either on the TAN generator or in the resulting SMS. Last, for documentation of statements gained through the method of thinking aloud we designed three templates for each of the three blocks (i.e. per add-on). It was also noted, whether the fraudulent assault was recognized as such, whether the transfer data have been re-examined before entering the TAN and whether doubts about the legitimacy of the requests came up.

2.2 Procedures and Tasks

At the beginning of the computer-based experiment the background of the study was disclosed to the participants. They were pointed to the recording of monitor movement (Mouse Movement and keyboard entry) as well as image and sound and then asked to sign a written consent form. Then the participants were seated at the PC workstation and familiarized with the virtual database environment (Declaration of login and transaction methods).

For Experiment 1 two different transfers were to make (Fig. 1). Therefore, the sequence of the transaction methods was randomized. After the subjects had become familiar with the surroundings, the first transfer was launched by means of an exercise sheet including the short reason for referral, transaction information and step-by-step manual of the current used transaction method. Upon completion of this the participants were asked to complete a questionnaire for user acceptance, usability and suitability about using the transaction method in each case at an adjacent laptop. Thereafter, the second transfer was carried out with the other transaction method and requested afterwards. At the end of the experiment, the participants were asked to provide information on their own affinity for technology and on their demography. Prior to the adoption compensation for participating was given to the participants. This procedure was quite similar for Experiment 2, except as there were three transfers to make instead of two. To bring in experience, which problems occur in individual steps during the transaction, the subjects were instructed to express their ideas loudly through the method of thinking aloud. Additionally, we used three add-ons to mock a fraudulent assault. The allocation of the transaction process happened again randomized and balanced across all three tasks, but at least had each method applied for a task. The distribution of the three add-ons also happened in randomized order. After completion of the transfer, the subjects were asked to fill up the questionnaires about users` acceptance, usability and serviceability regarding the method used, but only once per procedure. This means, that one of the three blocks was completed without the three questionnaires. At the end of the experiment, the subjects were asked to provide information to their own affinity for technology and for their demography. Terminatory, it was open to the volunteers to provide information on expectations and possible assistance regarding the general online-banking process through a paper-based questionnaire. Both experiments lasted for about one hour.

Fig. 1.
figure 1

Procedures of both experiments. Investigation Objective I: acceptance and suitability for use of the procedure. Investigation Objective II: survey of performance and visibility of safety-critical situations (i.e. add-ons)

Full size image

3 Results of Both Experiments

3.1 Experiment 1

Participants. With the help of the volunteers portal of the Institute of Psychology and Ergonomics of the Technical University Berlin 12 women and 13 men aged 24 to 65 years (M = 43.81, SD = 14.83) were acquired for the first part of the user study. Voluntary participation has been remunerated at € 10 per subject. On average, 22 volunteers use the online banking services of their bank about 8 years. The remaining three subjects reported no online banking usage since they have excessively high privacy concerns and a too strong sense of insecurity. Since the application of each transaction method and performing a transaction has been explained in detail before starting the recordings, the three subjects have not been excluded from the data. To engage in online banking operations, the participants used the laptop (12 subjects), the desktop computer (9 subjects), the smartphone (7 subjects) and the tablet (4 sub-jects). Again, multiple answers were possible. The participants were naive to the experimental hypotheses and signed a written consent form prior to the experiment, which regulated the use of personal information provided by the participants.

Results. The analysis of technology affinity questionnaire yielded an average value of 1.76 within a five-point scale of “1 = Strongly agree” to “5 = Strongly disagree” that was used in all subsequent questionnaires, except for Question 7 in the questionaire for acceptance, which ranged from 1 to 3. From this we conclude, that the participants can be described as more enthusiastic about technology and are representativ for a comparative study in the technical field. For assesing the user acceptance the following seven questions were used. Thereby, statistically significant differences in transaction procedures arised with regard to the acceptance of effort for the operation, which goes hand in hand with the use of this procedure (t (24) = 4.201, p < .000), the thinking that the procedure increases ones own independence (t (24) = 3.375, p < .01), the acceptance of the wherewithal for using the procedure like indexed lists or additional technology (t (24) = 2.520, p < .05) and the perception that the execution of the transaction using the given transaction procedure was easy for the participants (t (24) = 1.809, p < .000). The mTAN process thereby experiences a greater agreement regarding the expenditure, the independence, the use of additional resources and the ease of the through-guidance of a transaction. Rating the appealing of the transaction processes captured only a marginally significant difference (t (24) = 2.021, p = .055) between the two transaction process. This suggests that participants respond only marginally better to the mTAN process. The perceived confidentiality against attacks by third arties associated with own data was ansewered with medium consent for both transaction procedures (Sm@rtTAN = 2:52 mTAN = 2.64, t (24) = −0.514, p = .612). This could indicate, that the participants are unclear about the confidentiality of their data to third parties with respect to both transaction procedures. The operation duration of a transaction by means of respective transaction process presented no significant difference between the two methods. Both transaction processes were found to be appropriate in their duration. In summary, the procedure was rated positive for mTAN rather than for Sm@ rtTAN process. Supplementary, viewing the user-friendliness of the transaction processes, the two transaction procedures differ not significantly in ease of use nor in intention, but they differ significantly in terms of the perceived usefulness (t (24) = 1.718, p < .05). Therefore, both methods awarded a perceived ease of use and the use of the method was considered. However, the perceived utility for the mTAN method seems to be more pronounced. The serviceability of the chosen transaction processes produced significant values within the given test categories. Regarding the task appropriateness (t (24) = 4.974, p < .000), the system`s ability for self description (t (24) = 2.959, p < .01), the system`s support for learning to use it (t (24) = 3.413, p < .01) and the controllability (t (24) = 4.487, p < .000) the two transaction procedures differ significantly within the five-point scale, which militates in favor for a better perceived usability of the mTAN process.

3.2 Experiment 2

Participants. Again 15 women and 10 men aged 19 to 71 years (M = 27.56, SD = 9.14.68) were advertised for the second part of the user study with the help of the volunteers portal of the Institute of Psychology and Ergonomics of the Technical University of Berlin. Voluntary participation has been remunerated again with € 10 per participants. On average, all participants used about 7 years the online banking services of their bank. To engage in online banking operations, the participants used the laptop (23 subjects), the desktop computer (13 subjects), the smartphone (9 subjects) and the tablet (4 subjects). Again, multiple answers were possible. The participants were naive to the experimental hypotheses.

Results

Questionaires. The calculation of the control variable of the technology affinity questionnaire yielded an average value of 1.94 within the five-point scale. We conclude that participants can be described as more enthusiastic about technology and are also representativ for a comparative study in the technical field. Evaluating the users`acceptance, the mTAN process experiences a greater agreement regarding the expenditure (t (24) = 11.431, p < .000), the independence (t (24) = 7.856, p < .000), the use of additional funds (t (24) = 9.632, p < .000) and perceived facility of transaction execution (t (24) = 9.667, p < .000). In addition, it can be concluded, that the mTAN process responses better to participants than the Sm@rtTAN process. However, the perceived confidentiality with data was significantly better for the Sm@rtTAN process (mTAN = 2.72; Sm@rtTAN = 2.16; t (24) = −3.055, p < .005). This might indicate, that participants feel safer regarding the confidentiality of their data when using the Sm@ rtTAN process. In summary, however, the process of transaction was rated better for using the mTAN method rather than for using the Sm@rtTAN process. Viewing the usability of the transaction process, the two transaction methods differ significantly in the three categories usefulness (t (24) = −7.544, p < .000), ease of use (t (24) = −6.313, p < .000) and intention (t (24) = −6.656, p < .000). From this rating it can be concluded that the participants perceive the mTAN process as a more user-friendly procedure. A similar conclusion can be derived from analyzings the items of the questionnaire on serviceability. Except for the rating category conformity of expectation all remaining categories differ significantly between the two transaction processes The task appropriateness (t (24) = −7.927, p < .000), the system`s ability for self description (t (24) = −2.489, p < .020), the system`s support for learning to use it (t (24) = −5.458, p < .000), the controllability (t (24) = −6.556, p < .000) and the tolerance for faults (t (24) = −2.866, p < .009) the two transaction procedures differ significantly within the five-point scale. Based on the average values, a more positive perception of the suitability for using the mTAN method can be assumed.

Performance and the Method of Thinking Aloud. First, we show the results for Add-on1- Call for Refferal Back to Committee. In this fraud scenario 44 % of the participants did not realize, that it was a scam and 88 % of participants did not question the legitimacy of the request. 53 % of the participants did not verify the IBAN from the data in the TAN generator or in the SMS. Likewise, 47 % of the displayed amount was not additionally checked. However, since 67 % of the participants ignored the reffferal initially - partly by overreading, partly by the thought of carrying out the task - only 12 % of the participnats returned misleadingly the unclaimed money to the fraudulant account. When analyzing the results of thinking aloud reasons like real appearance of the banking website, actually transfered credit on virtual account and plausibility of the refferal were stated. In addition, some participants reportet, that a sense of pressure built by encouraging and reporting account lockout (Social Engineering) led to their referral back to committee. Upon appearance of the request of Add-on 2 (request to indicate the mobile phone number) seven participants stopped the login process due to safety concerns. Further evaluation of the performance of this fraud scenario showed, that 68 % of the participants entered the mobile phone number without hesitation to complete the login process. Only 44 % of the participants expressed doubts about the legitimacy of this query. Stated reasons therefore were a lack of concern about the dissemination of the mobile phone number and a rather greater sense of security with an additional request. In addition, some participants would enter the mobile phone number due to an existing time constraint without thinking about it. In 50 % of cases, the IBAN has not been checked with the information in the TAN generator nor in the SMS. Moreover, only 39 % of the participants stated also to check the amount. Analysing the fraud scenario of Add-on 3 (change of transaction data in the background) we could identify, that 71 % of the participants did not recognize the fraud attempt. 65 % of the participants did not verify the IBAN within the TAN generator or in the SMS and 70 % of the participants did not reconcile the amount. In this scenario, we could detect through the method of thinking aloud, that the participants mostly check the IBAN and the amount in the appropriate input box on the banking website, but then rely on “the technology and the bank itself” regarding further data tranfer. One participant caused furthermore a great concern by his statement. After recognizing a fraudlent attempt he would indeed sign off immediately, but then log in again and also enter the transfer data again. Should the error occur again, he then would rely on the device used. The evaluation of the statements on expectations and possible assistance regarding the online banking showed, that there is a great desire for faster and more easily help in case of fraud and for more information about possible fraud scenarios. When handling the Sm@rtTAN generator, the participants wanted a better view of the charging status of the transaction data transfer. The participants stated, that the previous continuous bar display would not show this status clearly, which is why some participants interrupted the transmission themselves. A building up bar display would therefore constitute the transfer progress better. In addition, a feedback optimization for handling the generator could be displayed on the monitor. This statement was mainly related to the tilt angle of the Sm@rtTAN unit on the monitor. This indicator could be further supported by an audio feedback of the device itself, in which the device could play a sound in case of successful or aborted transfer of data. The participants noted moreover, that if the transfer was canceled, it was not easy to see on the monitor since the generator had to be held in front of the display window. Here, the participants would prefer better displaying on the monitor (for example, a status indicator or an indication on top of the beam with the flicker code). Using the process of manual TAN generation, it was also noted, that the IBAN should be easily recognizable and depicted in chunks. Recently, the participants wanted a better haptic feedback for using the keys of the generator. The soft keys were indeed found to be pleasant, but the feedback was sensed as lacking when pressing the button.

Limitations. In connection with the review of the transaction process, however, testing in a laboratory environment must be considered. In this it remains questionable for us on the one hand, to which extent the participants were able to put themselves into the situation of a real transfer from their own online-banking account and therefore if they acted as they would do in their actual account. It is not clear, whether the study situation influenced the actions of the participants, so we wonder if a more task-oriented character or a more security based character prevailed during the transfer tasks. On the other hand, the requirement to learn thre usage of a new device might have distortde the situational awareness of the participants or have deviated them from a safe course of action. Since in this study the transfer by means of the TAN generator unfortunately often did not work immediately, which may also concern a home user, attention could have been so drawn to the device and its function, that after a successful transmission the verification of the data could have been fallen of to a kind of “tunnel vision” through facilitating or the peak of the stress level. Thereby, the longed for, rapid completion of the transaction could have been desired and that could be the reason why the transaction data were no longer verified. The disruption of the smooth transition by using the Sm@rtTAN process could therefore have led the deteriorated rating compared to Experiment 1 of the Sm@rtTAN process.

4 Conclusion and Future Ongoing

In both conditions, the task of the participants was to carry out a transfer using one of the transaction techniques mTAN or Sm@rtTAN in a virtual banking environment. In both study sections a questionnaire on usability, acceptability and serviceability of the transaction techniques has been completed following the transfer. Additionaly, Experiment 2 dealt with the expression of the participants` individual thoughts using the method of thinking aloud. While the first part of this study was exclusively dedicated to the usability of the selected transaction methods, the second part included three attacks on a virtual banking environment, which were imitated by specially programmed add-ons. When documenting the statements and the transfer implementation increasing attention has therefore been paid to the detection of attack scenarios to encounter primal conclusions about safety awareness and behavior of the participants. It was also noted if the subjects disclosed personal information, such as a mobile phone number, when requested and if they checked their own transaction data (IBAN and amount of money) that was available in connection with the transfer. In summary, the results of both parts of the study evidence that users do know little about some of the fraudulent assaults and they often lack in screening for attack- and safety features and rechecking the transaction data. We conclude that customers should be supported with more information regarding a safe online banking use in order to ensure a more sensitive handling. Farther, the usability and the stated acceptance of the two transaction techniques differ to a great extent. In both studies the Sm@rtTAN process was evaluated worse than the mTAN process. However, this can also be partly attributed to an improper function of the Sm@rtTAN device. The process of conveying the transaction data from the website to the TAN displaying medium often demanded too much attention and thereby createed a high level of stress, so that users may have forgotten to check the transaction data. Since this could cause immense financial damage in real banking transaction, a higher functionality of the device has to be created. This could be achieved for example by a re-design of the device in the desired direction (see Experiment 2/Section Additional Statements). Deepening this approach a follow-up study will investigate this results by questioning experts on usability and design. Thereby, possible graphical changes in the banking website are to be worked out in terms of an enhanced secure proper application. These suggestions can be discussed in a focus group discussion then to involve the users at an early stage in the development process. In addition, in order to embed the user as a security resource conducive to the transfer sequence the focus must be placed on the wide-ranging information of users. At this juncture, fraud cases should be explaned transparent and preventive protection measures for the home user should be defined in an understandable way. In order to draw the attention of users to the key parameters of an online transaction a kind of “persuasive technology” should be used in a subsequent step. Also reflections on interactions between design alternatives and the formation of mental models of users as well as the adequacy of mental models will be interesting in this context. Further research should examine this idea, too.