Keywords

1 Introduction

With the development of science and technology, human-computer interaction design has already affected every aspect of people’s daily life. Bill Gates, founder of Microsoft, believes that natural interface is a direct and natural interaction between human and computer. In the future, this new form may satisfy users’ diversified information exchange, efficient and accurate information transmission and functional requirements, among which the multidimensional nature of the natural user interface is more prominent [1]. In today’s mobile Internet context, social networking is still people’s rigid demand. Interaction design is the design of human behavior. In the process of information interaction feedback design, interaction design focuses on providing a good user experience rather than only realizing a certain function [2]. “Bullet message” app (Fig. 1) compared with the previous mobile Internet social products, through improvement on previous product information architecture level, the more efficient way of interaction, meet different scenarios of information expression and interaction function: when users need to communicate with people, users don’t need to enter each chat window can communicate, directly in the message list page to view messages or input. Users in meetings and other occasions is not convenient to listen to voice; inaudible speech in noisy environment or the received voice message has an accent which makes it difficult to hear clearly, the product will convert the input voice message into text and output it to the user.

Fig. 1.
figure 1

“Bullet message” software interface status diagram

Full size image

“Bullet message” not only improves users’ chatting experience, but also improves the efficiency of communication through the analysis of users’ demands in certain scenarios and the improvement of existing product structure layer in the market. Then, from the perspective of multi-dimensional perceptive design, can users have social products more suitable for the user groups in the information age in the context of mobile Internet? When users are chatting, does the designer have the space to design such an interface to increase the dimension of information in the process of input and output? Just as what Negroponte said: “there are considerable restrictions on the contact between perception and substance” [3]. Only when you design an input device that can provide tactile response can you elevate the tactile sense to a higher level [3], so as to better meet the social needs of users in the information age.

2 The Design Origin of “Sentient Input Method”

Masayuki Kurokawa, President of the Japan society of materiology, called the 21st century “a shift from the age of vision to the age of touch.” In this era, a new round of design reform – “tactile design” has gradually spread to all fields of contemporary design [4]. In recent years, the emergence of touch related 3D-touch technology has provided more diversified and convenient possibilities for the “input” part of human-computer interaction. At present, the application scenes of 3D-touch technology mainly include the following aspects: “shortcut entrance” function to supplement the existing operation process, users can pop-up “sweep”, “pay”, “riding code” and other quick operation by pressing the icon of the main screen Alipay; “Preview” function is that when the user presses a clickable content, a preview window will pop up. When the user presses harder, the preview window will become full screen display; Some scenario-specific applications, such as the use of brush tools, the emergence of functions such as expressing the depth of brush color according to strength and moving cursor by pressing text input, have brought better user experience for user touch and pressing functions. The increasingly integration of interactive behavior and mobile interface information communication provides new thinking for the subsequent development of mobile interface experience [5].

Through the analysis of the existing chat tools, user needs and the situation in which they interact, it is found that the simple text form cannot well convey the sender’s attitude, and the form of “text plus emoticons” in some situations reduces the efficiency of communication. In order to further verify this analysis, before designing the “sentient input method”, the research team used interviews and questionnaires to analyze user needs, and conducted an open one-to-one conversation around the points of interest in the product, the object of use and the desired form of text expression. The results show that 80.43% of the subjects will have the following situation when chatting on social software: when they want to express a certain emotion, but only using words to express it, the other party cannot identify with them; At the same time, 67.39% of the subjects had experienced misunderstanding due to their different understanding of words (such as different sentences and literal differences). Therefore, it is very necessary to conduct research on input interaction prototype that is more related to user emotion, and the advent of 3d-touch technology brings us a new exploration dimension.

3 The Prototype Design of “Sentient Input Method”

3.1 Information Input Mode Based on Pressure Sensing

In the prototype design stage of “sentient input method”, the product situation, interaction mode and user demand in the future were reflected and designed [6]. It is using 3D-Touch technology to measure real-time physical quantities such as pressure and time interval of information input by pressing the screen, combining with semantic recognition, users’real-time emotional information is obtained, which is appended to the original text information in the form of color and font, thus solving the semantic misunderstanding caused by the single expression of information in text communication, making the interaction way more diversified and communication more smoothly.

The input interface of “sentient input method” still imitates the default input method of ios system. The handwritten keyboard remains unchanged, the main changes are the English keyboard with 26 keys and the pinyin keyboard with 9 keys. To ensure that users can easily make personalized settings, both keyboards set the voice input key and the space key to the same key, and distinguish the two functions by pressing the length of time. Place the character setting key in the original voice key position. In order to reflect the user’s keystroke pressure in real time, the input method sets a pressure display bar on the right side of the selection bar. The pressure bar can display the user’s keystroke pressure in real time during the user input process. In addition, in the input process, a display preview box is set in the upper right corner outside the input bar to provide a preview of the output effect.

In the process of information input by users, the pressure display bar of “sentient input method” can display the pressure of users’ buttons in real time, and provide instant feedback through the status of the pressure display bar. In this case, three pressure bars of different shapes and colors were designed, and eye movement technology was used to test three college students, and the most suitable scheme was selected according to the results. All subjects used the same computer to test on the eye movement cloud test platform, and the eye tribe was used as The Eye tracker. Each subject was asked to browse three WeChat chat interfaces using the new input method (Fig. 2).

Fig. 2.
figure 2

Interface state of eye movement test

Full size image

All images are identical except for the pressure bar on the right side of the input method. After learning the function of the new input method, the subjects imagined that they were using the product to chat, then naturally scanned each picture. No time limit was set, after browsing, subjects were presented with an equal size white screen for interference, and then the next picture was presented. The heat map of the three pictures tested is shown in Fig. 3:

Fig. 3.
figure 3

Hotspots of eye movement test

Full size image

The prototype function of “sentient input method” is to judge the user’s current mood according to the pressure and semantics of the input text, and adjust the style of the text according to the mood. The input box content display semantics, the pressure display bar reflects the pressure, the dialogue content provides the context, the preview box displays the comprehensive analysis result and so on four parts should be the user most pays attention. Taken together, the pressure bar in Fig. 3(a) is the most appropriate choice.

3.2 Product Function Settings Based on User’s Wishes

In the early interview and questionnaire survey, some users showed the following concerns after understanding the function of “sentient input method”: the product’s emotion recognition may be wrong; the automatically generated text representation may not be to the user’s liking; sometimes users may not want to use special effects text, but it is cumbersome to turn off and on the input method every time.

Return to the design itself, which ultimately serves people [7]. For this reason, designers made targeted improvements in the subsequent iterative design scheme: first, users can customize colors and fonts to express different emotions according to their personal preferences, they can also download their favorite bubbles and decorations from the Internet. Second, users can select the text without special effects when entering each sentence, without needing to turn off the input method in the system settings. Third, the input method will provide a variety of types of text with special effects based on the analysis results of each sentence, and users can choose the one that most conforms to their emotions. Fourth, add the current pressure display bar in the user interface, users can be informed of their current input pressure, or choose to hide the pressure bar. Through the independent setting based on the user’s will, users’ needs in more situations can be met more intelligently and humanely.

3.3 Information Communication Standards Based on User Experience

  1. 1.

    Default font setting. In the color scheme of “sentient input method”, the color expression of five emotions is determined, which is the primary goal of the product presentation layer. On the basis of questionnaire survey and user interview, the five colors of yellow, red, blue, grey and black are determined as the basis of color scheme. The determination process was as follows: 46 users were asked to evaluate which of the following colors could better convey their emotions (multiple choices) by questionnaire, and the results were shown in Table 1.

    Table 1. Color preference of subjects in different emotions
    Full size table

    The test results in Table 1 clearly show that the subjects have different color preferences in different emotional states: yellow when happy (18 people), blue when calm (24 people), red when angry (27 people), gray when sad (24 people), and black when fearful (22 people). Therefore, this product also takes the user experience into full consideration when making improvements and satisfies the wishes of most users in the default color setting. At the same time, because the application needs to cooperate with WeChat, SMS, web page and other communication media to use, so according to the corresponding software color scheme, choose the auxiliary color system, color scheme design. Finally, the following color scheme was obtained: 1. Emotional color scheme,each mood color has 4 colors from light to deep, which can be applied to each interface according to the needs of the situation. 2. System color scheme, choose simple compatibility better color, can better with a variety of communication applications, do not advocate special changes.

  2. 2.

    Control standards for various design elements of the presentation layer. Control of important elements: logo, buttons, bars, etc. Considering the basic idea of color ring, circle is adopted as the motif of relevant color selection and logo design (Fig. 4).

    Fig. 4.
    figure 4

    Important elements

    Full size image

    Control of font elements: it is mainly divided into three categories of positive emotion, neutral emotion and negative emotion according to different emotions. Among them, the positive emotions were selected in cute fonts, such as founder meow. Neutral mood is used wireless foot font, such as Microsoft yahei. Negative emotions were expressed in more angular fonts, such as hanyi wheat (Fig. 5).

    Fig. 5.
    figure 5

    Font elements

    Full size image

    Control of emoticons: according to the questionnaire survey and interview, the main target users of this app are women. The whole expression element chooses cute style, imitation or animal image, and uses different colors to characterize and strengthen (Fig. 6).

    Fig. 6.
    figure 6

    Emoticons

    Full size image

4 Presentation of Design Scheme of “Sentient Input Method”

4.1 A Beginner’s Guide to the “Sentient Input Method”

“Sentient input method” first presents three input method introduction screens after installation. The first two pictures show the input method used in “happy” and “sad” respectively. The third screen presents the name, logo and purpose of the input method, and sets the “start using” button to guide users to start using the input method. Users can choose one of the five input methods (pinyin jiugongge, pinyin all-key, handwriting, stroke, wubi input) according to their personal input habits in the interface. After the user selects the keyboard, before the first use, the input method provides a concise input tutorial. The tutorial is divided into three pages: guide the user to input, click the preview box to send, and start using the input method. The tutorial highlights interaction and uses hand icons to guide users to click on keyboards (Fig. 7).

Fig. 7.
figure 7

Beginner’s guide

Full size image

4.2 Input Setting of “Sentient Input Method”

The “sentient input method” information architecture is shown in Fig. 8. The input interface contains the following interactive elements: when the user presses the key, the corresponding key turns gray, and the cursor of the pressure display bar moves. The distance of movement corresponds to the pressure on the keyboard. Click the word in the word selection box, the word becomes gray; The text in the input box is non-special effect text, and the preview box in the upper right corner outside the input method synchronously presents these texts according to the recognized emotion. Click the preview box to send special effect text, and click send to send no special effect text (Fig. 9).

Fig. 8.
figure 8

The sentient input method information architecture

Full size image
Fig. 9.
figure 9

Input interface

Full size image

4.3 The Function Setting of “Sentient Input Method”

The overall interface of the setting interface is a classic vertical arrangement of buttons, with one button per line leading to a specific setting interface (Fig. 10). The prototype realizes the personalized setting interface of emotion corresponding to color, and the interface for calibrating the reference pressure at input according to the different key pressure of different users (Fig. 11).

Fig. 10.
figure 10

Setup interface

Full size image
Fig. 11.
figure 11

Setup interface

Full size image

5 Evaluation of the Four-Dimensional Design of “Sentient Input Method”

After “sentient input method” basic determine the final prototype, this study by using four dimensions of information interaction design evaluation system model [8] on the subjective evaluation experiment, for the environment characteristics, user behavior psychology, technical properties, analyzes variable factors interact and self-evaluation, and divided into four angles. This design scheme in the dimension of “environment”: at present, there are a lot of input software, and people have been used to express emotions with emoticons in online communication. This trend of network social communication is becoming increasingly fierce, and simple text expression may not open the market in a short time. In “people” dimension: The input interface of “sentient input method” imitates the default input method of IOS system, and the modification on this basis basically does not change the user’s original input behavior mode. In the aspect of user’s psychology, it reduces the situation that users worry about the other party’s incompatibility and misunderstanding when they only use words to express themselves. In the dimension of “product”: based on the principle of 3D-touch, this design recognizes emotions by sensing the pressure of typing and presents them in the form of text. To some extent, it saves unnecessary time cost and makes communication more effective and convenient. In the dimension of “technology”: users worry that the emotion recognition is not accurate, so it is not as convenient as sending pictures directly; In the form of text presentation, everyone’s views and preferences are inconsistent, and it is currently impossible to set a complete presentation library based on big data.

In general, the prototype design of “sentient input method” has better design presentation and user evaluation in two dimensions of “product” and “person”, and there is still room for continuous optimization in two dimensions of “environment” and “technology”. Especially in the easy-to-use value, user value, communication value and other design point performance is more prominent, but in the value of popular science, aesthetic value should also be better to identify and match the user’s emotions, using more in line with the user’s expected text output form.

6 Conclusion

There was once a saying to describe human-computer interaction: the first generation relies on poke, the second on touch, the third on talk, and the fourth on wave [9]. From the way of poking and pressing to the way of realizing human-computer interaction through brainwave in the future, we can clearly see the leapfrog progress of human-computer interaction brought by the development of future technology. In fact, technology may be involved in the emergence, development and maturity of an interactive mode. The advantage of “sentient input method” lies in that it can endow users with the original written expression of the emotion they want to express, and make the communication between the two parties more close to the face-to-face communication, so as to reduce the misunderstanding caused by individual’s different ways of understanding a single text information in the form of text expression. Compared with the original way of expressing emotions through text and emoticons, it greatly simplifies the user’s operation process. The limitation is that everyone’s opinions and preferences are inconsistent. At present, it is impossible to set up a complete repository based on big data.

In the future, with the further development and improvement of big data technology and 3D-touch technology, people are expected to go beyond the single information input mode represented by text, and more diversified, efficient and interesting information interaction mode will become possible.