Skip to main content

Advertisement

SpringerLink
Log in

I see, you design: user interface intelligent design system with eye tracking and interactive genetic algorithm

  • Regular Paper
  • Published:
CCF Transactions on Pervasive Computing and Interaction Aims and scope Submit manuscript

Abstract

User interfaces are important for streamlining the interactions between humans and computers. However, there are few effective approaches for collecting users’ preferences implicitly and objectively for the purpose of user interface (UI) design optimization. This paper presents an effective approach to interactive genetic algorithm (IGA) optimization-based UI design via eye tracking, including eye-movement data based users’ preference inferring, gene coding for real UI components and design features, and the visualization and interaction mechanisms. Then we design and build a prototype system that applies IGA to generate and optimize music player UI solutions automatically. An evaluation of our prototype system suggests that it can generate and identify personalized UI designs reliably with minimal user intervention and high efficiency and user satisfaction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Anderson, J.R.: Cognitive Psychology and its Implications. Macmillan, Basingstoke (2009)

    Google Scholar 

  • Baldassi, S., Burr, D.C.: ‘Pop-out’ of targets modulated in luminance or colour: the effect of intrinsic and extrinsic uncertainty. Vis. Res. 44(12), 1227–1233 (2004)

    Article  Google Scholar 

  • Borji, A., Itti, L.: State-of-the-art in visual attention modeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 185–207 (2013)

    Article  Google Scholar 

  • Brabrand, G.B.: Dynamic layout optimization for newspaper web sites using a controlled annealed genetic algorithm. Dissertation, Gjøvik University College (2008)

  • Brochu, E., Brochu, T., de Freitas, N.: A Bayesian interactive optimization approach to procedural animation design. In: Proc. ACM SIGGRAPH/eurographics symposium on computer animation, pp. 103–112 (2010)

  • Cheng, S., Liu, Y.: Eye-tracking based adaptive user interface: implicit human-computer interaction for preference indication. J. Multimodal User Interfaces 5(1-2), 77–84 (2012)

    Article  Google Scholar 

  • Cheng, S., Liu, X., Yan, P., Zhou, J., Sun, S.: Adaptive user interface of product recommendation based on eye-tracking. In: Proc. EGIHMI, pp. 94–10 (2010)

  • De Jong, K.A.: An analysis of the behavior of a class of genetic adaptive systems. Dissertation. University of Michigan (1975)

  • Eisenstein, J., Puerta, A.: Adaptation in automated user-interface design. In: Proc. IUI. ACM Press, New York, pp. 74–81 (2000)

  • Engelbrecht, A.P.: Computational Intelligence: An Introduction, 2nd edn. Wiley, New York (2007)

    Book  Google Scholar 

  • Gajos, K.Z., Weld, D.S., Wobbrock, J.O.: Automatically generating personalized user interfaces with Supple. Artif. Intell. 174(12), 910–950 (2010)

    Article  Google Scholar 

  • Glaholt, M.G., Reingold, E.M.: Stimulus exposure and gaze bias: a further test of the gaze cascade model. Atten. Percept. Psychophys. 71(3), 445–450 (2009)

    Article  Google Scholar 

  • Goldberg, J.H., Kotval, X.P.: Computer interface evaluation using eye movements: methods and constructs. Int. J. Ind. Ergon. 24(6), 631–645 (1999)

    Article  Google Scholar 

  • Gong, D., Yao, X., Yuan, J.: Interactive genetic algorithms with individual fitness not assigned by human. J. Univ. Comput. Sci. 15(13), 2446–2462 (2009)

    Google Scholar 

  • Holmes, T., Zanker, J.: Eye on the prize: using overt visual attention to drive fitness for interactive evolutionary computation. In: Proc. GECCO. ACM Press, New York, pp. 1531–1538 (2008)

  • Holmes, T., Zanker, J.: Using an oculomotor signature as an indicator of aesthetic preference. i-Perception 3(7), 426–439 (2012)

    Article  Google Scholar 

  • Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., Van de Weijer, J.: Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford University Press, Oxford (2011)

    Google Scholar 

  • Ito, F., Hiroyasu, T., Miki, M., Yokouchi, H.: Discussion of offspring generation method for interactive genetic algorithms with consideration of multimodal preference. In: Proc. SEAL, 2008, pp. 349–359

  • Itti, L., Koch, C.: Computational modeling of visual attention. Nat. Rev. Neurosci. 2(3), 194–203 (2001)

    Article  Google Scholar 

  • Kelly, J., Papalambros, P.Y., Seifert, C.M.: Interactive genetic algorithms for use as creativity enhancement tools. In: Proc. AAAI Spring Symposium: Creative Intelligent Systems, pp. 34–39 (2008)

  • Kim, H.S., Cho, S.B.: Development of an IGA-based fashion design aid system with domain specific knowledge. Proc. IEEE SMC 3, 663–668 (1999)

    Google Scholar 

  • Kim, H.S., Cho, S.B.: Application of interactive genetic algorithm to fashion design. Eng. Appl. Artif. Intell. 13(6), 635–644 (2000)

    Article  Google Scholar 

  • Kitamura, S., Kanoh, H.: Developing support system for making posters with interactive evolutionary computation. Proc. ISCID 1, 48–51 (2011)

    Google Scholar 

  • Koivunen, K., Kukkonen, S., Lahtinen, S., Rantala, H., Sharmin, S.: Towards deeper understanding of how people perceive design in products. In: Proc. CADE (2004)

  • Lee, J.H., Kim, H.S., Cho, S.B.: Accelerating evolution by direct manipulation for interactive fashion design. In: Proc. ICCIMA, pp. 343–347 (2001)

  • Liapis, A., Yannakakis, G.N., Togelius, J.: Adapting models of visual aesthetics for personalized content creation. IEEE Trans. Comput. Intell. AI Games 4(3), 213–228 (2012)

    Article  Google Scholar 

  • Majaranta, P., Räihä, K.J.: Twenty years of eye typing: systems and design issues. In: Proc. ETRA, pp. 15–22 (2002)

  • Michalek, J., Choudhary, R., Papalambros, P.: Architectural layout design optimization. Eng. Optim. 34(5), 461–484 (2002)

    Article  Google Scholar 

  • Monmarche, N., Nocent, G., Venturini, G., Santini, P.: On generating html style sheets with an interactive genetic algorithm based on gene frequencies. In: Proc. AE, pp. 99–110 (1999)

  • Nothdurft, H.C.: Salience of feature contrast. In: Neurobiology of Attention, pp. 233–239 (2005)

    Chapter  Google Scholar 

  • Pallez, D., Brisson, L., Baccino, T.: Towards a human eye behavior model by applying Data Mining Techniques on Gaze Information from IEC. In: Proc. HCP, pp. 51–64 (2008)

  • Pallez, D., Collard, P., Baccino, T., Dumercy, L.: Eye-tracking evolutionary algorithm to minimize user fatigue in IEC applied to interactive one-max problem. In: Proc. GECCO, pp. 2883–2886, ACM Press, New York (2007)

  • Pallez, D., Cremene, M., Baccino, T., Sabou, O.: Analyzing human gaze path during an interactive optimization task. In: Proc. EGIHMI, pp. 12–19 (2010)

  • Peng, C.J., Lee, K.L., Ingersoll, G.M.: An introduction to logistic regression analysis and reporting. J. Educ. Res. 96(1), 3–14 (2002)

    Article  Google Scholar 

  • Quiroz, J.C., Banerjee, A., Louis, S.J., Dascalu, S.M.: Document design with interactive evolution. In: New Directions in Intelligent Interactive Multimedia Systems and Services-2, pp. 309–319 (2009)

    Chapter  Google Scholar 

  • Semet, Y.: Interactive Evolutionary Computation: a Survey of Existing Theory. University of Illinois, Illinois (2002)

    Google Scholar 

  • Shimojo, S., Simion, C., Shimojo, E., Scheier, C.: Gaze bias both reflects and influences preference. Nat. Neurosci. 6(12), 1317–1322 (2003)

    Article  Google Scholar 

  • Takagi, H.: Interactive evolutionary computation: fusion of the capacities of EC optimization and human evaluation. Proc. IEEE 89(9), 1275–1296 (2001)

    Article  Google Scholar 

  • Tzanidou, E., Petre, M., Minocha, S., Grayson, A.: Combining eye tracking and conventional techniques for indications of user-adaptability. In: Proc. INTERACT, LNCS, vol. 3585, pp. 753–766 (2005)

Download references

Acknowledgements

We thank all the participants who took part in the pre-study and the user study. This research was sponsored by the National Natural Science Foundation of China (61772468). We also appreciate all the reviewers for their constructive comments for this paper.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, 310023, China

    Shiwei Cheng

  2. Information School, University of Washington, Seattle, WA, 98195-2840, USA

    Anind K. Dey

Authors
  1. Shiwei Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anind K. Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiwei Cheng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, S., Dey, A.K. I see, you design: user interface intelligent design system with eye tracking and interactive genetic algorithm. CCF Trans. Pervasive Comp. Interact. 1, 224–236 (2019). https://doi.org/10.1007/s42486-019-00019-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42486-019-00019-w

Keywords

Search

Navigation