Skip to main content
SpringerLink
Log in

Short-Term Person Re-identification Using RGB, Depth and Skeleton Information of RGB-D Sensors

  • Research Paper
  • Published:
Iranian Journal of Science and Technology, Transactions of Electrical Engineering Aims and scope Submit manuscript

Abstract

Person re-identification is to search for a correct match for a person of interest across different camera views among a large number of impostors. While the approaches based on RGB modality have been widely studied to re-identify people, other modalities could be exploited as additional information sources, like depth and skeleton modalities. In this paper, we perform multi- and single-modal person re-identification using RGB, depth and skeleton modalities obtained by RGB-D sensors. First of all, the depth and RGB images are divided into three regions of head, torso and legs. Then, each region is explained by the histograms of local vector patterns (LVP). In depth and RGB modalities, the depth values of pixels and the gray levels of pixels are used for extracting LVPs, respectively. The skeleton features are extracted by computing the various Euclidean distances for the joint points of skeleton images. Then, features extracted by different modalities are combined as double and triple combinations using score-level fusion. The experiments are evaluated on two RGBD-ID and KinectREID databases, and results illustrate the acceptable performance of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Barbosa BI, Cristani M, Bue AD, Bazzani L, Murino V (2012) Re-identification with RGB-D sensors. In: 1st international workshop on re-identification, pp 433–442

  • Bazzani L, Cristani M, Perina A, Murino V (2012) Multiple-shot person re-identification by chromatic and epitomic analyses. Pattern Recognit Lett 33:898–903

    Article  Google Scholar 

  • Bazzani L, Cristani M, Murino V (2013) Symmetry-driven accumulation of local features for human characterization and re-identification. Comput Vis Image Underst 117:130–144

    Article  Google Scholar 

  • Chahla C, Snoussi H, Abdallah F, Dornaika F (2017) Discriminant quaternion local binary pattern embedding for person reidentification through prototype formation and color categorization. Eng Appl Artif Intell 58:27–33

    Article  Google Scholar 

  • Doretto G, Sebastian Th, Tu P, Rittscher J (2011) Appearance-based person reidentification in camera networks: problem overview and current approaches. J Ambient Intell Hum Comput 2:127–151

    Article  Google Scholar 

  • Fan KCh, Hung TY (2014) A novel local pattern descriptor—local vector pattern in high-order derivative space for face recognition. IEEE Trans Image Process 23:2877–2891

    Article  MathSciNet  Google Scholar 

  • Fang W, Hu HM, Hu Z, Liao Sh, Li B (2017) Perceptual hash-based feature description for person re-identification. Neurocomput. https://doi.org/10.1016/j.neucom.2017.07.019

    Article  Google Scholar 

  • Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2013) Symmetry-driven accumulation of local features for human characterization and re-identification. Comput Vis Image Underst 117:130–144

    Article  Google Scholar 

  • Fathi A, Nilchi AN (2012) Noise tolerant local binary pattern operator for efficient texture analysis. Pattern Recognit Lett 33:1093–1100

    Article  Google Scholar 

  • Forssen PE (2007) Maximally stable colour regions for recognition and matching. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8

  • Gala AB, Shah ShK (2014) A survey of approaches and trends in person re-identification. Image Vis Comput 32:270–286

    Article  Google Scholar 

  • Gala AB, Shah ShK (2011) Multiple person re-identification using part based spatio-temporal color appearance model. In: IEEE International conference on computer vision workshops (ICCV Workshops), pp 1721–1728

  • Guo Z, Zhang L, Zhang D (2010a) A completed modeling of local binary pattern operator for texture classification. IEEE Trans Image Process 19:1657–1663

    Article  MathSciNet  Google Scholar 

  • Guo Z, Zhang L, Zhang D (2010b) Rotation invariant texture classification using LBP variance (LBPV) with global matching. Pattern Recognit 43:706–719

    Article  Google Scholar 

  • Hadid A (2008) The local binary pattern approach and its applications to face analysis. In: 1st Workshop on image processing theory, tools and applications (IPTA), pp 1–9

  • Huang X, Li SZ, Wang Y (2004) Shape localization based on statistical method using extended local binary pattern. In: Proceedings of the 3rd international conference on image and graphics (ICIG), pp 184–187

  • Huang D, Shan C, Ardabilian M, Wang Y, Chen L (2011) Local binary patterns and its application to facial image analysis: a survey. IEEE Trans Syst Man Cybern Part C 41:765–781

    Article  Google Scholar 

  • Imani Z, Soltanizadeh H (2016) Person reidentification using local pattern descriptors and anthropometric measures from videos of kinect sensor. IEEE Sensors J 16:6227–6238

    Article  Google Scholar 

  • Imani Z, Soltanizadeh H (2018) Histogram of the node strength and histogram of the edge weight: two new features for RGB-D person re-identification. Sci China Inf Sci 61:092108. https://doi.org/10.1007/s11432-016-9086-8

    Article  Google Scholar 

  • Imani Z, Soltanizadeh H (2019) Local binary pattern, local derivative pattern and skeleton features for RGB-D person re-identification. Natl Acad Sci Lett 42:233–238. https://doi.org/10.1007/s40009-018-0736-9

    Article  Google Scholar 

  • Jin H, Liu Q, Lu H, Tong X (2004) Face detection using improved LBP under Bayesian framework. In: Proceedings of the 3rd international conference on image and graphics, pp 306–309

  • Krucki K, Asari V, Borel-Donohue C, Bunker D (2014) Human re-identification in multi-camera systems. In: IEEE applied imagery pattern recognition workshop (AIPR), pp 1–7

  • Layne R, Hospedales T, Gong S (2012) Person re-identification by attributes. In: Proceedings of the British machine vision conference (BMVC)

  • Liu C, Gong S, Loy CC, Lin X (2012) Person re-identification: What features are important? In: Proceedings of the 1st international workshop on re-identification, pp 391–401

  • Liu X, Song M, Tao D, Zhou X, Chen C, Bu J (2014) Semi-supervised coupled dictionary learning for person re-identification. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 3550–3557

  • Lowe DG (2004) Distinctive image features from scale-invariant key-points. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  • Ma B, Su Y, Jurie F (2014) Covariance descriptor based on bio-inspired features for person re-identification and face verification. Image Vis Comput 32:379–390

    Article  Google Scholar 

  • Martinel N, Micheloni C (2012) Re-identify people in wide area camera network. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW), RI, USA, pp 31–36

  • Munaro M, Fossati A, Basso A, Menegatti E, Gool LV (2014) One-shot person re-identification with a consumer depth camera. In: Advances in computer vision and pattern recognition. Springer, Berlin, pp 161–181

  • Murala S, Wu QMJ (2015) Spherical symmetric 3D local ternary patterns for natural, texture and biomedical image indexing and retrieval. Neurocomput 149:1502–1514

    Article  Google Scholar 

  • Murala S, Maheshwari RP, Balasubramanian R (2012) Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans Image Process 21:2874–2886

    Article  MathSciNet  Google Scholar 

  • Nanni L, Lumini A, Brahnam S (2010) Local binary patterns variants as texture descriptors for medical image analysis. Artif Intell Med 49:117–125

    Article  Google Scholar 

  • Nanni L, Brahnam S, Lumini A (2012) A simple method for improving local binary patterns by considering non-uniform patterns. Pattern Recognit 45:3844–3852

    Article  Google Scholar 

  • Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recognit 29:51–59

    Article  Google Scholar 

  • Oliveira IO, Pio JL (2009) People re-identification in a camera network. In: IEEE international conference on dependable, autonomic and secure computing (DASC), Chengdu, China, pp 461–466

  • Pala F, Satta R, Fumera G, Roli F (2015) Multi-modal person re-identification using RGB-D cameras. IEEE Trans Circuit Syst Video Tech 26:788–799

    Article  Google Scholar 

  • Prosser B, Zheng WS, Gong S, Xiang T (2010) Person re-identification by support vector ranking. In: Proceedings of the British machine vision conference (BMVA), pp 21.1–21.11

  • Ross A, Jain A (2003) Information fusion in biometrics. Pattern Recognit Let 24:2115–2125

    Article  Google Scholar 

  • Schwartz WR, Davis LS (2009) Learning discriminative appearance-based models using partial least squares. In: 22nd Brazilian symposium on computer graphics and image processing (SIBGRAPI), pp 322–329

  • Sun N, Zheng W, Sun C, Zou C, Zhao L (2006) Gender classification based on boosting local binary pattern. In: Proceedings of 3rd international symposium on neural networks, pp 194–201

  • Tao D, Guo Y, Song M, Li Y, Yu Zh, Tang YY (2016) Person re-identification by dual-regularized KISS metric learning. IEEE Trans Image Process 25:2726–2738

    Article  MathSciNet  Google Scholar 

  • Vezzani R, Baltieri D, Cucchiara R (2014) People reidentification in surveillance and forensics: a survey. ACM Comput Surveys 46:29

    Google Scholar 

  • Wu L, Shen C, Hengel A (2017) Deep linear discriminant analysis on Fisher networks: a hybrid architecture for person re-identification. Pattern Recognit 65:238–250

    Article  Google Scholar 

  • Xiang ZJ, Chen Q, Liu Y (2014) Person re-identification by fuzzy space color histogram. Multimed Tools Appl 73:91–107

    Article  Google Scholar 

  • Xiong F, Gou M, Camps O, Sznaier M (2014) Person re-identification using kernel-based metric learning methods. In: 13th European conference on computer vision (ECCV), pp 1–16

  • Yang B, Chen S (2013) A comparative study on local binary pattern (LBP) based face recognition: LBP histogram versus LBP image. Neurocomput 120:365–379

    Article  Google Scholar 

  • Zhang Y, Li S (2011) Gabor-lbp based region covariance descriptor for person re-identification. In: Sixth International Conference on Image and Graphics (ICIG), pp 368–371

  • Zhang B, Gao Y, Zhao S, Liu J (2010) Local derivative pattern versus local binary pattern: face recognition with high order local pattern descriptor. IEEE Trans Image Process 19:533–544

    Article  MathSciNet  Google Scholar 

  • Zhao R, Ouyang W, Wang X (2013) Unsupervised salience learning for person re-identification. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 3586–3593

  • Zhao Y, Zhao X, Luo R, Liu Y (2015) Person Re-identification by encoding free energy feature maps. Multimed Tools Appl 75:4795–4813

    Article  Google Scholar 

  • Zheng W, Gong S, Xiang T (2011) Person re-identification by probabilistic relative distance comparison. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 649–656

  • Zheng W, Gong S, Xiang T (2013) Reidentification by relative distance comparison. IEEE Trans Pattern Anal Mach Intell 35:653–668

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering Faculty, Semnan, Iran

    Zeynab Imani, Hadi Soltanizadeh & Ali A. Orouji

Authors
  1. Zeynab Imani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hadi Soltanizadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ali A. Orouji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hadi Soltanizadeh.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Imani, Z., Soltanizadeh, H. & Orouji, A.A. Short-Term Person Re-identification Using RGB, Depth and Skeleton Information of RGB-D Sensors. Iran J Sci Technol Trans Electr Eng 44, 669–681 (2020). https://doi.org/10.1007/s40998-019-00249-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40998-019-00249-9

Keywords

Search

Navigation