Keywords

1 Introduction

In recent years, automatic face recognition has made great progress, but most efforts are focused on the situations where face images are taken at a close distance with uniform illumination in the controllable scene [1]. Face recognition accuracy greatly reduced when the scene is not controllable, especially with the increase of distance and the influence of environment. In order to better illustrate the impact of external factors on image quality, face images taken at a distance of 150 m are shown in Fig. 1 and the images taken at 1, 60, 100 and 150 m are shown in Fig. 2.

Fig. 1.
figure 1

The examples of face image taken at 150 m

Full size image
Fig. 2.
figure 2

The comparison of face images taken at 1, 60, 100 and 150 m, respectively

Full size image

According to the Figs. 1 and 2, the characteristics of face images taken at a long distance can be summarized as: (1) the influence of illumination; (2) the loss of high-frequency components; (3) fewer facial pixels and detailed information. Generally, long-distance face images are captured in an uncontrolled outdoor environment. Therefore, it will be seriously affected by the illumination, as shown in the right image of Fig. 1, the face is in the shadow.

To solve the problems of image quality degradation and non-uniform illumination result from long distance, we proposed a new method to enhance the image quality and restore image detail features. In our algorithm, the nonlinear transformation is adopted, which is a combination of gamma and logarithmic transformation, giving raise to the name of the method: G-Log algorithm. By using our algorithm, both visual effect and recognition accuracy can be improved greatly.

2 Related Work

Our work is based on the LDHF database released in 2012 [2]. It contains 1-m indoor, 60-, 100-, and 150-m outdoor visible-light and near-infrared images of 100 subjects. The examples of LDHF are shown in Fig. 1. Most of the outdoor images in LDHF are influenced by the fog or the illumination, as shown in Fig. 1, the images are foggy or back-lighted.

Image enhancement is used to enhance the detail and useful information in the image, improve the visual effect, and purposefully emphasize the global or local features [3]. There are many existing works that focus on image enhancement, including MSRCR [4], MSR [5], wavelet decomposition [6], Guild filter [7], etc. All these algorithms are evaluated accordingly with respect to their performances of improving the recognition accuracy. Moreover, we have performed extensive experiments and summarized the results, according to the characteristics of the long-distance face, following which the G-Log is proposed. Subjective visual comparison is shown in Fig. 3. We can see that our G-Log algorithm shows the best results against others.

Fig. 3.
figure 3

Two examples enhanced facial images (from left to right): original 150-m images and images enhanced by G-Log, MSRCR, MSR, wavelet decomposition, Guild filter combined with dark channel.

Full size image

Recently, face recognition with Deep Learning has achieved the surprising result [8]. Therefore, to compare the difference between close-distance and long-distance images, we compare the deep feature maps. To illustrate the effect of our algorithm, we analyze the deep feature of images after enhanced by different enhancement methods, which will be described in detail in the subsequent sections.

3 Proposed G-Log Method

In this section, we firstly discuss this method in detail and analyze the effectiveness of the algorithm with respect to the improvement of image quality. Then we introduce the influence of different parameters and how to select them.

3.1 G-Log Analysis

G-Log can enhance both color and gray images. For color image, the process is identical to each channel. The algorithm is summarized in Table 1.

Table 1. The details of G-Log algorithm
Full size table

Firstly, the maximum and minimum pixel values of each color channel are got. We conduct the nonlinear transformation formulated as Eq. (2). This transformation is similar to the gamma transformation with some variations. When min = 24, max = 242, the transformation curves corresponding to different values of \( \upgamma \) are shown in Fig. 4. When \( \gamma < 1 \), the curve is convex, low pixel intensity values can be stretched, which can increase the image local contrast and compress pixel areas with high intensity values.

Fig. 4.
figure 4

The transformation curves of different values of \( \upgamma \)

Full size image

When \( \gamma > 1 \), the curve is concave, the transformation can stretch the range of high pixel intensity values and suppress low pixel intensity values. When the light is dark, the image pixel values are small and detailed information is lost in the low-light area. In such cases, we reduce the value \( \upgamma \) appropriately. On the contrary, when the light is bright, we increase the value of \( \upgamma \) appropriately. Therefore, depending on the situation of the image to be enhanced, the selection of \( \upgamma \) may be targeted.

With the increase of the distance, the low-light area information is easier to be lost than the information in the high-light area. In order to restore the darkness information as much as possible, the next step of our method is logarithm transformation as the curve shown in Fig. 5. The low intensity area is stretched and high intensity area is compressed which can better disclose the dark-area detail. However, it can be seen that logarithm transformation largely suppresses the pixel values. Therefore, in order to make up for this defect, we make a design which add to the image a constant value \( m \) before logarithm transformation.

Fig. 5.
figure 5

The logarithm transformation curve

Full size image

Finally, the image is normalized to 0-255 as defined in Eq. (4). To better illustrate our algorithm, we choose a specific example for the analysis: \( \gamma = 1.5, m = 23 \). The final transformation curve is shown in Fig. 6. The red solid line is G-Log transformation curve and the dotted line y = x is drawn for the comparison. The transformation suppresses pixel values below 90 while improving the pixel value above 90. So this transformation increases image contrast by making dark-area darker and bright-area brighter. Since the pixel distribution of most detail and edge information typically lie in 50 and 200. The pixels in the middle position are more crowed, which is not conductive to the detailed information representation. This transformation mapped pixels between 40 and 170 to 0 and 200 which stretches the middle pixels and balanced image histogram.

Fig. 6.
figure 6

The final transformation curve when \( \gamma = 1.5,\text{m} = 23 \) (Color figure online)

Full size image

3.2 Parameter Selection

Different parameters yield different enhanced image quality, thus affecting the final face recognition accuracy. We have done mounts of experiments to find the best parameter choice and how to choose parameters according to the original image quality. Figure 7 shows the relationship between the parameter \( m \) and the transformation curve. When \( \upgamma \) is fixed, the transformation curve translates upward as \( m \) increases. That is, the larger the value of \( m \), the larger image pixel value after processing. At the same time, it can be seen that as the pixel value gradually increases, the degree of pixel value increasing get smaller. This is consistent with our previous view that the information of pixel area with low levels is easier to be lost than the information in pixel area with high levels.

Fig. 7.
figure 7

The transformation curves when \( \upgamma \) is fixed and \( m \) is changed

Full size image

The relationship between parameter \( \upgamma \) and transformation curves is shown in Fig. 8. When \( \gamma < 1 \), the convex degree of curve increases as \( \upgamma \) decreases, the greater ability to stretch low pixels. When \( \gamma > 1 \), the curve translates downward as \( \upgamma \) increases and stretches middle pixels in a larger degree. So, parameter \( m \) can control the global brightness of the enhanced image and parameter \( \upgamma \) control image contrast.

Fig. 8.
figure 8

The transformation curves when \( m \) is fixed and \( \upgamma \) changes

Full size image

The increase of distance leads to low contrast of the image. If we take the influence of external factors such as illumination and weather factors out of consideration, we can properly increase the parameter \( m \) and \( \upgamma \). The image quality will be further reduced if we combined with the impact of all of the factors and the choice of parameters will be more complex. The influence of parameters is shown in Fig. 9 and the effect of parameters on similarity is shown in Table 2. The similarity is got by computing the cosine distance of face feature got from Convolutional Neural Network (CNN). Firstly, we get the similarity of 1-m and 150-m original images and then compare with the similarity of original 1-m and 150-m images enhanced by G-Log. From Table 2, the original image similarity of 1 m and 150 m is 0.6941. And the face image similarity can be up to 0.8218 after enhanced. Empirically, the optimal result can be attained when \( \gamma \in \left( {0.9,1.5} \right),\text{m} \in (14,26) \).

Fig. 9.
figure 9

Images enhanced with different parameters, the left column is original images and the rest is enhanced images with different parameters.

Full size image
Table 2. The similarities between 150-m images enhanced by different parameters and corresponding 1-m original images.
Full size table

4 Experimental Results and Analysis

The proposed G-Log image enhancement algorithm is evaluated by face recognition accuracy, histogram and CNN feature map [9] in this section. The database we use is LDHF, including 1-, 60-, 100-, 150-m images and face recognition method is SeetaFace Engine [10]. For the convenience, long-distance (60, 100, 150 m) face images and short distance (1 m) face images for matching are called probe and gallery images respectively. Given a probe image, the procedure of face alignment [11], enhancement and then matching is executed against all gallery images.

To compare the effect of different enhancement algorithms, all probe images are processed by different enhancement methods and the identical face alignment, recognition modules. Face recognition accuracy is shown in Table 3 and accuracy comparison of different enhancement methods is shown in Fig. 10. It can be seen that the face matching accuracy of original 150 m to 1 m is only 70% while 60 m to 1 m is up to 95%. Therefore, distance and environmental factors make a seriously influence on the face recognition. As the distance increases, recognition rate decreases significantly. After processed by G-Log algorithm, the 150-m and 100-m recognition rate are greatly improved, from 70% to 95% and 89% to 98% respectively. Compared with other algorithms involved in Fig. 10, G-Log method achieves the greatest improvement in face recognition rate. In addition, the visual effect also realizes the best result against other methods as shown in Fig. 3.

Table 3. Face recognition accuracy about different enhancement methods
Full size table
Fig. 10.
figure 10

Comparison of the 150-, 100-, 60-to-1 m face matching accuracy with different enhancement methods.

Full size image

For objective performance evaluation, we compare the Cumulative Match Characteristic (CMC) curve of the matching result under different enhancement algorithms in Fig. 11. From this figure, it can be seen that G-Log algorithm achieves the best result both in 100-m and 150-m matching accuracy. The first rank recognition rate is 95% and 98% for 150-m and 100-m face images respectively and it rapidly climbs up to 98% and 98% in rank 5.

Fig. 11.
figure 11

CMC curves of the 100-, 150-to-1 m face matching result enhanced with different methods.

Full size image

The histogram comparison is shown in Fig. 12. Except for G-Log method, MSRCR achieves the best result in face recognition accuracy in our experiment. Therefore, G-Log method is compared with MRSCR in the following experiments. It can be seen that the image histogram distribution is more uniform and the contrast is improved, after processed by G-Log method, which is helpful to the restoration of details and edge information. This method basically preserves the shape of histogram, and it does not change the corresponding relationships between pixels, so no additional noise is added.

Fig. 12.
figure 12

The histogram comparison of G-Log and MSRCR

Full size image

In order to illustrate the effect of the G-Log algorithm on image detail recovery, the deep feature maps are shown in Fig. 13. The corresponding position of different sub-images is the same feature and the different positions of each sub-image are different feature maps. So there are 30 different feature maps of a subject shown in Fig. 13. The same subject face is used in four sub-images. Compared with the 1-m image, some of the detailed and edge information are lost in feature map of the original 150-m image. The eyes, mouth and nose features appearing in the 1-m feature map are completely degraded in the 150-m feature map as shown in circle areas in Fig. 13(a) and (b). From the sub-image (c), after processed by G-Log method, the information lost in 150 m is restored greatly. The sub-image (d) in Fig. 13 is the 150-m feature map of image enhanced by MSRCR, lost information doesn’t get recovery and some noises are produced in the edge of the image.

Fig. 13.
figure 13

The feature maps of deep convolutional neural net: (a), (b) are original 1-m, 150-m feature maps respectively, and (c), (d) are 150-m feature maps of image enhanced by G-Log and MSRCR respectively.

Full size image

5 Conclusion

With the increase of distance and environmental factors, non-uniform illumination, low resolution, and the influence of weather lead to a significant reduction of face recognition rate. The face matching accuracy of original 150 m and 1 m is only 70% while 60 m and 1 m is up to 95%. An effective face image enhancement algorithm G-Log has been proposed in this paper to solve these problems. By using the G-Log algorithm, the face recognition accuracy is greatly improved, from 70%, 89%, 95%, to 95%, 98%, 98% for 150-, 100-, 60-m, and the edge information, details are restored. Experiments demonstrate and confirm the effectiveness of the proposed method.