Secure identity authentication is important in defense technology fields and public security. As a novel authentication mode, biometric recognition has been popular due to its notable advantages compared with the traditional authentication modes [1, 2]. Currently biometric samples can be directly acquired and used for authentication on mobile devices. Palmprint is a good biometric modality due to its rich texture information [3, 4], relatively low resolution requirement, low cost of acquisition equipment, etc. [5, 6]. In addition, palmprint can be fused with other hand biometric modalities [7–9] and used for multi-instance fusion .
Palmprint image acquisition can be briefly divided into contact and non-contact modes . In non-contact mode, users do not need to contact device surface, and the operation is relatively convenient . Although it improves the acquisition flexibility to a certain extent, it suffers from more severe challenges than contact mode, including background complexity, illumination variance, uncontrollable hand posture and position and so on . The environment of contact image collection is ideal, so the contact palmprint preprocessing methods cannot be used directly in non-contact mode due the aforementioned challenges.
The palmprint image can be captured with the built-in camera of the mobile device, and the hand does not need to contact the device. Therefore, the mobile mode can be regarded as a special case of non-contact mode [14, 15]. Since mobile devices can be used almost anytime anywhere, the challenges existing in non-contact mode are more remarkable in mobile mode . In addition, due to the limited resources on the mobile device, the computational complexity and storage overhead cannot be too high [17, 18].
In order to reduce the complexity of the segmentation and localization, some researchers designed assistant image acquisition techniques . Shoichiro et al. designed an assistant rectangle guide window to help users place their hands . Similarly, Kim et al. designed an assistant hand-shaped guide windows . Ramli et al. designed a hand template to control the appropriate distance between the hand and the input device . In , Lee et al. designed three inter-finger-valley (IFV) assistant lines to help users place their hands. In 2017, Leng et al. designed two assistant points to align the IFV key points (KPs)  and implemented the scheme on iOS mobile platform . In 2018, Leng et al. designed a dual-line-single-point acquisition method  and implemented the scheme on Android mobile platform . The dual-line-single-point assistant graphs include two parallel line segments with different lengths, and an anchor point. This method requires few restrictions and operations, and has a comfortable user experience. However, the Region of Interest (ROI) is directly cropped with assistant graph, so this method requires a high degree of user cooperation.
Palmprint preprocessing is important, because its segmentation and localization affect the accuracy of feature extraction and matching. Non-contact palmprint preprocessing methods can be summarized as follows:
Skin color model: In YCbCr, HSV, CIELab and other color spaces, segmentation and localization methods are based on hand skin color. In order to improve the robustness of the segmentation to illumination changes, Rotinwaakinbile et al. proposed a skin color neural network model . Zhao et al. first used the ellipse skin color model for skin color detection, and then performed secondary detection on skin color based on the clustering of split K-means . In 2017, Zhang et al. proposed an adaptive segmentation method based on Gaussian skin color model , but the method is susceptible to skin-like color, which leads to over- or under-segmentation.
Maximum inscribed circle: A maximum inscribed circle is detected as ROI, which is tangent to both sides of the hand and contains rich palm information [31, 32]. The disadvantage of this method is that it takes a lot of time to find the center of the largest inscribed circle.
Statistical model: Active Appearance Model (AAM) and Active Shape Model (ASM) are two typical statistical models . Fratric et al. proposed a real-time hand localization method based on Viola-Jone approach . Aykut et al. obtained prior knowledge through the shape and texture observed in a training set, and segmented the hand shape [35,36]. AAM method has a strong ability to resist the changes in hand posture and background, so it improves the accuracy and robustness of the segmentation algorithm. However, this method mixes two complex features of shape and texture, so the calculation overhead is large.
Projection: Huang et al. performed vertical projection on the segmented binary image. The length of the finger affects the accuracy of KP localization .
Tangent model: Zhang et al. used a line tangent to two IFV contours to localize IFV KPs .
Competitive hand valley detection (CHVD): This method performs four conditions on all pixels in the hand image. Only the pixels satisfying four conditions at the same time are determined to be candidate KPs .
Triple-perpendicular-directional Translation Residual (TPDTR): Leng et al. proposed TPDTR logical connection to detect IFVs. These three vertical directions are the direction of the middle fingertip and its two perpendicular directions [40,41]. However, when the background is complicated, some false IFVs could be detected.
In addition to the above palmprint KP localization methods, there are also some other methods based on radial distance function [42, 43], line fitting [44, 45], and morphological corner detection . However, these methods could have low localization accuracy in complex environment, or low applicable range. When the KP localizations are not accurate, subsequent operations, such as feature extraction [47, 48] and coding [49, 50], could be degraded.
In order to overcome the disadvantages of the above methods, this paper designs a method of KP localization based on intersecting circle (IC). This method is good at removing false contour and anti-interference, so both the localization accuracy and recognition accuracy are high.
Hand region and contour are segmented with an improved Gaussian skin-color model . According to the characteristic of large contour curvature at IFV, we design a method of KP localization based on IC. The larger the contour curvature is, the smaller the central angle is. According to this characteristic, the contours of fingertips and IFVs are retained. Then, IFV direction range is used to exclude fingertips and false IFVs. Finally, the lowest midpoint of IFV is determined as the KP.
We use the I-shaped guide assist method to take hand images, which is shown in Fig. 1. The coordinate system is established on the image, with the upper left corner as the origin. The positive directions of X-axis and Y-axis are horizontally right and vertically down, respectively. The length and width of hand image are H and W, respectively. The implementation steps are as follows:
Step 1: Extract edge.
Sobel operator is used to extract edge.
Step 2: Remove wrist.
The wrist affects on hand contour extraction, so it needs to be removed. As shown in Fig. 1, middle finger length to palm length ratio is about 3:4, AB:BC is 5:3, and the y coordinate of B is:
Step 3: Extract intersection points.
Each contour point is used as the center of an IC. The width occupied by each finger is about W/8~W/10. The selected IC radius should be smaller than the finger width. Therefore, the radius of an IC r < W/10, so we set r = W/13. (x0, y0) represents the central of IC, whose intersection points satisfy:
Where (x, y) represents the coordinates of contour points.
Step 4: Count the number of intersection points.
n denotes the number of intersection points between the IC and contour. Whether the point is retained depends on the value of n.
If n = 1, the point is close to the contour endpoints on the either side of wrist.
If n = 2, the point is at the fingertip or IFV contour.
If n = 3 or 4, the point is at the contour of finger body, because the distance between two adjacent fingers is less than the IC radius.
If n ≥ 5, the hand is abnormally collected, or there are major defects in the segmentation and contour extraction.
The contour points with different values of n are shown in Fig. 2. The contour points with n = 2 are retained.
A contour point is set as P, and the points where IC intersects with hand contour are E and F. If ∠EPF < 135°, keep the contour point P, otherwise, remove it.
IFV and fingertip contours have “U” or “V” shape characteristics. According to the central angle bisector direction of different contours, the fingertip contours and the false IFV contours are excluded.
In Fig. 3(a), Arrows 1, 2 and 3 respectively indicate central angle bisector direction ranges of fingertips, false IFV, and true IFV . The contour is removed when all its points satisfy that the angle between the central angle bisector and the vertical direction is larger than 75°. Finally, the four IFVs with a few interference points as shown in Fig. 3(b) are retained.
After preprocessing, there may be few unsatisfactory points in IFV contours, which result in the fracture of the IFV. In order to connect the broken IFV contour and keep the shape unchanged, a 3 × 3 template is used to perform a morphological close operation on all contour points. At last, keep the four longest contours as the IFV contours.
III. EXPERIMENTS AND DISCUSSION
The palmprint samples are acquired according to the I-shaped guide assist method. 5 to 7 images of each hand are collected. The database contains 918 samples of 160 hands of 80 people. The ratio of male to female is about 7:3, and the age is between 18 and 60. Most individuals are undergraduates. The collection mobile device is Huawei G750-t01. The collection environments include indoor, outdoor, various illuminations, even no illumination conditions.
Fig. 4 shows the palmprint images taken under different backgrounds. Several KP localization methods are compared. Red dots and blue boxes indicate KPs and ROIs.
In ideal background (Fig. 4(a)), all methods have good results. In the background with skin-like color (Fig. 4(b)), KP localization in our method is more accurate than others. In complex background (Fig. 4(c)), only our method succeeds to localize KP and ROI.
Our method has a lower error rate. In complex backgrounds, projection method and the tangent method have a weak ability to resist background interference. CHVD is easily affected by texture near IFVs, which results in localization errors. TPDTR method has good stability, but the “U” and “V” shape contours in complex background would lead to detect false IFV contour.
We judge the accuracy of KP localization with correct rate. The lowest points of the IFVs between the index finger and the middle finger as well as the ring finger and the little finger are manually labeled as the standard KPs. The images are resized to 600 × 338. When the distance between the KP and the marked point is within 5 pixels, the localization is judged to be correct. In Table 1, The correct rate of our method is 96.3%, which is the higher than the compared methods.
Equal error rate (EER), d’, and receiver operating characteristic (ROC) are used as palmprint verification performance indicators. The palmprint feature templates are Binary Orientation Co-occurrence Vector , which is a good coding scheme  and has good verification performance thanks for the fusion at score level [54, 55]. The experimental results are shown in Table 2 and Fig. 5.
In summary, our KP localization based on IC has higher recognition accuracy than the compared methods, and stronger ability to resist background interference.
Uncontrollable palm placement and other factors in mobile acquisition environment can easily lead to KP localization failure. KP localization based on IC first selects the appropriate radius and center angle threshold value to exclude contours with larger central angle, and then uses the central angle direction formed by IC and IFV contour to exclude fingertip contours and false IFV contours. A morphological close operation is performed to repair the IFV contours. According to the experimental results, this method is a practical scheme for palmprint verification in mobile environments.