Key Point Localization Based on Intersecting Circle for Palmprint Preprocessing in Public Security

In order to reduce the complexity of the segmentation and localization, some researchers designed assistant image acquisition techniques [19]. Shoichiro et al. designed an assistant rectangle guide window to help users place their hands [20]. Similarly, Kim et al. designed an assistant hand-shaped guide windows [21]. Ramli et al. designed a hand template to control the appropriate distance between the hand and the input device [22]. In [23], Lee et al. designed three inter-finger-valley (IFV) assistant lines to help users place their hands. In 2017, Leng et al. designed two assistant points to align the IFV key points (KPs) [24] and implemented the scheme on iOS mobile platform [25]. In 2018, Leng et al. designed a dual-line-single-point acquisition method [26] and implemented the scheme on Android mobile platform [27]. The dual-line-single-point assistant graphs include two parallel line segments with different lengths, and an anchor point. This method requires few restrictions and operations, and has a comfortable user experience. However, the Region of Interest (ROI) is directly cropped with assistant graph, so this method requires a high degree of user cooperation.

Palmprint preprocessing is important, because its segmentation and localization affect the accuracy of feature extraction and matching. Non-contact palmprint preprocessing methods can be summarized as follows:

Skin color model: In YCbCr, HSV, CIELab and other color spaces, segmentation and localization methods are based on hand skin color. In order to improve the robustness of the segmentation to illumination changes, Rotinwaakinbile et al. proposed a skin color neural network model [28]. Zhao et al. first used the ellipse skin color model for skin color detection, and then performed secondary detection on skin color based on the clustering of split K-means [29]. In 2017, Zhang et al. proposed an adaptive segmentation method based on Gaussian skin color model [30], but the method is susceptible to skin-like color, which leads to over- or under-segmentation.

Maximum inscribed circle: A maximum inscribed circle is detected as ROI, which is tangent to both sides of the hand and contains rich palm information [31, 32]. The disadvantage of this method is that it takes a lot of time to find the center of the largest inscribed circle.

Statistical model: Active Appearance Model (AAM) and Active Shape Model (ASM) are two typical statistical models [33]. Fratric et al. proposed a real-time hand localization method based on Viola-Jone approach [34]. Aykut et al. obtained prior knowledge through the shape and texture observed in a training set, and segmented the hand shape [35,36]. AAM method has a strong ability to resist the changes in hand posture and background, so it improves the accuracy and robustness of the segmentation algorithm. However, this method mixes two complex features of shape and texture, so the calculation overhead is large.

Projection: Huang et al. performed vertical projection on the segmented binary image. The length of the finger affects the accuracy of KP localization [37].

Tangent model: Zhang et al. used a line tangent to two IFV contours to localize IFV KPs [38].

Competitive hand valley detection (CHVD): This method performs four conditions on all pixels in the hand image. Only the pixels satisfying four conditions at the same time are determined to be candidate KPs [39].

Triple-perpendicular-directional Translation Residual (TPDTR): Leng et al. proposed TPDTR logical connection to detect IFVs. These three vertical directions are the direction of the middle fingertip and its two perpendicular directions [40,41]. However, when the background is complicated, some false IFVs could be detected.

In addition to the above palmprint KP localization methods, there are also some other methods based on radial distance function [42, 43], line fitting [44, 45], and morphological corner detection [46]. However, these methods could have low localization accuracy in complex environment, or low applicable range. When the KP localizations are not accurate, subsequent operations, such as feature extraction [47, 48] and coding [49, 50], could be degraded.

In order to overcome the disadvantages of the above methods, this paper designs a method of KP localization based on intersecting circle (IC). This method is good at removing false contour and anti-interference, so both the localization accuracy and recognition accuracy are high.

Hand region and contour are segmented with an improved Gaussian skin-color model [25]. According to the characteristic of large contour curvature at IFV, we design a method of KP localization based on IC. The larger the contour curvature is, the smaller the central angle is. According to this characteristic, the contours of fingertips and IFVs are retained. Then, IFV direction range is used to exclude fingertips and false IFVs. Finally, the lowest midpoint of IFV is determined as the KP.

2.2.1. Obtaining inter-finger-valley and fingertip contours

We use the I-shaped guide assist method to take hand images, which is shown in Fig. 1. The coordinate system is established on the image, with the upper left corner as the origin. The positive directions of X-axis and Y-axis are horizontally right and vertically down, respectively. The length and width of hand image are H and W, respectively. The implementation steps are as follows:

Fig. 1. Geometric structure of hand.

Download Original Figure

Step 1: Extract edge.

Sobel operator is used to extract edge.

Step 2: Remove wrist.

The wrist affects on hand contour extraction, so it needs to be removed. As shown in Fig. 1, middle finger length to palm length ratio is about 3:4, AB:BC is 5:3, and the y coordinate of B is:

$${\text{y}}_{\text{B}}=0.1\text{H+}\frac{5}{8}\times 0.9\text{H}\approx \text{0}\text{.6H}$$

(1)

Step 3: Extract intersection points.

Each contour point is used as the center of an IC. The width occupied by each finger is about W/8~W/10. The selected IC radius should be smaller than the finger width. Therefore, the radius of an IC r < W/10, so we set r = W/13. (x₀, y₀) represents the central of IC, whose intersection points satisfy:

(2)

Where (x, y) represents the coordinates of contour points.

Step 4: Count the number of intersection points.

n denotes the number of intersection points between the IC and contour. Whether the point is retained depends on the value of n.

If n = 1, the point is close to the contour endpoints on the either side of wrist.

If n = 2, the point is at the fingertip or IFV contour.

If n = 3 or 4, the point is at the contour of finger body, because the distance between two adjacent fingers is less than the IC radius.

If n ≥ 5, the hand is abnormally collected, or there are major defects in the segmentation and contour extraction.

The contour points with different values of n are shown in Fig. 2. The contour points with n = 2 are retained.

Fig. 2. The contour points with different intersection point numbers.

Download Original Figure

2.1.2. Inter-finger-valley detection

A contour point is set as P, and the points where IC intersects with hand contour are E and F. If ∠EPF < 135°, keep the contour point P, otherwise, remove it.

IFV and fingertip contours have “U” or “V” shape characteristics. According to the central angle bisector direction of different contours, the fingertip contours and the false IFV contours are excluded.

In Fig. 3(a), Arrows 1, 2 and 3 respectively indicate central angle bisector direction ranges of fingertips, false IFV, and true IFV [51]. The contour is removed when all its points satisfy that the angle between the central angle bisector and the vertical direction is larger than 75°. Finally, the four IFVs with a few interference points as shown in Fig. 3(b) are retained.

Fig. 3. Remove interference contours.

Download Original Figure

After preprocessing, there may be few unsatisfactory points in IFV contours, which result in the fracture of the IFV. In order to connect the broken IFV contour and keep the shape unchanged, a 3 × 3 template is used to perform a morphological close operation on all contour points. At last, keep the four longest contours as the IFV contours.

First find the row of the points at the bottom of IFV contour, then select the middle point of these points as the KP.

The palmprint samples are acquired according to the I-shaped guide assist method. 5 to 7 images of each hand are collected. The database contains 918 samples of 160 hands of 80 people. The ratio of male to female is about 7:3, and the age is between 18 and 60. Most individuals are undergraduates. The collection mobile device is Huawei G750-t01. The collection environments include indoor, outdoor, various illuminations, even no illumination conditions.

Fig. 4 shows the palmprint images taken under different backgrounds. Several KP localization methods are compared. Red dots and blue boxes indicate KPs and ROIs.

Fig. 4. Comparison of localization results in different environments (projection method, tangent method, CHVD, TPDTR, our method).

Download Original Figure

In ideal background (Fig. 4(a)), all methods have good results. In the background with skin-like color (Fig. 4(b)), KP localization in our method is more accurate than others. In complex background (Fig. 4(c)), only our method succeeds to localize KP and ROI.

Our method has a lower error rate. In complex backgrounds, projection method and the tangent method have a weak ability to resist background interference. CHVD is easily affected by texture near IFVs, which results in localization errors. TPDTR method has good stability, but the “U” and “V” shape contours in complex background would lead to detect false IFV contour.

We judge the accuracy of KP localization with correct rate. The lowest points of the IFVs between the index finger and the middle finger as well as the ring finger and the little finger are manually labeled as the standard KPs. The images are resized to 600 × 338. When the distance between the KP and the marked point is within 5 pixels, the localization is judged to be correct. In Table 1, The correct rate of our method is 96.3%, which is the higher than the compared methods.

Equal error rate (EER), d’, and receiver operating characteristic (ROC) are used as palmprint verification performance indicators. The palmprint feature templates are Binary Orientation Co-occurrence Vector [52], which is a good coding scheme [53] and has good verification performance thanks for the fusion at score level [54, 55]. The experimental results are shown in Table 2 and Fig. 5.

Fig. 5. ROC comparison.

Download Original Figure

In summary, our KP localization based on IC has higher recognition accuracy than the compared methods, and stronger ability to resist background interference.