https://doi.org/10.31449/inf.v47i9.4675 Informatica 47 (2023) 63–80 63 An Effective Hyperspectral Palmprint Identification System Based on Deep Learning and Band Selection Approaches Maarouf Korichi 1 , Djamel Samai 1 , Azeddine Benlamoudi 1 , Abdallah Meraoumia 2 , Khaled Bensid 1 1 Fac. des nouvelles technologies de l’information et de la communication. Lab. de Génie électrique (LAGE), Univ Ouargla, 30000 Ouargla, Algeria. 2 Laboratory of Mathematics Informatics and Systems (LAMIS) University of Larbi Tebessi Tebessa, Algeria. E-mail: korichi.maarouf@univ-ouargla.dz, samai.djamel@univ-ouargla.dz, benlamoudi.azeddine@univ-ouargla.dz, ameraoumia@univ-tebessa.dz, bensid.khaled@univ-ouargla.dz Keywords: biometrics, identification, hyperspectral palmprint, band selection, OCF, deep learning Received: February 11, 2023 Over the past two decades, biometric technologies have exploded because anything that identifies a person provides a source of information. The palmprint modality is a biometric characteristic of great interest to researchers, and its traits can be found in various representations, including grayscale, colour, and multi/hyperspectral representations. The most difficult challenge in developing a hyperspectral palmprint- based recognition system is determining how to use all the information available in these spectral bands. In this paper, we propose a hyperspectral palmprint identification system. In the first stage, an Optimal Clustering Framework (OCF) is proposed to extract the most representative bands. Then, two types of feature extraction methods (handcrafted and deep learning approaches) were used to determine the best method to describe palmprint features. After setting the number of selected bands to 4, we performed our set of experiments using the Hong Kong Polytechnic University (Poly U), which consists of 69 spectral bands. The results indicated that the proposed system offers the best performance, which qualifies it to be intended for usage in high-security situations. Povzetek:glavni cilj tega dela je oblikovati biometrični identifikacijski sistem z uporabo hiperspektralnega slikanja odtisa dlani (HSP) in nove sheme izbire pasu. 1 Introduction Due to the progression of information security technology, the number of risks to the recognition system and its resources has increased, necessitating the need to strengthen security measures. The necessity for faster and more reliable user authentication approaches has increased security concerns in these days of rapid improvements in communication, networking, and mobility [1]. Researchers suggest the use of human biometric traits as a solution to guarantee the identity of system clients [2]. Biometrics is a term that refers to technologies that use a person's unique physiological or behavioural features for identification and/or authentication. Fingerprints, palmprints, facial patterns, iris patterns, voice, signatures, and gait are unique biometric attributes, often known as "biometric data". Nowadays, many biometric systems are being designed and used, and the utility of biometric systems is becoming more commonly understood and accepted because of the uniqueness of human biometrics, which is essential in preventing imposter threats. These authentication systems have defeated traditional security measures based on knowledge (such as a password) or possession (such as a key), which can be faked or broken [3], [4]. Palmprint recognition has recently been identified as a novel biometric recognition technology, and it has received great interest. Palmprint recognition is the term for the technology that uses images of the palm skin on a human’s hands to verify that the person is who they say they are [5]. The palmprint is of great interest to researchers because it is the most advanced biometric technique and has been used for over a century. However, this interest is due to its many advantages over other biometric modalities, such as usability, long-term stability, affordability, availability of materials, higher accuracy, and so on [6]. The research in palmprint imaging-based systems can be found in various representations. One can find a multi- spectral (MSP) based palmprint representation that provides additional discriminating information. Because four spectral bands have peaks at various light wavelengths, such as vein networks where the near- infrared band (NIR) penetrates the skin, visualisation of the vein pattern is allowed [7]. Therefore, hyperspectral- based palmprint imaging is a new research area in which researchers in numerous domains have been interested because of its high ability to distinguish between customers (users). In comparison to multi-spectral 64 Informatica 47 (2023) 63–80 M. Korichi et al. palmprint identification, the hyperspectral palmprint sensor collects spectral information varying from 420 nm to 1100 nm from 69 different bands, which could provide more discriminative information. Nevertheless, two major difficulties must be solved before hyperspectral biometrics can be used. Firstly, how many spectral bands are required to distinguish between distinct palms? On the one hand, typically, more feature bands provide more information, implying more accuracy. In contrast, increasing the number of feature bands increases feature extraction and matching costs. Moreover, providing more information may not always improve accuracy due to redundancies between distinct spectra. Secondly, how do we choose representative spectra for a given number of feature bands? In hyperspectral palmprint identification, optimal band selection and band combination are major research topics [8–9]. However, exploiting all the possible information from hyperspectral imaging is a difficult challenge. To do that, two main strategies can be found in the literature to utilise all the possible information. Clustering, band combination, and band selection are the most commonly used approaches in hyperspectral-based palmprint recognition systems. In this paper, to maintain all of the important information about the palmprint and reduce the number of bands as much as possible, a selection of bands should be made first before palmprint identification. Thus, a band selection scheme is developed and adapted to extract the most representative hyperspectral palmprint bands. A multimodal palmprint-based recognition system is proposed, with two feature extraction approaches: handcrafted-based and deep learning-based. After extracting the relevant information from the selected hyperspectral palmprint band images, a Deep Rule-Based classifier (DRB) is deployed to determine whether the person who tries to access the system is genuine or an impostor. To summarise our work, the following contributions can be cited: ▪ An Optimal Clustering Framework (OCF) is proposed for automatically selecting the most discriminative hyperspectral palmprint bands. ▪ Efficient unimodal and multimodal HSP palmprint identification systems have been made based on the chosen bands. ▪ An extensive comparative analysis between the handcrafted features and the deep learning is performed to choose the best techniques that better describe the palmprint features. ▪ The impact of incorporating a deep rule-based classifier (DRB) for classification tasks was investigated. To construct our paper, we followed the following steps: Firstly, in Section 2, we introduce the state of the art of hyperspectral palmprint-based recognition systems. Then, Section 3 discusses the proposed scheme for an HSP palmprint-based identification system. After that, the optimal clustering framework for hyperspectral band selection is outlined in Section 4. Section 5 briefly describes the feature extraction descriptors used (handcrafted-based and deep learning-based). The architecture of the Deep Rule-Based classifier (DRB) is described in Section 6. Section 7 then presents the experimental results on the PolyU hyperspectral palmprint database. Section 8 concludes with conclusions and future work. 2 Hyperspectral palmprint recognition: A literature review Relevant research has been published for the identification of a person’s palmprint traits. So, many approaches for extracting features from palmprint scans have recently attracted a variety of research interests. The biggest challenge for constructing an HSP- or even MSP-based palmprint recognition system is how to exploit all the possible information from those images. Therefore, one of the crucial tasks in the HSP palmprint-based recognition system is the extraction of features. This step is all about getting the most important discriminative information from the region of interest (ROI) that has been identified and making the palmprint of different subjects more separable. In this context, we are going to talk in this section about the most pertinent studies that treat the subject of HSP-based palmprint recognition systems. Although the literature on hyperspectral palmprint identification is relatively limited, some works extract 2D features from each band image and fuse them for recognition. In contrast, clustering and band selection techniques are used in other works to select the most informative bands. Hereafter, we classify the feature extraction approaches into two primary categories: handcrafted-based techniques and deep learning-based techniques. 2.1 Handcrafted-based approaches Handcrafted features-based palmprint recognition depends on texture characteristics, which are low-level features that characterise a specified area of an image and give accurate features for the highest palmprint recognition rate [10]. Shen et al. [11] developed a 3D Gabor wavelet-based approach to concurrently extract information in the spatial and spectral domains. However, a series of three- dimensional Gabor wavelets with different frequencies and orientations were created and convolved with the cube to extract discriminative information. The authors of this study calculated the similarity between two hyperspectral cubes using the humming distance. As an extended version of their work, Shen in [12] and [13] suggested using clustering techniques and 3D Gabor wavelets to reduce the band’s number. To choose the most representative bands, they propose the use of a clustering- based method known as Affinity Propagation (AP) in their work. Another research work is based on using a 2D Gabor filter response for hyperspectral palmprint recognition [14]. The features descriptor is applied to each hyperspectral band An Effective Hyperspectral Palmprint Identification System Based… Informatica 47 (2023) 63–80 65 separately after a dimensionality reduction using (2D) 2 LDA. The similarity measure is performed through the use of SVM and KNN classifiers. Exploiting all the information offered by the hyperspectral images is a difficult challenge. Therefore, Korichi et al. [15] introduced a new feature descriptor called 3D local binary pattern (3DLBP). The proposed features descriptor is an extended version of the well-known descriptor LBP (Local Binary Pattern), and it can be applied to any image type (grayscale, colour, multi-spectral, and hyperspectral images). In their study [16], Guo et al. found that the spectral bands 700 nm and 960 nm provided the strongest discriminatory information. The extracted features are based on using a feature descriptor called (2D) 2 PCA, where the bands are selected according to their recognition accuracy. Also, using Principal Component Analysis (PCA) and the Hidden Markov Model, Meraoumia et al. [17] presented a biometric recognition system based on multispectral and hyperspectral palmprint images. Firstly, the feature vectors are extracted using PCA [18], and after that, the feature vectors are modelled using HMM. Sun [19] and Chlaoua [20] introduced the use of k-mean- based clustering techniques, which are aimed at reducing the number of spectral bands. Their method eliminates low-quality bands by computing image entropy and EER with a 2D Gabor filter response. In addition, the optimal band combination is chosen using a clustering technique that is validated by the band fusion technique after this procedure. Guo et al. [21] use hyperspectral palmprint data to investigate feature band selections. Although there were some promising results in accuracy and anti-spoofing capacity, the feature band selection from many hyperspectral palmprints needed to be faster, making it unsuitable for real-time deployment. 2.2 Deep learning-based approaches Deep learning is regarded as a breakthrough in computer vision and has been effectively applied in different domains. Neural networks and deep learning approaches are the most recent categories of methods. Because of their adaptability and representational capacity, these methods are gaining popularity in various fields, particularly palmprint-based biometric systems [18]. Zhao et al. [22] presented a joint deep convolution feature representation to recognise hyperspectral palmprints. All spectral bands can be used with a CNN stack to create a single convolutional feature. They tested their model's performance on a hyperspectral palmprint dataset with 53 bands. They were able to attain an EER of 0.01 %. The same author, Zhao [23], proposes combining features from different hyperspectral palmprints. This strategy combines global Deep Convolutional Neural Network (DCNN) features, texture LBP features, and direction LDP features. The hyperspectral palmprint images are subjected to the LBP, LDP, and DCNN algorithms to extract their features, resulting in three feature matrices. The 2D PCA is then applied to each feature matrix to remove redundant information and reduce the dimension. An EER of 0.11% was reported for a database including 53 distinct bands. Based on PCANet's deep learning features, Meraoumia et al. [24] propose a palmprint biometric identification system. After extracting the PCANet features, four classifiers (SVM, RBF, RFT, and KNN) are used to differentiate between the distinct palmprint feature vectors. In their research work, two multi-spectral palmprint databases are used. However, a recent research study introduced by Trabelsi and her team [10] aimed to construct a multispectral palmprint identification system. The suggested system relies on PalmNet and Log-Gabor capabilities, utilising feature selection and dimensionality reduction algorithms. The proposed method is evaluated using three different multispectral palmprint databases. Bensid et al. [25] suggested using a deep learning-based feature extraction technique called discrete cosine transform network (DCTNet) to construct a multi-spectral palmprint recognition system. The CASIA and PolyU palmprint databases are the most commonly used benchmarks for evaluating their proposed systems. Hence, after analyzing a variety of research articles that treat the use of deep learning-based approaches, one can find that the majority of those research articles use a database of 4 or 6 bands maximally (multi-spectral databases), where they try to select which band or which combination is the most appropriate to represent a palmprint identification system without the use of a band selection algorithm. On one hand, as we previously mentioned, hyperspectral imaging offers more information that can be useful to differentiate between users. On the other hand, the more the number of bands there are, the more redundant information we have, which influences the system accuracy. For this reason, we suggest using a new bands selection algorithm before constructing a deep learning-based hyperspectral palmprint system. This will help choose the most useful bands and eliminate redundant information. 3 Proposed system design The objective of this study is to develop a hyperspectral palmprint identification system and an Optimal Clustering Framework (OCF) for the selection of hyperspectral bands. To do this, an OCF is first applied to the utilised hyperspectral palmprint database (the Hong Kong University hyperspectral palmprint database), where the objective is to extract the most representative bands. After choosing the bands, the proposed hyperspectral palmprint recognition system is made up of two separate subsystems that interface with each other at the matching score level. As shown in Figure 1, there are four steps in each subsystem: preprocessing, feature extraction, matching, and making a decision. To get into the system's database (enrollment), a person has to give a set of training palmprint modalities. Typically, two approaches for feature vector extraction are used, namely, a handcrafted approach where Local Phase Quantization (LPQ) and Binarized Statistical Image Feature (BSIF) are used. The second approach is based on the use of a deep learning 66 Informatica 47 (2023) 63–80 M. Korichi et al. approach, specifically the use of well-known pre-trained networks like AlexNet, VGG16, and VGG19. For identification, the same feature descriptors are used to extract the feature vectors from the test palmprint. Then, the Deep Rule-Based classifier (DRB) is used to compare the test palmprint templates with the database of reference feature vectors. Finally, for the multimodal system, each subsystem generates its own matching score, which is then combined to obtain a final score (fusion at the score level). This last score is used to determine whether the user should be accepted or rejected. 4 Hyperspectral palmprint band selection From such an implementation viewpoint, a well-chosen band set should maintain just the most informative bands while removing those that offer little discriminating information. Because the first step of our work is selecting the representative hyperspectral bands, we suggest the use of an Optimal Clustering Framework (OCF). In this section, we will briefly present the proposed method for selecting the appropriate hyperspectral palmprint bands. The next three steps in installing the proposed framework are as follows: ▪ Firstly, an optimal clustering framework is developed to find the best HSI clustering structure. However, the hyperspectral palmprint database can be analysed using the proposed OCF, which can determine the optimal solution under an acceptable constraint. Furthermore, because the proposed OCF is a global framework, many types of objective functions can be optimised using the same approach as long as they accord with the framework's specific form. Also, the suggested OCF is a global framework. This implies that the same method can be used to optimise different types of objective functions, as long as they fit the specific form of the framework. ▪ After the clustering structure is achieved, an RCS (ranking on clusters strategy) is proposed as a useful criterion to select the most representative bands. With the addition of an arbitrary ranking algorithm, the RCS may better use the advantages of both clustering and ranking-based techniques. This will produce band subsets with lower correlation and more discriminative information. Some ranking criteria, such as MVPCA (Maximum-Variance Principal Component Analysis), Enhanced Fast Density- Peak-based Clustering (E-FDPC), and Information Entropy (IE), can also be used to finish the RCS task. ▪ Finally, an automated approach for determining the required number of bands is proposed because the goal is to reduce the correlation between bands to determine how much unique information may be generated by a given number of bands. The overall procedure for designing a band selection algorithm is shown in Figure 2. 5 Feature extraction One of the difficult tasks in constructing such a biometric- based identification system is extracting the relevant information that can be used to differentiate between users. Typically, several algorithms are utilised in the feature extraction process; the approach differs based on the type of biometric identification being used. Our study uses two feature extraction approaches: a handcrafted- based and a deep learning-based approach. In this section, we will briefly explain the feature descriptors used. 5.1 Local phase quantization (LPQ) Ojansivu et al. [26] proposed local phase quantization as a texture descriptor. Also, it was difficult to blur and outperform for texture classification compared to other descriptors. For the first time, the LPQ descriptor has been designated for use in the classification of texture blur. LPQ is designed to preserve an image's local invariant Figure 1: Architecture of the proposed hyperspectral palmprint recognition system. An Effective Hyperspectral Palmprint Identification System Based… Informatica 47 (2023) 63–80 67 information against artefacts caused by various types of blurring. 5.2 Binarized statistical image features (BSIF) Kannala et al. [27] developed a texture descriptor called Binarized Statistical Image Features. The BSIF is a binary code string representing a given image's pixels. A pixel's code value is regarded as a local descriptor of the image in its surroundings. BSIF has been developing local image descriptors that efficiently encapsulate texture information and can be used to represent image regions using histograms. The method generates a binary code for each pixel by linearly projecting local image patches onto a subspace whose basis vectors are binarized by thresholding after being learned from natural images using Independent Component Analysis (ICA). The number of basis vectors determines the length of the binary code string. Histograms of pixel binary codes can be used to represent image regions. 5.3 AlexNet AlexNet has eight weighted layers, five convolutional layers, and three fully linked layers. Except for the last layer, which outputs a softmax with a distribution over the 1000 class labels, ReLu activation is performed after each layer. In the first two completely connected layers, dropout is used. Alex Krizhevsky and his team won the 2012 ImageNet Large Scale Visual Recognition Challenge with AlexNet, which used an 8-layer CNN [28]. This network showed for the first time that learning-based features can do better than manually designed ones, shattering the old paradigm in computer vision. 5.4 VGG models VGG (Visual Geometry Group) is a convolutional neural network designed by K. Simonyan and A. Zisserman [29] of the University of Oxford that achieved recognition in 2014 after winning the ILSVRC (ImageNet Large Scale Visual Recognition Challenge). On Imagenet, the model achieved 92.7% accuracy, one of the greatest results ever. It was a step forward from earlier models in that it proposed convolution kernels with lower dimensions (33) in the convolution layers than had been done previously. Using cutting-edge graphics cards, the model was trained over several weeks. VGG16 [29] is a convolutional neural network trained on images from the ImageNet database. The network has an image input size of (224 × 224) and can categorise images into 1000 item categories with 16 layers. It has about 138 million parameters, making evaluation challenging and necessitating a large amount of memory. As a result, the network has learned a variety of rich feature representations for various images. The VGG19 model is a variant of the VGG model with 19 layers (16 convolution layers, 5 MaxPool layers, 1 SoftMax layer, and three fully connected layers). 6 Matching and fusion Once each image's distinctive and relevant characteristics have been determined, the images are divided into numerous groups, and a decision is made to realise whether the user is a client or an impostor. In our system, the Deep Rule Based classifier (DRB) is devoted to performing the matching task. Figure 2: Diagram showing the required steps to conduct the OCF-based band selection process. Figure 2: The DRB classifier's general architecture 68 Informatica 47 (2023) 63–80 M. Korichi et al. 6.1 Deep Rule-Based classifier The Deep Rule-Based classifier (DRB) [30] is a Semi- Supervised Multi-Layer Neuro-fuzzy Modeling (SSMNM) technique based on image prototypes that produce visible and human-comprehensible fuzzy rules (IF... THEN. Once new classes are added to the system, an incremental learning algorithm can be utilised in real-time applications to monitor new concepts, deal with uncertainty, self-evolve, and adapt their network and meta-parameters. The DRB's layers are as follows: 1) the scaling layer, 2) the normalisation layer, 3) the feature descriptor layer, 4) the FRB layer, and 5) the decision- maker layer. A. Layer of Scaling (Optional): The Scaling Layer is in charge of shrinking the original image to reduce computational complexity, improve generalisation, etc. All images were resized to (128 x 128) as in the study, according to the requirements of the feature descriptor layer. B. Normalization Layer (A standard pre-processing step): The Normalisation Layer is in charge of translating image pixel values into the feature descriptor's specified range. This study's pixels are normalised between 0 and 1 because of the feature descriptor layer requirements. C. The Feature Descriptor Layer: The Feature Descriptor Layer extracts the global feature vectors from the images. The retrieved features are applied to the classification model's training and validation. D. Base layer using Fuzzy-Rule (Model engine): The DRB classifier's basic engine is the Fuzzy-Rule base (FRB) layer, composed of a massively parallel ensemble of type 0-order fuzzy rules. This layer is responsible for producing a highly accurate classifier while simultaneously providing interpretability and transparent models for human comprehension, in contrast to ordinary deep learning techniques, known as black boxes, since they do not offer insights into the network's structure. E. The Decision-Maker Layer: This layer is responsible for making the final decision by labelling the validation images depending on the degree of prototype similarity determined by the FRB layer. The essential layers of the DRB classifier architecture are shown in Figure 3 [30]. 6.1.1 The DRB classifier algorithm First, we describe the important notations utilised in the DRB classifier algorithm in Table 1. Table 1: DRB classifier algorithm key notation description Notations Descriptions C The dataset's classes number D The feature vector's dimensionality K The observed training images number/instance of the current time I A single input image X The I corresponding feature vector NC The c th class identified prototypes number 𝝁 𝒄 The c th class training images feature vector’s global average. Ic,k The c th class k th training image 𝝌 𝒄 ,𝒌 The I c,k corresponding feature vector Pc,i The c th class i th prototype pc,i The average of training images feature vectors associated with P c,i Sc,i The training images number associated with P c,i rc,i The data cloud influences area radius associated with P c,i 𝝀 𝒄 The confidence score is given by the local decision-maker of the c th fuzzy rule. The four main phases of the DRB classification algorithm are as follows [30]: 1. Step 01: Pre-processing block- Most of the time, the DRB classifier uses simple and common pre-processing techniques, such as image normalisation, rotation, scaling, and image segmentation. 2. Step 02: Feature extraction layer- Various types of feature descriptors can be used on this layer. 3. Step 03: Massive parallel FRB layer- This step consists of four parts, which are as follows: initialization, preparation, system updating, and fuzzy rule creation, as stated below: Massive parallel FRB layer stages Stage 1: Initialization The identical DRB algorithm parameters are initialized by: 𝑘 ← 1; 𝜇 𝑐 ← 𝜒 ̅ 𝑐 1 ; Ν 𝑐 ← 1; Ρ 𝑐 ,Ν 𝑐 ← Ι 𝑐 ,1 ; 𝑝 𝑐 ,Ν 𝑐 ← 𝜒 ̅ 𝑐 1 ; 𝑆 𝑐 ,Ν 𝑐 ← 1; 𝑟 𝑐 ,Ν 𝑐 ← 𝑟 0 Stage 2: Preparation The 𝜒 ̅ 𝑐𝑘 and 𝐷 (Ρ 𝑐 ,𝑖 ) for each image Ι 𝑐 ,𝑘 are calculated by: 𝜒 ̅ 𝑐𝑘 = 𝜒 𝑐𝑘 ‖𝜒 𝑐𝑘 ‖ ; 𝜇 𝑐 = 𝑘 −1 𝑘 𝜇 𝑐 + 𝜒 ̅ 𝑐𝑘 𝐷 (Ρ 𝑐 ,𝑖 ) = 1 1+‖𝐶 −𝜇 𝑐 ‖ 2 /𝜎 2 Stage 3: Updating system An Effective Hyperspectral Palmprint Identification System Based… Informatica 47 (2023) 63–80 69 The DRB algorithm verifies two conditions (Conditions 1 and 2) before updating the parameters of all images in stage 1. If condition 1 is true then add a new data cloud else find the nearest prototype 𝑃 𝑐 ,𝑛 corresponding to Ι 𝑐 ,𝑘 and go to the condition 2 If condition 2 is true, Update 𝑃 𝑐 ,𝑛 , 𝑆 𝑐 ,𝑛 and 𝑟 𝑐 ,𝑛 2 else add a new data cloud Stage 4: Fuzzy rules generation-Generate rule type Rulec 𝒊𝒇 (𝑰 ~𝑃 𝑐 ,1 ) 𝑂𝑅 ⋯ 𝑂𝑅 (𝑰 ~𝑃 𝑐 ,𝑁 𝑐 ) 𝑻𝒉𝒆𝒏 (𝑐𝑙𝑎𝑠𝑠𝑒 𝑐 ) Condition 1 𝒊𝒇 (𝐷 (Ι 𝑐 ,𝑘 ) > 𝑖 =1,2,3,⋯,𝑁 𝑐 𝑚𝑎𝑥 (𝐷 (Ρ 𝑐 ,𝑖 ))) 𝑂𝑅 (𝐷 (Ι 𝑐 ,𝑘 ) < 𝑖 =1,2,3,⋯,𝑁 𝑐 𝑚𝑖𝑛 ( 𝐷 (Ρ 𝑐 ,𝑖 ))) 𝑻𝒉𝒆𝒏 Ι 𝑐 ,𝑘 is new prototype 𝑁 𝑐 ← 𝑁 𝑐 + 1 Ρ 𝑐 ,Ν 𝑐 ← Ι 𝑐 ,𝑘 ; 𝑝 𝑐 ,Ν 𝑐 ← 𝜒 ̅ 𝑐 ,𝑘 ; 𝑆 𝑐 ,Ν 𝑐 ← 1; 𝑟 𝑐 ,Ν 𝑐 ← 𝑟 0 Else Find 𝑃 𝑐 ,𝑛 by 𝑃 𝑐 ,𝑛 = 𝑎𝑟𝑔𝑚𝑖𝑛 (‖𝜒 ̅ 𝑐 ,𝑘 − 𝑃 𝑐 ,𝑗 ‖ where 𝑗 = 1,2, ⋯ , Ν 𝑐 Condition 2 𝑖𝑓 (‖𝜒 ̅ 𝑐 ,𝑘 − 𝑃 𝑐 ,𝑛 ‖ < 𝑟 𝑐 ,Ν 𝑐 ) 𝑻𝒉𝒆𝒏 Ι 𝑐 ,𝑘 is assigned to 𝑃 𝑐 ,𝑛 Else 𝑁 𝑐 ← 𝑁 𝑐 + 1; Ρ 𝑐 ,Ν 𝑐 ← Ι 𝑐 ,𝑘 ; 𝑝 𝑐 ,Ν 𝑐 ← 𝜒 ̅ 𝑐 ,𝑘 ; 𝑆𝑐 , Ν 𝑐 ← 1; 𝑟 𝑐 ,Ν 𝑐 ← 𝑟 0 Endif Endif 4. Step 4. Decision maker layer: For each image 𝐼 in the test data, find the confidence score 𝜆 𝑐 (𝐼 ) based on the image's feature vector by using the following formula: 𝜆 𝑐 (𝐼 ) = 𝑎𝑟𝑔𝑚𝑎𝑥 (𝑒𝑥𝑝 (−‖𝜒 − 𝑃 𝑐 ,𝑗 ‖ 2 ), 𝑗 = 1,2, ⋯ , Ν 𝑐 . - the labels are identified by: 𝑙𝑎𝑏𝑒𝑙 (𝐼 ) = 𝑎𝑟𝑔𝑚𝑎𝑥 (𝜆 𝑐 (𝐼 ) , 𝑐 = 1,2, ⋯ , 𝑐 . 6.2 Fusion process It has been demonstrated in several studies that combining the results from different unimodal biometric identification systems generally leads to better system efficiency. Fusion at the matching-score level involves integrating the scores generated by each biometric system that describe the similarity between the biometrics acquired and their models [31]. We experimented with four different schemes during the system's design: ▪ SUM-score (SUM): 𝑆 = ∑ 𝑆 𝑖 𝑁 𝑖 =1 (1) ▪ SUM-Weighting-Score (WSum): 𝑆 = ∑ w i S i N i=1 (2) ▪ Production-score (Prod): 𝑆 = ∏ 𝑆 𝑖 𝑖 =1,2,...,𝑁 (3) ▪ Production-Weighting- score (WProd): 𝑺 = ∏ 𝒘 𝒊 𝑺 𝒊 𝒊 =𝟏 ,𝟐 ,...,𝑵 (4) Where 𝑤 𝑖 denotes the weight of the 𝑖 𝑡 ℎ biometric attribute of the 𝑘 𝑡 ℎ user's matching score, which is determined as follows: 𝑤 𝑖 = 1 𝐸𝐸𝑅 𝑖 ∑ 1 𝐸𝐸𝑅 𝑖 𝑁 𝑖 =1 (5) 7 Experiment results and discussion As mentioned previously, this work's main objective is to design a biometric identification system using hyperspectral palmprint (HSP) imaging. A band selection scheme is first implemented before constructing our system and reducing the spectral band's number. When the representative bands have been established, two feature extraction methods are used to perform the feature extraction task: handcrafted and deep learning-based approaches. To test and evaluate the proposed system, experiments were conducted using the hyperspectral palmprint database collected by the HK PolyU Institute. 7.1 Hyperspectral palmprint database description In the experiment test phase, the publicly available HK- PolyU Hyperspectral palmprint database is used [32]. The Hong Kong Polytechnic University's Biometric Research Centre (UGC/CRC) has designed a hyperspectral palmprint capture device to collect images from 420 nm to 1100 nm. The Hong Kong PolyU hyperspectral palmprint database is a large database comprised of 190 volunteer images. However, the volunteers in this database range from 20 to 60 years old. The samples were collected in two separate sessions. The patient was instructed to create seven palmprint images, with a size of 128 x 128 pixels, for each wavelength throughout each session. A set of 69 spectral bands with a step length of 10 nm might be used to image a palmprint over a range of 420 nm–1100 nm. With the first and second sessions separated by a month, there is typically a month between them. The database contains 5,240 images from 380 different palms at one wavelength. 7.2 OCF based hyperspectral palmprint bands selection The problem of huge processing complexity occurred with HSI's extensive dimensional data, and it remained a 70 Informatica 47 (2023) 63–80 M. Korichi et al. significant challenge because of the strong correlations and dependencies among said data. The first step in designing our proposed hyperspectral palmprint system is to reduce the number of spectral bands and only use the most representative ones. This section investigates the implementation of the OCF framework to select HIS's most representative bands. However, three RCS were tested and evaluated to achieve the optimum results, including MVPCA, E. FDPC, and IE. Table 2 shows the results that came from using these methods. It should be noted that the PolyU hyperspectral palmprint database comprises two samples taken from the right and left hands. To accomplish the band selection process, the RCS procedures are applied independently to the two samples. While implementing the OCF framework, we set the number of the output selected bands to 4. It is possible to justify this by comparing it to the MSP palmprint database, which has four different modalities (Red, Green, Blue, and NIR). As previously stated, one of our objectives is to select the most representative HSI spectral bands. As a result, various experiments were carried out and evaluated using three RCS methods. When looking at and analysing Table 1, If the left-hand spectral bands are employed, the MVPCA RCS techniques generate bands of wavelength (rank) as follows: 610 nm (20), 710 nm (30), 810 nm (40), and 960 nm (55). When the right-hand spectral bands are employed, the selectable bands are 610 nm (20), 710 nm (30), 870 nm (46) and 940 nm (940 nm) (53). However, by employing the left-hand spectral bands and the E FDPC techniques as an RCS methodology, the resulting OCF- based systems are 470 nm (06), 690 nm (28), 960 nm (55), and 990 nm (58). The system-selected bands for the right- hand spectral bands are 480 nm (07), 590 nm (18), 850 nm (44) and 990 nm (58). Finally, if the IE-based RCS technique is employed in conjunction with the left-hand spectral bands, the selected bands are 610 nm (20), 690 nm (28), 850 nm (44) and 940 nm (53). The IE selectable bands for the right-hand spectral bands are 610 nm (20), 760 nm (35), 850 nm (44) and 940 nm (53). We noticed through manual visual analysis that the first two bands chosen by E_FDPC-based RCS techniques (the 06th spectral band (470 nm) for the left hand and the 07th (480 nm) with the 18th (590 nm) spectral bands for the right hand) carry little to no information, making them useless for our proposed system. As a result, we concluded that it would produce undesirable effects on the proposed system's performance. This leaves us with the two remaining techniques that produce relatively similar results, compelling us to experiment with the selected bands. In conclusion, the entropy bands produced a more desired effect than the MVPCA bands. An example of the HSI-selected bands using the IE-based RCS techniques is shown in Figure 4. Hence, the selected bands for both the right and left hand are then used to construct a unimodal/multimodal biometric identification system where two feature extraction approaches are used Table 2: Hyperspectral palmprint band selection using OCF framework Modality RCS method 1 st band 2 nd 3 rd band 4 th band LEFT Hand MVPCA 610 nm (20) 710 nm (30) 810 nm (40) 960 nm (55) E_FDPC 470 mm (06) 690 nm (28) 960 nm (55) 990 nm (58) IE 610 nm (20) 690 nm (28) 850 nm (44) 940 nm (53) RIGHT Hand MVPCA 610 nm (20) 710 nm (30) 870 nm (46) 940 nm (53) E_FDPC 480 nm (07) 590 nm (18) 850 nm (44) 990 nm (58) IE 610 nm (20) 760 nm (35) 850 nm (44) 940 nm (53) Figure 4: Information Entropy (IE) based RCS techniques most representative An Effective Hyperspectral Palmprint Identification System Based… Informatica 47 (2023) 63–80 71 along with the DRB classifier. The next section will describe and assess the proposed identification system's performance. 7.3 Unimodal systems test results After extracting the most representative HSI palmprint bands based on the OCF framework, our experiment's next step is to deploy them to design a reliable biometric identification system. It was necessary to conduct three separate experiments in order to evaluate the performance of our proposed hyperspectral palmprint identification system. In the first experiment, we used only one spectral band. In contrast, we will integrate multiple spectral bands in the second and third experiments to construct a multimodal biometric system. Note that in all experiments, two approaches for feature extraction are used: handcrafted approaches (LPQ and BSIF) and deep learning techniques with three pre-trained networks named VGG16, VGG19, and the AlexNet network. As we reported earlier, the palmprint image database contains 190 persons. Each person has 12 images of the left hand and 12 of the right hand, which were taken in 4 bands. We chose six images from 12 for training [𝟏 𝟑 𝟓 𝟕 𝟗 𝟏𝟏 ] and the other six [𝟐 𝟒 𝟔 𝟖 𝟏𝟎 𝟏𝟐 ] for the test (this applies to both the left-hand and right-hand palmprint images). Table 3 shows the performance of the proposed unimodal system by using palmprint images captured at the selected band wavelength. For the best biometric identification system performance, which means minimising the Error Equal Rate (EER) for the open-set identification mode and maximising the Recognition of One Rate (ROR) for the closed-set identification mode, our experiments aim to find the best selected spectral bands and the best feature descriptor (handcrafted or deep learning-based) to represent the palmprint features. According to the results shown in Table 3 and Figures 5, 6, 7, and 8, one can observe that: The first remark is that the band 610 nm-based system performance achieves the lowest performance for both identification modes and feature extraction approaches. Also, one can observe that the performance of the deep learning feature-based feature extraction system Figure 5: Proposed open-set identification system based on Handcrafted feature extraction approach. Figure 6: Proposed closed-set identification system based on Handcrafted feature extraction approach. 72 Informatica 47 (2023) 63–80 M. Korichi et al. outperforms the system performance when using handcrafted feature ex traction techniques. Hereafter, we will discuss and analyse the proposed unimodal system's performance. In the case of the open-set identification mode, handcrafted feature descriptors are used. Our results show that the system performs best when utilising the left-hand-based spectral bands, except when the 850 nm right-hand-based spectral band is employed, where the system achieves its optimum performance, which results in an EER of 0.0318% at the threshold T0 = 0.8173 when the LPQ is used as a feature descriptor. Furthermore, if the LPQ is deployed, the EER ranges between [0.3508%–5.2631%] for the left-hand spectral bands and [0.3911%–6.4035% for the right-hand spectral bands. Whenever the proposed system uses the BSIF as a feature descriptor, the results are noticeably better than when the LPQ features are used. The EER of left-hand selectable bands based on BSIF features ranged from [0.1754%–4.2982%], whereas it ranged from [0.2154%– 5,4385%] for the right-hand selected bands. The second part of this section's experiments will evaluate the proposed system's performance when some well- known deep pre-trained networks are used to extract deep features from the selected bands. Once the AlexNet model is used as a feature descriptor, it is possible to see that the proposed system achieves its best performance by providing the lowest value of EER (0.0129%) at a threshold value of T0 = 0.8701. This is the case when using the 850 nm right-hand spectral bands. However, through this series of experiments, we are convinced that the right-hand band of the 850 nm spectrum contains the most discriminative information. Now, turning our attention to the system's performance when deep learning feature descriptors were applied, what stands out in Table 3 is that the proposed deep feature descriptors gave perfect results in certain bands for both the left and right hand. As can be seen, when the VGG16 is used with the right-hand- Figure 7: Proposed open-set identification system based on deep learning deep features approach Figure 8: Proposed closed-set identification system based on deep learning deep features approach. An Effective Hyperspectral Palmprint Identification System Based… Informatica 47 (2023) 63–80 73 based bands, the system produces an EER between [0.3508%-5.5263%], whereas when the left-hand-based bands are utilized, the system produces an EER between [0.0877%-4.1228%]. For the VGG19-based proposed system performance, one can notice that the system achieves approximately similar results to those obtained when using VGG16. Finally, Figures 5 and 7 are plotted to show how the open-set unimodal system performs when using the two feature approaches. In the case of closed-set identification, we notice from Table 3 that the previous observations made in the open- set mode still apply to the closed-set mode. The proposed system achieves its best performance when the 850 nm spectrum right-hand band is deployed along with AlexNet-based deep learning techniques, where the system gives a ROR of 99.9123% with an RPR equal to 4. In addition, if the system works under a handcrafted- based features descriptor, the best performance is obtained using the same spectrum and an LPQ descriptor (ROR = 99.5614%, RPR = 9). However, it can be shown that both hands' two wavelengths, 850 nm and 940 nm, performed similarly. Compared to the other networks, the system's efficiency is clearly superior when the AlexNet network is utilized. To give a concise summary of the performance of the proposed closed-set identification mode for both feature extraction approaches, Figures 6 and 8 have been plotted. 7.4 Multimodal systems test results This stage of experiments aims to improve the results given by the unimodal biometric identification system by using information from various bands and instances (left and right palmprints). Two fusion scenarios were used to accomplish this. The first is based on the fusion of four bands from the same instance (left or right hand). The second fusion scheme employs all bands from the two instances. The fusion process is carried out at the score level in our study work. When using matching score level fusion, the two sub-systems' matching scores are blended to produce a single score. The ultimate decision is then based on the outcome. To accomplish this, we experiment with rules-based fusion, which employs four distinct rules (described in Section 6.2). Therefore, several tests were carried out to choose the best fusion scenario and the fusion rule. The results are displayed in Tables 4 and 5. Once again, the EER is used to test the performance of the multimodal system in open- set identification mode. From Table 4, where the first fusion scenario is applied, one can see that the basis of fusion for both feature extraction approaches enhance the efficiency of the unimodal system. This can be justified by the results obtained in Table 4 compared to those in Table 3. As an example, one can observe that the EER is reduced to 0.000% at a threshold of 0.8440 when the right-hand spectral bands are fused using the PROD fusion rule and the AlexNet as a feature descriptor. This table also makes it clear how much the performance of the handcrafted feature-based approach has improved. For instance, in the right-hand fused bands, the maximum value of EER when utilising the LPQ descriptor was 6.4035%, but after the fusion, it was only 0.5263%. This observation was also confirmed when the pre-trained deep learning networks were used, which served to justify the usefulness of incorporating the fusion process. By fusing all selected bands from the left and right hand (second fusion scenario), the system achieves its optimum performances under both feature approaches, where the EER is reduced to 0.000% for all feature descriptors (see Table 5) under the use of SUM and WSUM-based fusion rules (in the case of AlexNet-based system performances, the system achieves the same performances for all fusion rules). To validate our concepts, we have to run other experiments for closed-set identification. For the first scenario, we can also see from Table 4 that the AlexNet- based system performs well where the system achieves a ROR of 100% with an RPR equal to one under the fuse of right-hand bands with the PROD fusion rules. Contrary to the second fusion scenario, the system, like the open-set experiment tests, achieves optimal performance under all feature descriptors when using SUM and WSUM fusion rules. The system produces a ROR of 100% and an RPR of 1. 7.5 Comparison with state of the art To show the proposed protocol's efficiency, a comparison study with the most relevant and related research works is performed. It is necessary to mention that the comparison is performed to the research works where the protocols are evaluated using the PolyU hyperspectral palmprint database. Although, as mentioned in Section 2, the literature on hyperspectral palmprint identification is very limited, some works concentrate on extracting 2D information from each band image and fusing them for recognition. In contrast, other works use clustering and band selection approaches to select the most informative bands. However, the main objective in designing such a hyperspectral palmprint system is to obtain the optimum system performance using minimum features. For example,[21] achieves an EER of 0.0780% by increasing the number of clusters to 4. Also, [19] research is based on the clustering technique (3 clusters), where the obtained EER was 0.17325%. In addition, [12] designed a hyperspectral palmprint system and a clustering technique (10 clusters), and he obtained an EER=3.26%. Other works also extract information from the hyperspectral cube. For example, [11] obtained an EER= 4.00% using a hyperspectral palmprint of 52 bands, whereas [23] and [22] used a cube of 53 bands and obtained (EER=0.11%, ROR=99.76%) and (EER=0.015%, ROR=99.625%) respectively. By manually evaluating all 69 hyperspectral bands, [14] made a simple hyperspectral palmprint system with a ROR of 99.92%. This method is not preferred because it takes more processing time. The results in Table 6 clearly show that our proposed hyperspectral palmprint system exceed most of the cited relevant work in term of EER and ROR by achieving a 74 Informatica 47 (2023) 63–80 M. Korichi et al. (EER = 0.0123%, ROR=99.9123%) in the case of unimodal system conception and a (EER=0.000%, ROR=100%) in the case of multimodal system. 8 Conclusion Through technological improvements, we are becoming highly dependent on data, which is increasing in size and becoming vital and influential in our lives. Recently, palmprint-based biometric recognition systems have emerged as a promising field with a high ability to distinguish between people. Among the palmprint recognition systems, one can find systems that use hyperspectral palmprint images as a biometric modality. Due to the larger system database size and longer processing times, the most challenging part of designing such a hyperspectral palmprint system is utilizing all the information these images may offer. As a solution, we proposed optimal framework clustering (OCF) that automatically extracts the most representative hyperspectral bands to reduce the system database size and eliminate redundant information. After that, a biometric identification system based on the selected bands is constructed, and two feature descriptors are applied. The handcrafted features are extracted using LPQ and BSIF descriptors, and the AlexNet, VGG16, and VGG19 pre-trained networks are used to extract the deep- learning features. The DRB classifier is devoted to performing the classification tasks. The proposed system is evaluated and tested using the PolyU hyperspectral palmprint database. The obtained unimodal experimental results showed that the bands of 850 nm and 940 nm can be considered the most representative’ bands according to the proposed protocol. Also, the experiments showed that the deep learning-based approach outperformed the handcrafted methods. However, the multimodal biometric system was introduced in order to overcome the limitations of the unimodal systems. As a result, according to the proposed fusion scenario, one can observe that the EER is reduced to 0.000% in the open-set identification mode and obtains a ROR equal to 100% in the closed-set mode. For future work directions, we will extend our proposed framework for band selection to other hyperspectral traits, such as face or iris hyperspectral images. We will also look into combining deep learning or handcrafted features from different bands to make a better biometric system. 9 Acknowledgment The authors gratefully acknowledge the Directorate General for Scientific Research and Technological Development (DGRSDT) of Algeria for the financial support for this work. The authors also acknowledge resources and support from the electrical engineering laboratory (laboratoire de génie électrique (LAGE)) of KASDI Merbah University, Ouargla, Algeria. References [1] Altarawneh, Mokhled Sueliman. Cancelable fingerprint features using chaff points encapsulation. Informatica, 2018, vol. 42, no 3.. https://doi.org/10.31449/inf.v42i3.1855 [2] Sharif, M., Raza, M., Shah, J.H., Yasmin, M., Fernandes, S.L. (2019). An Overview of Biometrics Methods. In: Singh, A., Mohan, A. (eds) Handbook of Multimedia Information Security: Techniques and Applications. Springer, Cham. https://doi.org/10.1007/978-3-030-15887- 3_2 [3] A. Benlamoudi, S. E. Bekhouche, M. Korichi, K. Bensid, and A. H. and A. T.-A. Abdeldjalil Ouahabi, “Face Presentation Attack Detection Using Deep Background Subtraction,” sensors, vol. 22, no. 10, pp. 1–17, 2022. https://doi.org/10.3390/s22103760 [4] S. A. Abdulrahman and B. Alhayani, “A comprehensive survey on the biometric systems based on physiological and behavioural characteristics,” Materials Today: Proceedings, 2021. https://doi.org/10.1016/j.matpr.2021.07.005 [5] Jia, W., Xia, W., Zhao, Y. et al. 2D and 3D Palmprint and Palm Vein Recognition Based on Neural Architecture Search. Int. J. Autom. Comput. 18, 377–409 (2021). https://doi.org/10.1007/s11633-021-1292-1 [6] Raouia Mokni, Hassen Drira, Monji Kherallah. Deep-Analysis of Palmprint Representation based on Correlation Concept for Human Biometrics identification. International Journal of Digital Crime and Forensics, vol. 12, no. 2, pp. 40–58, 2020. https://doi.org/10.4018/ijdcf.2020040103 [7] Y. Aberni, L. Boubchir and B. Daachi, "Multispectral palmprint recognition: A state-of- the-art review," 2017 40th International Conference on Telecommunications and Signal Processing (TSP), 2017, pp. 793-797, https://doi.org/10.1109/tsp.2017.8076097 . [8] THAMRI, Essia, ALOUI, Kamel, et NACEUR, Mohamed Saber. Selection of hyperspectral bands by adopting a dimension reduction strategy for recognition of multispectral palmprint. In : 2017 International Conference on Advanced Systems and Electric Technologies (IC_ASET). IEEE, 2017. p. 91-96. https://doi.org/10.1109/aset.2017.7983672 [9] Q. Wang, F. Zhang and X. Li, "Optimal Clustering Framework for Hyperspectral Band Selection," in IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 10, pp. 5910-5922, Oct. 2018, https://doi.org/10.1109/tgrs.2018.2828161 . [10] Trabelsi, S., Samai, D., Dornaika, F. et al. Efficient palmprint biometric identification systems using deep learning and feature selection methods. Neural Comput & Applic 34, 12119– An Effective Hyperspectral Palmprint Identification System Based… Informatica 47 (2023) 63–80 75 12141,(2022). https://doi.org/10.1007/s00521-022-07098-4 [11] L. Shen, W. Wu, S. Jia and Z. Guo, "Coding 3D Gabor Features for Hyperspectral Palmprint Recognition," 2014 International Conference on Medical Biometrics, 2014, pp. 169-173, https://doi.org/10.1109/icmb.2014.36 . [12] L. Shen, Z. Dai, S. Jia, M. Yang, Z. Lai and S. Yu, "Band selection for Gabor feature based hyperspectral palmprint recognition," 2015 International Conference on Biometrics (ICB), 2015, pp. 416-421, https://doi.org/10.1109/icb.2015.7139104 [13] Jie Zhou· Yunhong Wang· Zhenan Sun · Yong Xu Linlin Shen · Jianjiang Feng Shiguang Shan · Yu Qiao Zhenhua Guo · Shiqi Yu, Biometric recognition, Proceedings:12th Chinese Conference, CCBR 2017, Shenzhen, China, October 28-29, vol. 449, no. 7158,2017. https://doi.org/10.1007/978-3-319-69923-3 [14] Khandizod, A.G., Deshmukh, R.R. (2019). Optimal Band Selection for Improvement of Hyperspectral Palmprint Recognition System by Using SVM and KNN Classifier. In: Santosh, K., Hegadi, R. (eds) Recent Trends in Image Processing and Pattern Recognition. RTIP2R 2018. Communications in Computer and Information Science, vol 1036. Springer, Singapore. https://doi.org/10.1007/978-981-13- 9184-2_38 [15] M. Korichi and A. Meraoumia, “Improved biometric identification system using a new scheme of 3D local binary pattern,” International Journal of Information and Communication Technology, Vol. 14, No. 4, p. 439-455, 2019. https://doi.org/10.1504/ijict.2019.101863 [16] Z. Guo, L. Zhang and D. Zhang, "Feature Band Selection for Multispectral Palmprint Recognition," 2010 20th International Conference on Pattern Recognition, 2010, pp. 1136-1139, https://doi.org/10.1109/icpr.2010.284 . [17] A. Meraoumia, S. Chitroub, and A. Bouridane, “An efficient palmprint identification system using multispectral and hyperspectral imaging,” In: Modeling Approaches and Algorithms for Advanced Computer Applications. Studies in Computational Intelligence, vol 488. Springer, Cham. https://doi.org/10.1007/978-3-319-00560-7_20 [18] V. Roşca and A. Ignat, "Quality of Pre-trained Deep-Learning Models for Palmprint Recognition," 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), 2020, pp. 202-209, https://doi.org/10.1109/synasc51798.2020.00041 [19] Junwen Sun, Waleed Abdulla, Weiming Wang, Qiong Wang, and Hai Zhang, "Band Selection for Palmprint Recognition," Journal of Advances in Information Technology Vol. 7, No. 4, pp. 287- 290, November, 2016. https://doi.org/10.12720/jait.7.4.287-290 [20] R. Chlaoua, A. Meraoumia, M. Korichi and K. Aiadi, "Visible spectrum bands of palmprint image for a robust biometric identification system," 2016 International Conference on Information Technology for Organizations Development (IT4OD), 2016, pp. 1-4, https://doi.org/10.1109/it4od.2016.7479292 [21] Z. Guo, D. Zhang, L. Zhang and W. Liu, "Feature Band Selection for Online Multispectral Palmprint Recognition," in IEEE Transactions on Information Forensics and Security, vol. 7, no. 3, pp. 1094-1099, June 2012, https://doi.org/10.1109/tifs.2012.2189206 [22] S. Zhao, B. Zhang, and C. L. Philip Chen, “Joint deep convolutional feature representation for hyperspectral palmprint recognition,” Inf. Sci. (Ny)., vol. 489, pp. 167–181, 2019. https://doi.org/10.1016/j.ins.2019.03.027. [23] S. Zhao, W. Nie and B. Zhang, "Multi-Feature Fusion Using Collaborative Residual for Hyperspectral Palmprint Recognition," 2018 IEEE 4th International Conference on Computer and Communications (ICCC), 2018, pp. 1402- 1406, https://doi.org/10.1109/compcomm.2018.878074 8 [24] Meraoumia, A., Kadri, F., Bendjenna, H., Chitroub, S., Bouridane, A. (2017). Improving Biometric Identification Performance Using PCANet Deep Learning and Multispectral Palmprint. In: Biometric Security and Privacy. Signal Processing for Security Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-47301-7_3 [25] BENSID, Khaled, SAMAI, Djamel, LAALLAM, Fatima Zohra, et al. Deep learning feature extraction for multispectral palmprint identification. Journal of Electronic Imaging, 2018, vol. 27, no 3, p. 033018. https://doi.org/10.1117/1.JEI.27.3.033018 [26] J. Heikkila and V. Ojansivu, "Methods for local phase quantization in blur-insensitive image analysis," 2009 International Workshop on Local and Non-Local Approximation in Image Processing, 2009, pp. 104-111, https://doi.org/10.1109/lnla.2009.5278397 [27] ADJIMI, Ahlem, HACINE-GHARBI, Abdenour, RAVIER, Philippe, et al. Extraction and selection of binarised statistical image features for fingerprint recognition. International Journal of Biometrics , vol. 9, no 1, p. 67-80, 2017. https://doi.org/10.1504/ijbm.2017.10005054 [28] Alex Krizhevsky, “ImageNet Classification with Deep Convolutional Neural Networks Alex,” Handb. Approx. Algorithms Metaheuristics, pp. 1–9, 2017. https://doi.org/10.1145/3065386 [29] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 3rd Int. Conf. Learn. Represent. 76 Informatica 47 (2023) 63–80 M. Korichi et al. ICLR 2015 - Conf. Track Proc., pp. 1–14, 2015. https://doi.org/10.48550/arXiv.1409.1556 [30] X. Gu, P. P. Angelov, C. Zhang and P. M. Atkinson, "A Massively Parallel Deep Rule- Based Ensemble Classifier for Remote Sensing Scenes," in IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 3, pp. 345-349, March 2018, https://doi.org/10.1109/lgrs.2017.2787421 . [31] K. Nandakumar, Yi Chen, A. K. Jain and S. C. Dass, "Quality-based Score Level Fusion in Multibiometric Systems," 18th International Conference on Pattern Recognition (ICPR'06), 2006, pp. 473-476, https://doi.org/10.1109/icpr.2006.951 [32] Department of Computing, the Hong Kong Polytechnic University (PolyU), Hyperspectral Palmprint database, PolyU, available at: http://www4.comp.polyu.edu.hk/~biometrics /Hyperspectral Palmprrint /HSP .htm. An Effective Hyperspectral Palmprint Identification System Based… Informatica 47 (2023) 63–80 77 Table 3: proposed unimodal system performance (handcrafted & deep learning approach) Method Modality Band EER (%) T0 ROR (%) RPR LPQ 27 Left 610 (20) 5.2631 0.9683 89.4737 188 690 (28) 0 . 8 1 7 0 0 . 7 8 1 3 96.2281 171 850 (44) 0.0877 0.7698 99.5614 17 940 (53) 0.3508 0.7210 99.1228 92 Right 610 (20) 6.4035 0.7194 88.0702 188 760 (35) 2.2807 0.7539 94.7368 188 850 (44) 0.0318 0.8173 99.5614 9 940 (53) 0.3911 0.7444 99.2105 88 BSIF Left 610 (20) 4.2982 0.6489 90.5263 190 690 (28) 0.7894 0.7895 96.4912 143 850 (44) 0.1619 0.7158 99.6491 19 940 (53) 0.1754 0.7381 99.2982 71 Right 610 (20) 5.4385 0.6744 89.2982 188 760 (35) 1.2595 0.7753 95.8772 187 850 (44) 0.0877 0.7515 99.5614 6 940 (53) 0.2154 0.7124 99.5614 91 CNN- VGG16 Left 610 (20) 4.1228 0.7972 88.1579 180 690 (28) 1.0337 0.8197 96.1404 98 850 (44) 0.0877 0.8553 99.3860 147 940 (53) 0.0877 0.8465 99.4737 108 Right 610 (20) 5.5263 0.8395 86.2281 190 760 (35) 0.4385 0.8436 97.2807 67 850 (44) 0.1896 0.8371 98.9474 12 940 (53) 0.3508 0.8175 98.8596 32 CNN- VGG19 Left 610 (20) 3. 8 1 7 9 0. 8 2 6 5 87.6316 175 690 (28) 1 . 3 1 5 7 0 . 8 0 5 3 96.0526 123 850 (44) 0.1754 0.8317 99.6491 93 940 (53) 0.0877 0.8742 99.3860 69 Right 610 (20) 5.4321 0.8389 86.0526 189 760 (35) 0.5047 0.8267 97.3684 29 850 (44) 0 . 1 7 5 4 0 . 8 4 2 1 99.1228 21 940 (53) 0 . 9 4 30 0.8670 99.9123 3 CNN- AlexNet Left 610 (20) 1.9298 0.7842 93.5088 113 690 (28) 0.7017 0.7851 97.8070 165 850 (44) 0.0877 0.7642 99.9123 55 940 (53) 0.0877 0.7687 99.7368 31 Right 610 (20) 3.0701 0.7839 90.2632 145 760 (35) 0.2631 0.8091 99.2982 107 850 (44) 0.0129 0.8701 99.9123 4 940 (53) 0.0349 0.8513 99.8246 5 78 Informatica 47 (2023) 63–80 M. Korichi et al. Table 4: Multimodal open/closed-set test results (first scenario) Method Modality Band EER (%) T0 ROR (%) RPR LPQ 27 Left Bands Sum 0.0877 0.7535 99.6491 26 WSum 0.0877 0.7819 99.6491 15 Prod 0.7260 0.2372 97.8070 135 WProd 0.0877 0.7449 99.6491 15 Right Bands Sum 0.0352 0.8184 99.6491 10 WSum 0.2190 0.7505 99.0351 83 Prod 0.5358 0.3085 98.3333 171 WProd 8.2667.10 -3 0.8820 99.5614 7 BSIF Left Bands Sum 0.0647 0.7137 99.8246 17 WSum 0.0536 0.7446 99.7368 11 Prod 0.5263 0.2070 98.5088 187 WProd 0.2631 0.6821 99.6491 187 Right Bands Sum 0.0129 0.8351 99.8246 13 WSum 0.1754 0.7570 99.5614 75 Prod 0.2227 0.3485 99.0351 88 WProd 0.0445 0.7715 99.8246 3 CNN- VGG16 Left Bands Sum 0.0877 0.7973 99.8246 80 WSum 0.0877 0.8177 99.7368 36 Prod 0.0877 0.5151 99.7368 63 WProd 0.0877 0.8168 99.8246 117 Right Bands Sum 0.0459 0.8435 99.2982 5 WSum 0.0877 0.8288 99.2982 16 Prod 0.0877 0.5232 99.2982 188 WProd 0.1754 0.7980 99.0351 188 CNN- VGG19 Left Bands Sum 0.0877 0.7935 99.9123 41 WSum 0.0536 0.8296 99.7368 15 Prod 0.0877 0.4917 99.7368 29 WProd 0.0877 0.8242 99.9123 69 Right Bands Sum 0 . 0 1 2 0 0 . 9 1 3 1 99.4737 2 WSum 0 . 0 2 9 7 0 . 8 8 8 3 99.1228 6 Prod 0 . 0 8 7 7 0 . 5 0 0 5 99.6491 188 WProd 0.1754 0.8036 99.2982 188 CNN- AlexNet Left Bands Sum 7.4259.10 -3 0.8030 99.9123 13 WSum 5.5111.10 -3 0.8930 99.7368 5 Prod 6.4977.10 -3 0.6360 99.8246 6 WProd 0.0877 0.7267 99.9123 30 Right Bands Sum 1.8564.10 -3 0.9760 99.9123 2 WSum 0.0102 0.9431 99.7368 9 Prod 0 0.8440 100 1 WProd 3.6741.10 -3 0.9110 99.8246 3 An Effective Hyperspectral Palmprint Identification System Based… Informatica 47 (2023) 63–80 79 Table 5: Multimodal open/closed-set test results (second scenario) Method Band EER (%) T0 ROR (%) RPR LPQ 27 Sum 0 0.9130 100 1 WSum 0 0.9850 100 1 Prod 0.0891 0.2130 99.3860 62 WProd 9.2824x10-4 0.9740 99.9123 2 BSIF Sum 0 0.9600 100 1 WSum 0 0.9590 100 1 Prod 0.1754 0.0725 99.7368 184 WProd 0.1754 0.4195 99.8246 184 CNN- VGG16 Sum 0 0.8210 100 1 WSum 0 0.8770 100 1 Prod 0.0877 0.1327 99.9123 186 WProd 0.0877 0.5821 99.9123 186 CNN- VGG19 Sum 0 0.8050 100 1 WSum 0 0.8950 100 1 Prod 0.0877 0.1221 99.9123 184 WProd 0.0877 0.5761 99.9123 184 CNN- AlexNet Sum 0 0.7930 100 1 WSum 0 0.8260 100 1 Prod 0 0.1510 100 1 WProd 0 0.7450 100 1 Table 6: proposed system performance comparison with the state-of-the-art Method Band selection evaluation way Number of selected bands Feature extraction approach EER ROR [21] Clustering (4 clusters) 4 Handcrafted 0.0780% / [11] / 52 Handcrafted 4.00% / [12] Clustering (10 cluster) / Handcrafted 3.26% / [19] Clustering (3 clusters) 3 Handcrafted 0.17325% / [13] / All Handcrafted / 99.43% [23] / 53 Deep learning 0.11% 99.76% [22] / 53 Deep learning 0.015% 99.62% [14] Manual 2 Handcrafted / 99.92% Proposed approach Automatically 4 Handcrafted/deep learning 0.000% 100% 80 Informatica 47 (2023) 63–80 M. Korichi et al.