Not able to detect all faces, what logic could have gone wrong?

I am trying to build a face recognition model, which I have trained using the Xception model in keras. I am getting the accuracy as 97% and val_acc as 88.6%. Now when I am using the below provided code, it is only predicting one person, for others it giving output as “Not matching”. But when I change if(pred[0][1]>0.5) for each person it is able to predict them but when I write elif and try to predict all three together it ends up predicting one person only. I** have set a threshold of 0.5 for each class as I am using softmax as activation function. How can I rectify the code.

Here is the code:

from PIL import Image
from keras.applications import preprocess_input
import base64
from io import BytesIO
import json
import random
import cv2
from keras.models import load_model
import numpy as np


from keras.preprocessing import image

model = load_model(‘facefeatures_Xcep_model.h5’)#saved model is loaded here

#loading frontface cascade
face_cascade = cv2.CascadeClassifier(r’C:\Users\Hitesh Pant\AppData\Local\Programs\Python\Python37\Lib\site-packages\cv2\data\haarcascade_frontalface_default.xml’)


def face_extractor(img):

faces = face_cascade.detectMultiScale(img, 1.7, 5)
if faces is ():
    return None

for (x,y,w,h) in faces:
    cv2.rectangle(img, (x,y), (x+w, y+h), (255,0,0), 2)
    cropped_face = img[y:y+h, x:x+w]

return cropped_face


video_capture = cv2.VideoCapture(0)

while True:
_, frame =

face = face_extractor(frame)
if type(face) is np.ndarray: #checking for the image value encoded in numpmy array
    face = cv2.resize(face, (224,224))
    im = Image.fromarray(face, 'RGB')
    img_array = np.array(im)
    img_array = np.expand_dims(img_array, axis = 0)
    pred = model.predict(img_array)
    name = "No Match Found"
    #defining a threshold for each class/person for prediction
    if (pred[0][0]>0.5):
        name = "Sam"
    if (pred[0][1]>0.5):
        name = "xyz"
    if (pred[0][2]>0.5):
    name= 'zzz'
    cv2.putText(frame, name, (50,50), cv2.FONT_HERSHEY_COMPLEX, 1, (0,255,0), 2)
    cv2.putText(frame, "No Face Found", (50,50), cv2.FONT_HERSHEY_COMPLEX, 1, (0,255,0), 2)
cv2.imshow('Video', frame)

if cv2.waitKey(1) & 0xFF == ord('q'):


Maybe I’m missing something, but it looks to me like face_extractor only returns one face. I.e. cropped_face is not defined as an array, so only a single value is ever returned (the last face found in faces).

1 Like