mirror of
https://github.com/serengil/deepface.git
synced 2025-06-07 12:05:22 +00:00
retinaface added
This commit is contained in:
Sefik Ilkin Serengil
parent
577408ad69
commit
974f8bf27b
54
README.md
54
README.md
@ -17,7 +17,7 @@ The easiest way to install deepface is to download it from [`PyPI`](https://pypi
|
||||
pip install deepface
|
||||
```
|
||||
|
||||
## Face Recognition
|
||||
## Face Recognition
|
||||
|
||||
A modern [**face recognition pipeline**](https://sefiks.com/2020/05/01/a-gentle-introduction-to-face-recognition-in-deep-learning/) consists of 4 common stages: [detect](https://sefiks.com/2020/08/25/deep-face-detection-with-opencv-in-python/), [align](https://sefiks.com/2020/02/23/face-alignment-for-face-recognition-in-python-within-opencv/), [represent](https://sefiks.com/2018/08/06/deep-face-recognition-with-keras/) and [verify](https://sefiks.com/2020/05/22/fine-tuning-the-threshold-in-face-recognition/). Deepface handles all these common stages in the background. You can just call its verification, find or analysis function in its interface with a single line of code.
|
||||
|
||||
@ -30,7 +30,6 @@ Verification function under the deepface interface offers to verify face pairs a
|
||||
```python
|
||||
from deepface import DeepFace
|
||||
result = DeepFace.verify("img1.jpg", "img2.jpg")
|
||||
#results = DeepFace.verify([['img1.jpg', 'img2.jpg'], ['img1.jpg', 'img3.jpg']])
|
||||
print("Is verified: ", result["verified"])
|
||||
```
|
||||
|
||||
@ -38,7 +37,7 @@ print("Is verified: ", result["verified"])
|
||||
|
||||
Herein, face pairs could be exact image paths, numpy array or base64 encoded images.
|
||||
|
||||
**Face recognition** - [`Demo`](https://youtu.be/Hrjp-EStM_s)
|
||||
**Face recognition** - [`Demo`](https://youtu.be/Hrjp-EStM_s)
|
||||
|
||||
Face recognition requires to apply face verification several times. Herein, deepface offers an out-of-the-box find function to handle this action. It stores the representations of your facial database and you don't have to find it again and again. In this way, you can apply [face recognition](https://sefiks.com/2020/05/25/large-scale-face-recognition-for-deep-learning/) data set as well. The find function returns pandas data frame if a single image path is passed, and it returns list of pandas data frames if list of image paths are passed.
|
||||
|
||||
@ -46,12 +45,11 @@ Face recognition requires to apply face verification several times. Herein, deep
|
||||
from deepface import DeepFace
|
||||
import pandas as pd
|
||||
df = DeepFace.find(img_path = "img1.jpg", db_path = "C:/workspace/my_db")
|
||||
#dfs = DeepFace.find(img_path = ["img1.jpg", "img2.jpg"], db_path = "C:/workspace/my_db")
|
||||
```
|
||||
|
||||
<p align="center"><img src="https://raw.githubusercontent.com/serengil/deepface/master/icon/stock-6-v2.jpg" width="95%" height="95%"></p>
|
||||
|
||||
Herein, image path argument could be exact image path, numpy array or base64 encoded image. Also, you are expected to store your facial image data base in the folder that you passed to the db_path argument with .jpg or .png extension.
|
||||
Herein, image path argument could be exact image path, numpy array or base64 encoded image. Also, you are expected to store your facial image data base in the folder that you passed to the db_path argument with .jpg or .png extension.
|
||||
|
||||
**Face recognition models** - [`Demo`](https://youtu.be/i_MOwvhbLdI)
|
||||
|
||||
@ -68,7 +66,7 @@ FaceNet, VGG-Face, ArcFace and Dlib [overperforms](https://youtu.be/i_MOwvhbLdI)
|
||||
|
||||
**Similarity**
|
||||
|
||||
Face recognition models are regular [convolutional neural networks](https://sefiks.com/2018/03/23/convolutional-autoencoder-clustering-images-with-neural-networks/) and they are responsible to represent faces as vectors. Decision of verification is based on the distance between vectors. We can classify pairs if its distance is less than a [threshold](https://sefiks.com/2020/05/22/fine-tuning-the-threshold-in-face-recognition/).
|
||||
Face recognition models are regular [convolutional neural networks](https://sefiks.com/2018/03/23/convolutional-autoencoder-clustering-images-with-neural-networks/) and they are responsible to represent faces as vectors. Decision of verification is based on the distance between vectors. We can classify pairs if its distance is less than a [threshold](https://sefiks.com/2020/05/22/fine-tuning-the-threshold-in-face-recognition/).
|
||||
|
||||
Distance could be found by different metrics such as [Cosine Similarity](https://sefiks.com/2018/08/13/cosine-similarity-in-machine-learning/), Euclidean Distance and L2 form. The default configuration finds the cosine similarity. You can alternatively set the similarity metric while verification as demostratred below.
|
||||
|
||||
@ -105,12 +103,11 @@ Here, you can find some implementation demos of deepface with a-nn libraries: [`
|
||||
|
||||
**Facial Attribute Analysis** - [`Demo`](https://youtu.be/GT2UeN85BdA)
|
||||
|
||||
Deepface also offers facial attribute analysis including [`age`](https://sefiks.com/2019/02/13/apparent-age-and-gender-prediction-in-keras/), [`gender`](https://sefiks.com/2019/02/13/apparent-age-and-gender-prediction-in-keras/), [`facial expression`](https://sefiks.com/2018/01/01/facial-expression-recognition-with-keras/) (including angry, fear, neutral, sad, disgust, happy and surprise) and [`race`](https://sefiks.com/2019/11/11/race-and-ethnicity-prediction-in-keras/) (including asian, white, middle eastern, indian, latino and black) predictions. Analysis function under the DeepFace interface is used to find demography of a face.
|
||||
Deepface also offers facial attribute analysis including [`age`](https://sefiks.com/2019/02/13/apparent-age-and-gender-prediction-in-keras/), [`gender`](https://sefiks.com/2019/02/13/apparent-age-and-gender-prediction-in-keras/), [`facial expression`](https://sefiks.com/2018/01/01/facial-expression-recognition-with-keras/) (including angry, fear, neutral, sad, disgust, happy and surprise) and [`race`](https://sefiks.com/2019/11/11/race-and-ethnicity-prediction-in-keras/) (including asian, white, middle eastern, indian, latino and black) predictions. Analysis function under the DeepFace interface is used to find demography of a face.
|
||||
|
||||
```python
|
||||
from deepface import DeepFace
|
||||
obj = DeepFace.analyze(img_path = "img4.jpg", actions = ['age', 'gender', 'race', 'emotion'])
|
||||
#objs = DeepFace.analyze(["img1.jpg", "img2.jpg", "img3.jpg"]) #analyzing multiple faces same time
|
||||
print(obj["age"]," years old ",obj["dominant_race"]," ",obj["dominant_emotion"]," ", obj["gender"])
|
||||
```
|
||||
|
||||
@ -122,7 +119,7 @@ Herein, image path argument could be exact image path, numpy array or base64 enc
|
||||
|
||||
**Streaming and Real Time Analysis** - [`Demo`](https://youtu.be/-c9sSJcx6wI)
|
||||
|
||||
You can run deepface for real time videos as well.
|
||||
You can run deepface for real time videos as well.
|
||||
|
||||
Calling stream function under the DeepFace interface will access your webcam and apply both face recognition and facial attribute analysis. Stream function expects a database folder including face images. VGG-Face is the default face recognition model and cosine similarity is the default distance metric similar to verify function. The function starts to analyze if it can focus a face sequantially 5 frames. Then, it shows results 5 seconds.
|
||||
|
||||
@ -170,45 +167,28 @@ The both face recognition and facial attribute analysis are covered in the API.
|
||||
|
||||
**Face Detectors** - [`Demo`](https://youtu.be/GZ2p2hj2H5k)
|
||||
|
||||
Face detection and alignment are early stages of a modern face recognition pipeline. [`OpenCV`](https://sefiks.com/2020/02/23/face-alignment-for-face-recognition-in-python-within-opencv/), [`SSD`](https://sefiks.com/2020/08/25/deep-face-detection-with-opencv-in-python/), [`Dlib`](https://sefiks.com/2020/07/11/face-recognition-with-dlib-in-python/) and [`MTCNN`](https://sefiks.com/2020/09/09/deep-face-detection-with-mtcnn-in-python/) methods are wrapped in deepface as a detector. You can optionally pass a custom detector to functions in deepface interface. MTCNN is the default detector if you won't pass any detector.
|
||||
Face detection and alignment are early stages of a modern face recognition pipeline. [`OpenCV`](https://sefiks.com/2020/02/23/face-alignment-for-face-recognition-in-python-within-opencv/), [`SSD`](https://sefiks.com/2020/08/25/deep-face-detection-with-opencv-in-python/), [`Dlib`](https://sefiks.com/2020/07/11/face-recognition-with-dlib-in-python/), [`MTCNN`](https://sefiks.com/2020/09/09/deep-face-detection-with-mtcnn-in-python/) and [`RetinaFace`](https://github.com/serengil/retinaface) methods are wrapped in deepface as a facial detector. You can optionally pass a custom detector to functions in deepface interface. MTCNN is the default detector if you won't pass any detector.
|
||||
|
||||
```python
|
||||
backends = ['opencv', 'ssd', 'dlib', 'mtcnn']
|
||||
backends = ['opencv', 'ssd', 'dlib', 'mtcnn', 'retinaface']
|
||||
for backend in backends:
|
||||
#face detection and alignment
|
||||
detected_face = DeepFace.detectFace("img.jpg", detector_backend = backend)
|
||||
|
||||
|
||||
#face verification
|
||||
obj = DeepFace.verify("img1.jpg", "img2.jpg", detector_backend = backend)
|
||||
|
||||
|
||||
#face recognition
|
||||
df = DeepFace.find(img_path = "img.jpg", db_path = "my_db", detector_backend = backend)
|
||||
|
||||
|
||||
#facial analysis
|
||||
demography = DeepFace.analyze("img4.jpg", detector_backend = backend)
|
||||
```
|
||||
|
||||
<p align="center"><img src="https://raw.githubusercontent.com/serengil/deepface/master/icon/deepface-detectors.png" width="90%" height="90%"></p>
|
||||
|
||||
[MTCNN](https://sefiks.com/2020/09/09/deep-face-detection-with-mtcnn-in-python/) seems to overperform in detection and alignment stages but it is slower than [SSD](https://sefiks.com/2020/08/25/deep-face-detection-with-opencv-in-python/).
|
||||
|
||||
**Passing pre-built face recognition models**
|
||||
|
||||
You can build models once and pass to deepface functions as well. This speeds you up if you are going to call deepface several times. Consider this approach if you are going to call deepface functions in a for loop.
|
||||
|
||||
```python
|
||||
#face recognition
|
||||
models = ['VGG-Face', 'Facenet', 'OpenFace', 'DeepFace', 'DeepID', 'Dlib']
|
||||
for model_name in models:
|
||||
model = DeepFace.build_model(model_name)
|
||||
DeepFace.verify("img1.jpg", "img2.jpg", model_name = model_name, model = model)
|
||||
|
||||
#facial analysis
|
||||
models = {}
|
||||
actions = ['Age', 'Gender', 'Emotion', 'Race']
|
||||
for action in actions:
|
||||
models[action.lower()] = DeepFace.build_model(action)
|
||||
DeepFace.analyze("img1.jpg", models=models)
|
||||
```
|
||||
|
||||
## FAQ and Troubleshooting
|
||||
|
||||
Pre-trained weights of custom models will be downloaded from Google Drive source to your environment once. Download limit of my Google Drive account might be exceeded sometimes. In this case, you might have an exception like "Too many users have viewed or downloaded this file recently. Please try accessing the file again later". You can still download the pre-trained weights from Google Drive manually. You should then download the pre-trained weights to {HOME_FOLDER}/.deepface/weights folder. It won't try to download the weight file if it exists in the weights folder. You can find out your HOME_FOLDER as shown below.
|
||||
@ -219,6 +199,14 @@ home = str(Path.home())
|
||||
print("HOME_FOLDER is ",home)
|
||||
```
|
||||
|
||||
If you are going to call deepface functions in a for loop, then you should build the model once and pass to the functions. This avoids to cause a memory problem and also it will speed you up.
|
||||
|
||||
```python
|
||||
model_name = "Facenet"
|
||||
model = DeepFace.build_model(model_name)
|
||||
DeepFace.verify("img1.jpg", "img2.jpg", model_name = model_name, model = model)
|
||||
```
|
||||
|
||||
## Contribution
|
||||
|
||||
Pull requests are welcome. You should run the unit tests locally by running [`test/unit_tests.py`](https://github.com/serengil/deepface/blob/master/tests/unit_tests.py). Please share the unit test result logs in the PR. Deepface is currently compatible with TF 1 and 2 versions. Change requests should satisfy those requirements both.
|
||||
|
||||
@ -7,18 +7,24 @@ def findCosineDistance(source_representation, test_representation):
|
||||
return 1 - (a / (np.sqrt(b) * np.sqrt(c)))
|
||||
|
||||
def findEuclideanDistance(source_representation, test_representation):
|
||||
if type(source_representation) == list:
|
||||
source_representation = np.array(source_representation)
|
||||
|
||||
if type(test_representation) == list:
|
||||
test_representation = np.array(test_representation)
|
||||
|
||||
euclidean_distance = source_representation - test_representation
|
||||
euclidean_distance = np.sum(np.multiply(euclidean_distance, euclidean_distance))
|
||||
euclidean_distance = np.sqrt(euclidean_distance)
|
||||
return euclidean_distance
|
||||
|
||||
def l2_normalize(x):
|
||||
return x / np.sqrt(np.sum(np.multiply(x, x)))
|
||||
return x / np.sqrt(np.sum(np.multiply(x, x)))
|
||||
|
||||
def findThreshold(model_name, distance_metric):
|
||||
|
||||
|
||||
base_threshold = {'cosine': 0.40, 'euclidean': 0.55, 'euclidean_l2': 0.75}
|
||||
|
||||
|
||||
thresholds = {
|
||||
'VGG-Face': {'cosine': 0.40, 'euclidean': 0.55, 'euclidean_l2': 0.75},
|
||||
'OpenFace': {'cosine': 0.10, 'euclidean': 0.55, 'euclidean_l2': 0.55},
|
||||
@ -30,5 +36,5 @@ def findThreshold(model_name, distance_metric):
|
||||
}
|
||||
|
||||
threshold = thresholds.get(model_name, base_threshold).get(distance_metric, 0.4)
|
||||
|
||||
return threshold
|
||||
|
||||
return threshold
|
||||
|
||||
@ -32,13 +32,13 @@ elif tf_version == 2:
|
||||
#--------------------------------------------------
|
||||
|
||||
def initialize_input(img1_path, img2_path = None):
|
||||
|
||||
|
||||
if type(img1_path) == list:
|
||||
bulkProcess = True
|
||||
img_list = img1_path.copy()
|
||||
else:
|
||||
bulkProcess = False
|
||||
|
||||
|
||||
if (
|
||||
(type(img2_path) == str and img2_path != None) #exact image path, base64 image
|
||||
or (isinstance(img2_path, np.ndarray) and img2_path.any()) #numpy array
|
||||
@ -46,103 +46,107 @@ def initialize_input(img1_path, img2_path = None):
|
||||
img_list = [[img1_path, img2_path]]
|
||||
else: #analyze function passes just img1_path
|
||||
img_list = [img1_path]
|
||||
|
||||
|
||||
return img_list, bulkProcess
|
||||
|
||||
def initialize_detector(detector_backend):
|
||||
|
||||
|
||||
global face_detector
|
||||
|
||||
|
||||
home = str(Path.home())
|
||||
|
||||
|
||||
#eye detector is common for opencv and ssd
|
||||
if detector_backend == 'opencv' or detector_backend == 'ssd':
|
||||
opencv_path = get_opencv_path()
|
||||
eye_detector_path = opencv_path+"haarcascade_eye.xml"
|
||||
|
||||
|
||||
if os.path.isfile(eye_detector_path) != True:
|
||||
raise ValueError("Confirm that opencv is installed on your environment! Expected path ",eye_detector_path," violated.")
|
||||
|
||||
|
||||
global eye_detector
|
||||
eye_detector = cv2.CascadeClassifier(eye_detector_path)
|
||||
|
||||
|
||||
#------------------------------
|
||||
#face detectors
|
||||
if detector_backend == 'opencv':
|
||||
opencv_path = get_opencv_path()
|
||||
face_detector_path = opencv_path+"haarcascade_frontalface_default.xml"
|
||||
|
||||
|
||||
if os.path.isfile(face_detector_path) != True:
|
||||
raise ValueError("Confirm that opencv is installed on your environment! Expected path ",face_detector_path," violated.")
|
||||
|
||||
|
||||
face_detector = cv2.CascadeClassifier(face_detector_path)
|
||||
|
||||
|
||||
elif detector_backend == 'ssd':
|
||||
|
||||
|
||||
#check required ssd model exists in the home/.deepface/weights folder
|
||||
|
||||
|
||||
#model structure
|
||||
if os.path.isfile(home+'/.deepface/weights/deploy.prototxt') != True:
|
||||
|
||||
|
||||
print("deploy.prototxt will be downloaded...")
|
||||
|
||||
|
||||
url = "https://github.com/opencv/opencv/raw/3.4.0/samples/dnn/face_detector/deploy.prototxt"
|
||||
|
||||
|
||||
output = home+'/.deepface/weights/deploy.prototxt'
|
||||
|
||||
|
||||
gdown.download(url, output, quiet=False)
|
||||
|
||||
|
||||
#pre-trained weights
|
||||
if os.path.isfile(home+'/.deepface/weights/res10_300x300_ssd_iter_140000.caffemodel') != True:
|
||||
|
||||
|
||||
print("res10_300x300_ssd_iter_140000.caffemodel will be downloaded...")
|
||||
|
||||
|
||||
url = "https://github.com/opencv/opencv_3rdparty/raw/dnn_samples_face_detector_20170830/res10_300x300_ssd_iter_140000.caffemodel"
|
||||
|
||||
|
||||
output = home+'/.deepface/weights/res10_300x300_ssd_iter_140000.caffemodel'
|
||||
|
||||
|
||||
gdown.download(url, output, quiet=False)
|
||||
|
||||
|
||||
face_detector = cv2.dnn.readNetFromCaffe(
|
||||
home+"/.deepface/weights/deploy.prototxt",
|
||||
home+"/.deepface/weights/deploy.prototxt",
|
||||
home+"/.deepface/weights/res10_300x300_ssd_iter_140000.caffemodel"
|
||||
)
|
||||
|
||||
|
||||
elif detector_backend == 'dlib':
|
||||
import dlib #this is not a must library within deepface. that's why, I didn't put this import to a global level. version: 19.20.0
|
||||
|
||||
|
||||
global sp
|
||||
|
||||
|
||||
face_detector = dlib.get_frontal_face_detector()
|
||||
|
||||
|
||||
#check required file exists in the home/.deepface/weights folder
|
||||
if os.path.isfile(home+'/.deepface/weights/shape_predictor_5_face_landmarks.dat') != True:
|
||||
|
||||
print("shape_predictor_5_face_landmarks.dat.bz2 is going to be downloaded")
|
||||
|
||||
|
||||
print("shape_predictor_5_face_landmarks.dat.bz2 is going to be downloaded")
|
||||
|
||||
url = "http://dlib.net/files/shape_predictor_5_face_landmarks.dat.bz2"
|
||||
output = home+'/.deepface/weights/'+url.split("/")[-1]
|
||||
|
||||
|
||||
gdown.download(url, output, quiet=False)
|
||||
|
||||
|
||||
zipfile = bz2.BZ2File(output)
|
||||
data = zipfile.read()
|
||||
newfilepath = output[:-4] #discard .bz2 extension
|
||||
open(newfilepath, 'wb').write(data)
|
||||
|
||||
|
||||
sp = dlib.shape_predictor(home+"/.deepface/weights/shape_predictor_5_face_landmarks.dat")
|
||||
|
||||
|
||||
elif detector_backend == 'mtcnn':
|
||||
face_detector = MTCNN()
|
||||
|
||||
elif detector_backend == 'retinaface':
|
||||
from retinaface import RetinaFace
|
||||
face_detector = RetinaFace.build_model()
|
||||
|
||||
def initializeFolder():
|
||||
|
||||
|
||||
home = str(Path.home())
|
||||
|
||||
|
||||
if not os.path.exists(home+"/.deepface"):
|
||||
os.mkdir(home+"/.deepface")
|
||||
print("Directory ",home,"/.deepface created")
|
||||
|
||||
|
||||
if not os.path.exists(home+"/.deepface/weights"):
|
||||
os.mkdir(home+"/.deepface/weights")
|
||||
print("Directory ",home,"/.deepface/weights created")
|
||||
@ -156,339 +160,358 @@ def loadBase64Img(uri):
|
||||
def get_opencv_path():
|
||||
opencv_home = cv2.__file__
|
||||
folders = opencv_home.split(os.path.sep)[0:-1]
|
||||
|
||||
|
||||
path = folders[0]
|
||||
for folder in folders[1:]:
|
||||
path = path + "/" + folder
|
||||
|
||||
|
||||
return path+"/data/"
|
||||
|
||||
def load_image(img):
|
||||
|
||||
|
||||
exact_image = False
|
||||
if type(img).__module__ == np.__name__:
|
||||
exact_image = True
|
||||
|
||||
|
||||
base64_img = False
|
||||
if len(img) > 11 and img[0:11] == "data:image/":
|
||||
base64_img = True
|
||||
|
||||
|
||||
#---------------------------
|
||||
|
||||
|
||||
if base64_img == True:
|
||||
img = loadBase64Img(img)
|
||||
|
||||
|
||||
elif exact_image != True: #image path passed as input
|
||||
if os.path.isfile(img) != True:
|
||||
raise ValueError("Confirm that ",img," exists")
|
||||
|
||||
|
||||
img = cv2.imread(img)
|
||||
|
||||
|
||||
return img
|
||||
|
||||
|
||||
def detect_face(img, detector_backend = 'opencv', grayscale = False, enforce_detection = True):
|
||||
|
||||
|
||||
home = str(Path.home())
|
||||
|
||||
|
||||
img_region = [0, 0, img.shape[0], img.shape[1]]
|
||||
|
||||
|
||||
#if functions.preproces_face is called directly, then face_detector global variable might not been initialized.
|
||||
if not "face_detector" in globals():
|
||||
initialize_detector(detector_backend = detector_backend)
|
||||
|
||||
|
||||
if detector_backend == 'opencv':
|
||||
|
||||
|
||||
faces = []
|
||||
|
||||
try:
|
||||
|
||||
try:
|
||||
faces = face_detector.detectMultiScale(img, 1.3, 5)
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
if len(faces) > 0:
|
||||
x,y,w,h = faces[0] #focus on the 1st face found in the image
|
||||
detected_face = img[int(y):int(y+h), int(x):int(x+w)]
|
||||
return detected_face, [x, y, w, h]
|
||||
|
||||
|
||||
else: #if no face detected
|
||||
|
||||
if enforce_detection != True:
|
||||
|
||||
if enforce_detection != True:
|
||||
return img, img_region
|
||||
|
||||
|
||||
else:
|
||||
raise ValueError("Face could not be detected. Please confirm that the picture is a face photo or consider to set enforce_detection param to False.")
|
||||
|
||||
elif detector_backend == 'ssd':
|
||||
|
||||
|
||||
ssd_labels = ["img_id", "is_face", "confidence", "left", "top", "right", "bottom"]
|
||||
|
||||
|
||||
target_size = (300, 300)
|
||||
|
||||
|
||||
base_img = img.copy() #we will restore base_img to img later
|
||||
|
||||
|
||||
original_size = img.shape
|
||||
|
||||
|
||||
img = cv2.resize(img, target_size)
|
||||
|
||||
|
||||
aspect_ratio_x = (original_size[1] / target_size[1])
|
||||
aspect_ratio_y = (original_size[0] / target_size[0])
|
||||
|
||||
|
||||
imageBlob = cv2.dnn.blobFromImage(image = img)
|
||||
|
||||
|
||||
face_detector.setInput(imageBlob)
|
||||
detections = face_detector.forward()
|
||||
|
||||
|
||||
detections_df = pd.DataFrame(detections[0][0], columns = ssd_labels)
|
||||
|
||||
|
||||
detections_df = detections_df[detections_df['is_face'] == 1] #0: background, 1: face
|
||||
detections_df = detections_df[detections_df['confidence'] >= 0.90]
|
||||
|
||||
|
||||
detections_df['left'] = (detections_df['left'] * 300).astype(int)
|
||||
detections_df['bottom'] = (detections_df['bottom'] * 300).astype(int)
|
||||
detections_df['right'] = (detections_df['right'] * 300).astype(int)
|
||||
detections_df['top'] = (detections_df['top'] * 300).astype(int)
|
||||
|
||||
|
||||
if detections_df.shape[0] > 0:
|
||||
|
||||
|
||||
#TODO: sort detections_df
|
||||
|
||||
#get the first face in the image
|
||||
|
||||
#get the first face in the image
|
||||
instance = detections_df.iloc[0]
|
||||
|
||||
|
||||
left = instance["left"]
|
||||
right = instance["right"]
|
||||
bottom = instance["bottom"]
|
||||
top = instance["top"]
|
||||
|
||||
|
||||
detected_face = base_img[int(top*aspect_ratio_y):int(bottom*aspect_ratio_y), int(left*aspect_ratio_x):int(right*aspect_ratio_x)]
|
||||
|
||||
|
||||
return detected_face, [int(left*aspect_ratio_x), int(top*aspect_ratio_y), int(right*aspect_ratio_x) - int(left*aspect_ratio_x), int(bottom*aspect_ratio_y) - int(top*aspect_ratio_y)]
|
||||
|
||||
|
||||
else: #if no face detected
|
||||
|
||||
|
||||
if enforce_detection != True:
|
||||
img = base_img.copy()
|
||||
return img, img_region
|
||||
|
||||
|
||||
else:
|
||||
raise ValueError("Face could not be detected. Please confirm that the picture is a face photo or consider to set enforce_detection param to False.")
|
||||
|
||||
|
||||
elif detector_backend == 'dlib':
|
||||
|
||||
detections = face_detector(img, 1)
|
||||
|
||||
|
||||
if len(detections) > 0:
|
||||
|
||||
|
||||
for idx, d in enumerate(detections):
|
||||
left = d.left(); right = d.right()
|
||||
top = d.top(); bottom = d.bottom()
|
||||
|
||||
|
||||
detected_face = img[top:bottom, left:right]
|
||||
|
||||
|
||||
return detected_face, [left, top, right - left, bottom - top]
|
||||
|
||||
|
||||
else: #if no face detected
|
||||
|
||||
if enforce_detection != True:
|
||||
|
||||
if enforce_detection != True:
|
||||
return img, img_region
|
||||
|
||||
|
||||
else:
|
||||
raise ValueError("Face could not be detected. Please confirm that the picture is a face photo or consider to set enforce_detection param to False.")
|
||||
|
||||
raise ValueError("Face could not be detected. Please confirm that the picture is a face photo or consider to set enforce_detection param to False.")
|
||||
|
||||
elif detector_backend == 'mtcnn':
|
||||
|
||||
|
||||
img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) #mtcnn expects RGB but OpenCV read BGR
|
||||
detections = face_detector.detect_faces(img_rgb)
|
||||
|
||||
|
||||
if len(detections) > 0:
|
||||
detection = detections[0]
|
||||
x, y, w, h = detection["box"]
|
||||
detected_face = img[int(y):int(y+h), int(x):int(x+w)]
|
||||
return detected_face, [x, y, w, h]
|
||||
|
||||
|
||||
else: #if no face detected
|
||||
if not enforce_detection:
|
||||
if not enforce_detection:
|
||||
return img, img_region
|
||||
|
||||
|
||||
else:
|
||||
raise ValueError("Face could not be detected. Please confirm that the picture is a face photo or consider to set enforce_detection param to False.")
|
||||
|
||||
|
||||
elif detector_backend == 'retinaface':
|
||||
img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) #retinaface expects RGB but OpenCV read BGR
|
||||
|
||||
from retinaface import RetinaFace
|
||||
faces = RetinaFace.extract_faces(img_rgb, align = True)
|
||||
|
||||
if len(faces) > 0:
|
||||
face = faces[0]
|
||||
return face, img_region
|
||||
|
||||
else: #if no face detected
|
||||
if not enforce_detection:
|
||||
return img, img_region
|
||||
else:
|
||||
raise ValueError("Face could not be detected. Please confirm that the picture is a face photo or consider to set enforce_detection param to False.")
|
||||
|
||||
|
||||
else:
|
||||
detectors = ['opencv', 'ssd', 'dlib', 'mtcnn']
|
||||
raise ValueError("Valid backends are ", detectors," but you passed ", detector_backend)
|
||||
|
||||
def alignment_procedure(img, left_eye, right_eye):
|
||||
|
||||
|
||||
#this function aligns given face in img based on left and right eye coordinates
|
||||
|
||||
|
||||
left_eye_x, left_eye_y = left_eye
|
||||
right_eye_x, right_eye_y = right_eye
|
||||
|
||||
|
||||
#-----------------------
|
||||
#find rotation direction
|
||||
|
||||
|
||||
if left_eye_y > right_eye_y:
|
||||
point_3rd = (right_eye_x, left_eye_y)
|
||||
direction = -1 #rotate same direction to clock
|
||||
else:
|
||||
point_3rd = (left_eye_x, right_eye_y)
|
||||
direction = 1 #rotate inverse direction of clock
|
||||
|
||||
|
||||
#-----------------------
|
||||
#find length of triangle edges
|
||||
|
||||
|
||||
a = distance.findEuclideanDistance(np.array(left_eye), np.array(point_3rd))
|
||||
b = distance.findEuclideanDistance(np.array(right_eye), np.array(point_3rd))
|
||||
c = distance.findEuclideanDistance(np.array(right_eye), np.array(left_eye))
|
||||
|
||||
|
||||
#-----------------------
|
||||
|
||||
|
||||
#apply cosine rule
|
||||
|
||||
|
||||
if b != 0 and c != 0: #this multiplication causes division by zero in cos_a calculation
|
||||
|
||||
|
||||
cos_a = (b*b + c*c - a*a)/(2*b*c)
|
||||
angle = np.arccos(cos_a) #angle in radian
|
||||
angle = (angle * 180) / math.pi #radian to degree
|
||||
|
||||
|
||||
#-----------------------
|
||||
#rotate base image
|
||||
|
||||
|
||||
if direction == -1:
|
||||
angle = 90 - angle
|
||||
|
||||
|
||||
img = Image.fromarray(img)
|
||||
img = np.array(img.rotate(direction * angle))
|
||||
|
||||
|
||||
#-----------------------
|
||||
|
||||
|
||||
return img #return img anyway
|
||||
|
||||
|
||||
def align_face(img, detector_backend = 'opencv'):
|
||||
|
||||
|
||||
home = str(Path.home())
|
||||
|
||||
|
||||
if (detector_backend == 'opencv') or (detector_backend == 'ssd'):
|
||||
|
||||
|
||||
detected_face_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) #eye detector expects gray scale image
|
||||
|
||||
|
||||
eyes = eye_detector.detectMultiScale(detected_face_gray)
|
||||
|
||||
|
||||
if len(eyes) >= 2:
|
||||
|
||||
|
||||
#find the largest 2 eye
|
||||
|
||||
|
||||
base_eyes = eyes[:, 2]
|
||||
|
||||
|
||||
items = []
|
||||
for i in range(0, len(base_eyes)):
|
||||
item = (base_eyes[i], i)
|
||||
items.append(item)
|
||||
|
||||
|
||||
df = pd.DataFrame(items, columns = ["length", "idx"]).sort_values(by=['length'], ascending=False)
|
||||
|
||||
|
||||
eyes = eyes[df.idx.values[0:2]] #eyes variable stores the largest 2 eye
|
||||
|
||||
|
||||
#-----------------------
|
||||
#decide left and right eye
|
||||
|
||||
|
||||
eye_1 = eyes[0]; eye_2 = eyes[1]
|
||||
|
||||
|
||||
if eye_1[0] < eye_2[0]:
|
||||
left_eye = eye_1; right_eye = eye_2
|
||||
else:
|
||||
left_eye = eye_2; right_eye = eye_1
|
||||
|
||||
|
||||
#-----------------------
|
||||
#find center of eyes
|
||||
|
||||
|
||||
left_eye = (int(left_eye[0] + (left_eye[2] / 2)), int(left_eye[1] + (left_eye[3] / 2)))
|
||||
right_eye = (int(right_eye[0] + (right_eye[2]/2)), int(right_eye[1] + (right_eye[3]/2)))
|
||||
|
||||
|
||||
img = alignment_procedure(img, left_eye, right_eye)
|
||||
|
||||
|
||||
return img #return img anyway
|
||||
|
||||
|
||||
elif detector_backend == 'dlib':
|
||||
|
||||
|
||||
import dlib #this is not a must dependency in deepface
|
||||
|
||||
|
||||
detections = face_detector(img, 1)
|
||||
|
||||
|
||||
if len(detections) > 0:
|
||||
detected_face = detections[0]
|
||||
img_shape = sp(img, detected_face)
|
||||
img = dlib.get_face_chip(img, img_shape, size = img.shape[0])
|
||||
|
||||
|
||||
return img #return img anyway
|
||||
|
||||
|
||||
elif detector_backend == 'mtcnn':
|
||||
|
||||
|
||||
img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) #mtcnn expects RGB but OpenCV read BGR
|
||||
detections = face_detector.detect_faces(img_rgb)
|
||||
|
||||
|
||||
if len(detections) > 0:
|
||||
detection = detections[0]
|
||||
|
||||
|
||||
keypoints = detection["keypoints"]
|
||||
left_eye = keypoints["left_eye"]
|
||||
right_eye = keypoints["right_eye"]
|
||||
|
||||
|
||||
img = alignment_procedure(img, left_eye, right_eye)
|
||||
|
||||
|
||||
return img #return img anyway
|
||||
|
||||
elif detector_backend == 'retinaface':
|
||||
#we used extract_faces function of retinaface. it applies alignment anyway.
|
||||
return img #return img anyway
|
||||
|
||||
def preprocess_face(img, target_size=(224, 224), grayscale = False, enforce_detection = True, detector_backend = 'opencv', return_region = False):
|
||||
|
||||
|
||||
#img_path = copy.copy(img)
|
||||
|
||||
|
||||
#img might be path, base64 or numpy array. Convert it to numpy whatever it is.
|
||||
img = load_image(img)
|
||||
base_img = img.copy()
|
||||
|
||||
|
||||
img, region = detect_face(img = img, detector_backend = detector_backend, grayscale = grayscale, enforce_detection = enforce_detection)
|
||||
|
||||
|
||||
#--------------------------
|
||||
|
||||
|
||||
if img.shape[0] > 0 and img.shape[1] > 0:
|
||||
img = align_face(img = img, detector_backend = detector_backend)
|
||||
else:
|
||||
|
||||
|
||||
if enforce_detection == True:
|
||||
raise ValueError("Detected face shape is ", img.shape,". Consider to set enforce_detection argument to False.")
|
||||
else: #restore base image
|
||||
else: #restore base image
|
||||
img = base_img.copy()
|
||||
|
||||
|
||||
#--------------------------
|
||||
|
||||
|
||||
#post-processing
|
||||
if grayscale == True:
|
||||
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
|
||||
|
||||
|
||||
img = cv2.resize(img, target_size)
|
||||
img_pixels = image.img_to_array(img)
|
||||
img_pixels = np.expand_dims(img_pixels, axis = 0)
|
||||
img_pixels /= 255 #normalize input in [0, 1]
|
||||
|
||||
|
||||
if return_region == True:
|
||||
return img_pixels, region
|
||||
else:
|
||||
return img_pixels
|
||||
|
||||
def find_input_shape(model):
|
||||
|
||||
|
||||
#face recognition models have different size of inputs
|
||||
#my environment returns (None, 224, 224, 3) but some people mentioned that they got [(None, 224, 224, 3)]. I think this is because of version issue.
|
||||
|
||||
|
||||
input_shape = model.layers[0].input_shape
|
||||
|
||||
|
||||
if type(input_shape) == list:
|
||||
input_shape = input_shape[0][1:3]
|
||||
else:
|
||||
input_shape = input_shape[1:3]
|
||||
|
||||
|
||||
if type(input_shape) == list: #issue 197: some people got array here instead of tuple
|
||||
input_shape = tuple(input_shape)
|
||||
|
||||
|
||||
return input_shape
|
||||
|
||||
|
||||
BIN
icon/deepface-detectors.png
Normal file
BIN
icon/deepface-detectors.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 4.2 MiB |
@ -11,3 +11,4 @@ Flask>=1.1.2
|
||||
mtcnn>=0.1.0
|
||||
lightgbm>=2.3.1
|
||||
dlib>=19.20.0
|
||||
retina-face>=0.0.1
|
||||
4
setup.py
4
setup.py
@ -4,7 +4,7 @@ with open("README.md", "r", encoding="utf-8") as fh:
|
||||
long_description = fh.read()
|
||||
|
||||
setuptools.setup(
|
||||
name="deepface",
|
||||
name="deepface",
|
||||
version="0.0.51",
|
||||
author="Sefik Ilkin Serengil",
|
||||
author_email="serengil@gmail.com",
|
||||
@ -20,5 +20,5 @@ setuptools.setup(
|
||||
"Operating System :: OS Independent"
|
||||
],
|
||||
python_requires='>=3.5.5',
|
||||
install_requires=["numpy>=1.14.0", "pandas>=0.23.4", "tqdm>=4.30.0", "gdown>=3.10.1", "Pillow>=5.2.0", "opencv-python>=3.4.4", "tensorflow>=1.9.0", "keras>=2.2.0", "Flask>=1.1.2", "mtcnn>=0.1.0"]
|
||||
install_requires=["numpy>=1.14.0", "pandas>=0.23.4", "tqdm>=4.30.0", "gdown>=3.10.1", "Pillow>=5.2.0", "opencv-python>=3.4.4", "tensorflow>=1.9.0", "keras>=2.2.0", "Flask>=1.1.2", "mtcnn>=0.1.0", "retina-face>=0.0.1"]
|
||||
)
|
||||
|
||||
@ -47,6 +47,10 @@ print("-----------------------------------------")
|
||||
|
||||
print("Face detectors test")
|
||||
|
||||
print("retinaface detector")
|
||||
res = DeepFace.verify(dataset, detector_backend = 'retinaface')
|
||||
print(res)
|
||||
|
||||
print("ssd detector")
|
||||
res = DeepFace.verify(dataset, detector_backend = 'ssd')
|
||||
print(res)
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user