Learn about face detection with Haar cascades, gender recognition with convolutional neural networks, and age recognition with convolutional neural networks.
We all have heard about OpenCV, the famous C++ library for computer vision-related applications. Since it is written in C++, many wrappers/bindings (such as OpenCV Python) have been created to provide the functionalities provided by the OpenCV library to other languages. There’s also emgucv, the. NET wrapper for OpenCV. OpenCV has a Java binding, too, but that is not JavaCV. JavaCV is more than a Java binding.
JavaCV is a collection of wrappers for many famous libraries used for image processing and computer vision-related applications. JavaCV includes wrappers for OpenCV, FFmpeg, OpenKinect, and much more. JavaCV has everything required for computer vision and image processing in one place. Currently, JavaCV has released its latest version (1.3) of wrappers which support the OpenCV 3.1 through JNI. Even though JavaCV comes with a lot of features (OpenCV and much more) , it is quite less famous among developers. Therefore, I’ve decided to put some light on JavaCV and expose what it’s capable of doing. Let’s get started!
Here, we will be doing some face recognition/detection stuff and furthermore and use convolutional neural networks (CNNs) for age and gender predictions. Since most of you have seen how to do face detection using Haar cascades and how to do face recognition using fisherfaces and so on, the interesting part will be the usage of CNNs for age and gender predictions. First, let’s create a Maven project and add a JavaCV dependency as follows. I will be using JavaCV version 1.2. The latest version available is JavaCV 1.3, which has several improvements over the version that I’ m using here.
Here, the parameter taken in by the constructor of FFmpegFrameGrabber is the camera device (“/dev/video0” is our default laptop webcam; if you connect another camera, it will be identified by the Linux system as “/dev/video01” and so on) . Since I’ m a Linux fan, I have done this example on an Ubuntu machine. The camera device and format (“video4linux2”) may differ in Windows environments. However, the steps to be followed are still the same.
Next, the captured frame’s width and height are specified. If you have a 720p HD camera, this resolution is optimal. Once set up, we can start the frame grabber.
Once started, we have to repeatedly grab frames from the frame grabber as follows.
In the above code, we are capturing frames from the webcam. These frames are not shown in a UI, so let’s do that part along with adding the gender and age recognition functionality to our code.
This is a part most of us at least have heard of. OpenCV/JavaCV provide direct methods to import Haar cascades and use them to detect faces. I will not be explaining this part in deep detail. You can refer this class in GitHub and this blog post to learn more on using Haar cascades to detect faces. I have written the HaarFaceDetector class to detect faces.
The following code snippet initializes the Haar cascade classifier:
This method detects faces in a grabbed frame and crops the faces. I have put the cropped faces along with the coordinates of the faces (as CvRect objects) into a Map.
That’s it for face detection. Let’s see how these detected faces are processed to detect gender and age.
Gender recognition using OpenCV’s fisherfaces implementation is quite popular. But, in this example, I will be using a different approach to recognize gender. This method was introduced by two Israel researchers, Gil Levi and Tal Hassner, in 2015. I have used the CNN models trained by them in this example. We are going to use the OpenCV’s DNN (Deep Neural Networks) package.
A Caffe model has two associated files:
The following code segment is responsible for loading the CNN definitions and the trained models (CNNGenderDetector.java) :
…which do the importing and population of the CNN.
Following is the processing part:
First, I resize the face (of type Mat passed as a method parameter) and fill the resized Mat to a new Mat. This is because the trained model has been trained to take inputs of size 256×256. Therefore, I resize the face Mat to that size. I have normalized the Mat.
Then, I create a Blob using the face Mat and set that Blob as the input of the first layer of the CNN. .data layer. genderNet.forward () will then move the input through the layers of the neural network. Finally, we can take the output of the last layer of the CNN, which is the probability layer (prob) . In JavaCV, we have to use indexers to iterate matrices and blobs since JavaCV is just using the JNI to invoke OpenCV methods. This gender recognition CNN outputs two values which are indexed as (0,0) and (0,1) in a one-dimensional matrix. The value at (0,0) corresponds to the probability of the face being a male and (0,1) to the probability of it being female. Since these values sum up to 1, the greater one will be the predicted gender.
The complete class is available here .
This is almost similar to the gender detection part, except that the corresponding prototxt file and the Caffe model file are “deploy_agenet.prototxt” and “age_net.caffemodel”. Furthermore, the CNN’s output layer (probability layer) in this CNN consists of eight values for eight age classes (“0–2”, “4–6”, “8–13”, “15–20”, “25–32”, “38–43”, “48–53”, and “60-“) . The CNNAgeDetector class is written to process detected faces and predict their age classes.
Similarly to the gender recognition scenario, first I resize the face (of type Mat passed as a method parameter) and fill the resized Mat to a new Mat because the model has been trained to take inputs of size 256×256. Now, I have normalized the Mat.
Then, I create a Blob using our face Mat and set that Blob as the input of the first layer of the CNN. .data layer. ageNet.forward () will then move the input through the layers of the neural network. Finally, we can take the output of the last layer of the CNN which is the probability layer (prob) . minMaxLoc (prob.matRefConst () , null, pointer, null, max, null) gets the predicted age class. The neural network returns an array of eight values consisting of probabilities of each age class being the predicted age of the user. The complete class can be found here.
I have created the main class JavaCVExample to put together the things we have done so far and come up with a small UI application.
