Face Detection and Face Landmark Using JavaCV and CameraX in Android

There are a lot of examples that detect faces in an image or from a live feed. But very few are worth reading.

We can detect faces and landmarks using OpenCV, Dlib, Tensorflow Lite, ML Kit, and so on. Today we are going to learn about face detection and face landmarks using the JavaCV library (Java wrapper for OpenCV).

Why JavaCV? Why not OpenCV Java SDK?

JavaCV provides some OpenCV extension which we can’t have from OpenCV Java SDK.

So let’s get started.

At first, we need some dependencies to work with the JavaCV library.

Add this below code in your app-level build.gradle file.

android {
    packagingOptions {
        exclude 'META-INF/native-image/**'
    }
}

Now add these dependencies

implementation "org.bytedeco:javacv:1.5.5"
implementation group: 'org.bytedeco', name:  'opencv-platform', version: '4.5.1-1.5.5'
implementation group: 'org.bytedeco', name:  'opencv-platform', version: '4.5.1-1.5.5:android-arm'
implementation group: 'org.bytedeco', name:  'openblas-platform', version: '0.3.13-1.5.5:android-arm'

To use CameraX library

// CameraX Library
def camerax_version = "1.1.0-alpha01"
// CameraX core library using camera2 implementation
implementation "androidx.camera:camera-camera2:$camerax_version"
// CameraX Lifecycle Library
implementation "androidx.camera:camera-lifecycle:$camerax_version"
// CameraX View class
implementation "androidx.camera:camera-view:1.0.0-alpha21"
// If you want to additionally use the CameraX Extensions library
implementation "androidx.camera:camera-extensions:1.0.0-alpha21"

To use Lifecyclescope

implementation("androidx.lifecycle:lifecycle-runtime-ktx:2.3.0")

Now sync your project. And we are good to go. In this tutorial, we are focusing only on face detection and face landmarks. So we won’t discuss anything about CameraX library.

To detect faces we are going to use the haarcascade_frontalface_default.xml model.

To detect face landmarks we are going to use the facemarks API and face_landmark_model.dat model.

fun detectFaces(mat: Mat): RectVector {
    // convert to grayscale and equalize histograe for better detection
    val grayMat = Mat()
    cvtColor(mat, grayMat, COLOR_BGR2GRAY)
    equalizeHist(grayMat, grayMat)

    // Find faces on the image
    val faces = RectVector()
    faceCascade.detectMultiScale(grayMat, faces)

    Log.d(TAG, "Faces detected: ${faces.size()}")

    return faces
}

In the above code, we are passing a mat object (converting an image proxy to bitmap and bitmap to mat object) and detect faces using Cascade Classifier.

fun detectLandmarks(mat: Mat, faces: RectVector): Point2fVectorVector {
    val landmarks = Point2fVectorVector()

    // Run landmark detector
    facemark.fit(mat, faces, landmarks)

    return landmarks
}

Then we detect landmarks from the faces.

Happy Coding!!! :)

https://github.com/shaon2016/Face-Landmark-Using-JavaCV