Using Microsoft Cognitive Services to Add Facial Recognition to Your Apps

Introduction

In my first article of the Cognitive Face API series, “Adding Facial Recognition to Your Apps,” I explained the process of face detection, features, subscription steps, and API service calls. In this tutorial on the Face API, I will create a Universal Windows Platform (UWP) app consuming the Microsoft Azure Cognitive Services API to demonstrate some of the characteristics of face recognition.

Prerequisites

1) You need to subscribe to Microsoft Azure Cognitive Services and get the subscription keys of the Face APIs. In my previous article, I explained the subscription steps.

2) Be sure you’re running the Microsoft Windows 10 OS. Universal Windows Platform (UWP) requires the Windows 10 operating system.

3) You’ll need Visual Studio 2015 Community Edition or the Preview Version of Visual Studio 2017 (free downloadable software is available from the Microsoft Webs ite).

4) You’ll need to reference the following NuGet packages. These packages can be downloaded and installed from the Visual Studio Solution Explorer window.

  • Microsoft.ProjectOxford.Face
  • Newtonsoft.Json
  • Win2D.uwp

Implementing the Face API

The face API can be accessed via C# code by using the Microsoft.ProjectOxford.Face NuGet package. By reading this article, you can learn how to use the Cognitive Service Face API in Universal Windows Apps development with Azure, XAML, and C#. The application development steps are simple and it cover the features mentioned below.

  1. Implement the Image upload functionality in UWP.
  2. Detect faces in the uploaded image.
  3. Show the detected faces highlighted in the XAML UI.

Setting Up the Project

Following are the steps to follow for app development.

Step 1

Open Visual Studio 2015. Go to Start -> New Project-> Select Universal (under Visual C# -> Windows ) -> Blank App -> Give a name for your app (MyNewFaceAPIApp) -> and click OK.

New project window
Figure 1: New project window

Entering MyNewFaceAPIApp as a new project name
Figure 2: Entering MyNewFaceAPIApp as a new project name

Select the target and minimum platform version that your UWP application will support.

UWP app target and minimum version
Figure 3: UWP app target and minimum version

Step 2

By default, developer mode is not enabled in Windows 10. Go to Start -> Windows Settings-> For Developers. Install the required packages to enable developer mode.

Windows settings developer mode
Figure 4: Windows settings developer mode

During developer mode enablement, I had to troubleshoot the following errors. I have mentioned a fix of each of the errors. These errors are due to failures during a Windows upgrade.

Error: DEP0100: Deployment failed due to a Developer Licensing issue. Could not obtain a developer license due to error 800704C7.

Fix: https://stackoverflow.com/questions/36324300/ensure-that-target-device-has-developer-mode-enabled-could-not-obtain-a-develop

Error: Developer mode package failed to install. Error code 0x8004005

Fix: http://windows10trickss.blogspot.in/2016/10/How-to-fix-Developer-Mode-package-failed-to-install-Error-code-0x80004005.html and https://social.msdn.microsoft.com/Forums/en-US/a7e94e5b-db19-492f-a1c1-d5fa3aa87d0d/enabling-developer-mode-fails-with-error-code-0x80004005?forum=Win10SDKToolsIssues

Step 3

Add the following references in the project by using the NuGet package manager.

  • Microsoft.ProjectOxford.Face
  • Newtonsoft.Json
  • Win2D.uwp

To add the Microsoft.ProjectOxford.Face reference, right-click your project (MyNewFaceAPIApp) and select “Manage NuGet Packages”.

NuGet package manager
Figure 5: NuGet package manager

Choose “Browse” and search “Microsoft.ProjectOxford.Face”. Select the package and install it.

Microsoft.ProjectOxford.Face installation
Figure 6: Microsoft.ProjectOxford.Face installation

Package manager will ask for confirmation before installation.

Confirmation before Installation
Figure 7: Confirmation before Installation

Accept the license agreement.

Accepting the license
Figure 8: Accepting the license

After successful installation, a Microsoft.ProjectOxford.Face reference will be added to the project.

Successfully installed Microsoft.ProjectOxford.Face package and added reference
Figure 9: Successfully installed Microsoft.ProjectOxford.Face package and added reference

Next, using the NuGet package manager, install a Newtonsoft.Json reference.

Newtonsoft.Json installation
Figure 10: Newtonsoft.Json installation

Choose “Browse” and search for “Newtonsoft.Json”. Select the package and install it. After successful installation, a Newtonsoft.Json reference will be added to the project.

Successfully installed Newtonsoft.Json package and added reference
Figure 11: Successfully installed Newtonsoft.Json package and added reference

Now, add a Win2D.uwp reference, used for drawing a rectangle in the face. From the NuGet package manager, choose “Browse” and search “Win2D.uwp”. Select the package and install it.

Win2D.uwp Installation
Figure 12: Win2D.uwp Installation

Package Manager will ask for confirmation before installation.

Confirm before installation
Figure 13: Confirm before installation

Accept the license agreement.

Accepting the license
Figure 14: Accepting the license

After successful installation, a Win2D.uwp reference will be added to the project.

Successfully installed Win2D.uwp package and added reference
Figure 15: Successfully installed Win2D.uwp package and added reference

The following NuGet packages will be added as a project reference.

All NuGet package references
Figure 16: All NuGet package references

Step 3

Add a facial image to the “Assets” folder for detecting. I have added my photo.

Added image in asset folder
Figure 17: Added image in asset folder

Step 4

Add the following XAML code in MainPage.xaml. I have used the Win2d library to draw a rectangle on the canvas.

<Page
   x_Class="MyNewFaceAPIApp.MainPage"
   
   xmlns_x="http://schemas.microsoft.com/winfx/2006/
      xaml"
   xmlns_local="using:MyNewFaceAPIApp"
   xmlns_d="http://schemas.microsoft.com/expression/
      blend/2008"
   xmlns_mc="http://schemas.openxmlformats.org/
      markup-compatibility/2006"
   xmlns_customCanvas="using:Microsoft.Graphics.Canvas
      .UI.Xaml"
   >
   <Grid Background="{ThemeResource
      ApplicationPageBackgroundThemeBrush}">
   <Image Source="Assets/tapas.jpg"
      Stretch="None"/>
   <customCanvas:CanvasControl Draw="MyCanvas_Draw"
      x_Name="MyCanvas"/>
   </Grid></Page>

Step 5

Add the following namespaces in Mainpage.xaml.cs for face detection and drawing the rectangle.

using Windows.Graphics.Imaging;
using Windows.Storage;
using Windows.Storage.Pickers;
using Microsoft.ProjectOxford.Face;
using Microsoft.ProjectOxford.Face.Contract;
using Microsoft.Graphics.Canvas.UI.Xaml;

Step 6

Add the following code in MainPage.xaml.cs. The MyfaceAPIClient object will enable connections with the API. The DetectMyFaces function will upload my image, calling the API for facial recognition. Once the face is recognized, MyCanvas_Draw will draw a rectangle in the picture.

private readonly IFaceServiceClient MyfaceAPIClient =
      new FaceServiceClient("36ea74183e9a4213975ae3da1059b854");
   FaceRectangle[] MyfaceRectangles;
   public MainPage()
   {
      this.InitializeComponent();
      DetectMyFaces("ms-appx:///Assets/tapas.png");
   }

   async void DetectMyFaces(string imageFilePath)
   {
      try
      {
         StorageFolder MyappFolder = Windows.ApplicationModel
            .Package.Current.InstalledLocation;
         StorageFolder assets = await MyappFolder
            .GetFolderAsync("Assets");
         var MyFile = await assets.GetFileAsync("tapas.png");
         var MyAccessStream = await MyFile.OpenReadAsync();

         using (Stream stream =
            MyAccessStream.AsStreamForRead())
         {
            var myfaces = await
               MyfaceAPIClient.DetectAsync(stream);
            var myfaceRects =
               myfaces.Select(face => face.FaceRectangle);
            MyfaceRectangles = myfaceRects.ToArray();
            MyCanvas.Invalidate();
         }
      }
      catch (Exception ex)
      {
         System.Diagnostics.Debug.WriteLine(ex.Message);
      }
   }

   void MyCanvas_Draw(CanvasControl sender,
      CanvasDrawEventArgs args)
   {
      if (MyfaceRectangles != null)
         if (MyfaceRectangles.Length > 0)
         {
            foreach (var faceRect in MyfaceRectangles)
            {
               args.DrawingSession.DrawRectangle(faceRect.Left,
                  faceRect.Top, faceRect.Width, faceRect.Height,
                  Colors.Red);
            }
         }
      }

Run the application. If a face is present in the image, a rectangle will be drawn in the picture.

Application execution output
Figure 18: Application execution output

Summary

In this article, I tried to show how to use the Microsoft Cognitive Services Face API with the Universal Windows 10 App. There are more Cognitive APIs to use, such as Computer Vision API, Emotion API, Speaker Recognition API, and Text Analytics API. You can subscribe for these API’s from Azure portal and build intelligent applications. That’s all for today. Stay tuned for more.

More by Author

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Must Read