This article introduces a design that aims threw the combination of open source and closed source technologies, to make a, simple to implement,low-cost and high-performing face recognition solution. The solution provides identification, emotions and facial features recognition as well asdangerous objects spotting.This article exposes the concept of the solution, explains its importance on the market and provides details of a proof of concept prototype.
The market of face and image recognition technologies is booming and forecasted a brilliant future. Although it is seen more and more in specialised magazine or promoted by giants of information technologies, many smaller actors are left behind as they perceive the technology as inaccessible or too expensive.
Numerous researchesaboutthose systems have been made in the recent times and during those years
of research, computers science has evolved beyond measure.But what really have changed since a few
years, are the cameras. What makes this ground of research more prolific than ever todays is that we
all havephones in our pockets which sensors have an average of 14 megapixels, that we can buy full
HD webcams for less than a hundred dollars. 15 years ago, a digital camera’s resolution would be a
fifth of what a webcam hasnow and be ten times its price. [
Although face recognitions attempts have been around for more than 50 years now, it still appears
as a new technology to most of people. If we had indeed technologies able to perform those tasks back
in the sixties, pictures would have to be taken according to very precise specifications. Attempts were
multiplied; it became a trend in the nineties, some artefacts from that time, such as the ORL Database
if Faces from Cambridge are even still in use today. In the beginning of the two-thousands, an
international contest has even been thrown on the subject of face recognition. [
Nowadays, we have the tools, we have the necessary sensors for an efficient recognition and new
actors on this market are emerging every day.Those solutions represent a trend on the security market
of course; it allows to recognise not only people, but also specific objects and track them if necessary.
The industry alsostarts to use emotion recognition systems to understand better their customer. [
As the market is still emerging but have been around for a long time, both open-source and closed
source solutions exists. Closed source solutions are efficient to spot faces,can differentiate them,
making an authentication possible. Those solutions, however, falls short when it comes to analyse a
picture’s details, such as emotions, facial details or objects. Closed source image recognition
providers, however, are usually specialized and therefore extremely good when it comes to identifying
those details. [
The design presented in this paper tries to take profit of this reality. Combining open source technologies and closed source ones, taking to both worlds what they are good at, allows making a first analysis on a local computer, even one having low computation capabilities and, over the internet, using solutions provided by the majors of image recognition, to analyse pictures in-depth,beyond the capacities of open source solutions.
As mentioned, the face recognition market is still emerging. It is expected to be worth between 7.5 and
10 billion dollars by 2022, 2 to 3 time more than it was worth in 2016. The year before that, the main
client of those systems was US Homeland Security.By now the use of such solution for security has
already spread in several countries and is used by such actors as the British police. Since its beginning
this technology has been viewed as a major asset in security systems. [
Open source solutions are forecasted to improve their algorithms in 2D and thermal face
recognition, while it is believed that online services will keep the specialized market (complex
emotions, facial features details, 3D modelling, etc...) , although open source alternatives exists and
will also improve, but not with the same precision rate. [
Making profitable for SMEs (Small and medium-sized enterprises), which are 98% of economic environment, a multi-billions digital economy market threw the design presented is a breakthrough for face recognition as it makes it an accessible tool.
In this design, if the picture is of sufficient quality for an optimal analysis[
If the face hasn’t been recognized on the first database, a query is sent to the second one, which can store up to a thousand of faces, depending of the capacities of the computer (2). This database is typically conceived to store the faces of all the employees of a company and manage access controls.
If no match is found in the second database (the confidency of the comparison between the shown face and the existing ones is too low), the system querries online services, that can analyse the picture, confirm that the idividual in unknown via an online database (3), and differenciate its emotions, facial features as well as alaysing its environement, detecting immediate threats, such as weapons.
Finnaly, the result of this detection will be added to the RAM-loaded database to avoid detecting and alaysing again the same face (4). Each query to an online service having of course a cost.
The performance reached by the test program fitted all of our expectations, if sometimes the
description suffers small imprecisions it offers a real-time identification on video with 5 frames per
second, spotting simultaneously several object[
Although some obvious progress are to be made on the hair colour detection, the features calculated are generally close to reality and most importantly allowsa human identification of a person, even without a the subsequent picture.
A software has been developed as a design proof of concept. It has been developed in C#, using an OpenCV wrapper for this platform, OpenCvSharp, Fisherfaces recognition algorithm and Microsoft’s Face and Vision API.
The use of Fisherfaces has been motivated over other methods for its search of discrimination
criteria, which is more reliable to exclude possible faces match, enhancing the security offered by the
solution. We widely favour a false negative, which leads to a control on the server that the person truly
isn’t identified in our database, than a false positive, which would allow an intruder to get through the
system. [
The use of the Microsoft cognitive systems has been decided as it fitted the technical needs of the environment, offered a good transparency and as they send back details from the analysis of the image such as face coordinates, allowing further extrapolation. The other considered providers whichbilling systems were adapted to this design were Google Cloud Platform and IBM Watson.
The goal being to make the market as accessible as possible, it was important to reduce every
source of costs. The system has been tested on several Microsoft Windows platforms (Win 7 and
superior versions), it function and manage real-time recognition on computers having 4Gb of RAM, a
dual core processor and a SSD of 64 GB,inferior configurations haven’t been tested.
3.4. Costs
The design described here is of course flexible, meaning any online service could be used alternatively
to Microsoft’s. The calculation of costs for such an access control system was made considering an
arbitrary a company size of a hundred workers (big company on the SME environment). Considering
each of the employees comes into the company’s building twice a day every working day, it is 4000
controls a month. If those faces are all stored locally, they should be recognized and therefore not
generate any cost. If every day fifty unknown person comes into the building it will make about a
thousand controls that are not perceived by the local recognition system. Those numbers all falls under
the “free calls pool” of Microsoft Azure subscription, even considering that some queries of the
analysis must be done in several steps, generating as many calls. However, this represents a laboratory
reality which always differs from the “field”. For the same amount of people, used in a production
environment, the price of the online analysis has been calculated to be about 10 to 15 dollars a month,
taking in account all the frequent errors of the software. [
As a onetime cost, it is necessary to get a small computer and a webcam to run the software.
Multiple devices have been assessed on that purpose, all in a price range of 250 to 350 dollars for the
computer and as for the camera between 50 and 80 dollars. For a total cost of 300-430 dollars a door.
Counting the cost of electricity to power the system, the total cost of the installation is estimated to
1500 dollars for a period of 5 years (total cost of ownership).
4. Reliability
The test program realized to proof this design is able to distinguish similar faces such as twins easily.
The confidence criterion has been configured severely, to make sure the local recognition system
wouldn’t give any false positive. This confidence has been set according to previous researches. [
To assess the efficiency of the software, some further tests and comparison have been conducted.
The computer has been presented pictures from five pairs of twins identified in the database and two
pairs of pictures of the same person on different pictures and has to differentiate them. Humans, on the
other hand, have been presented a similar set of pictures and were simply asked, having two seconds
for each picture to tell which subjects were twins and which were not. [
The precision of the software couldn’t be assessed with accuracy, as so far, the program hasn’t been cheated on successfully. Whenever the confidence of the local face analyser is too low, faces are sent online for analysis. Since the program is at its final stage of development, the success rate has been of a hundred percent. Therefore, the upcoming paragraph, assessing the reliability of such systems, is based on external information and other systems.
Unveiling its last iPhone, Apple claimed its face recognition system has a reliability of one in a
million, meaning that once in a million times two faces would be confused and recognised as being the
same, this is the closest comparison possible to do to the online services used. [
As underlined in this presentation, face recognition is a fast-developing market at the moment, much is already done but much is left to be built and this design has a place in the development of the market. Every major actor involved in security should now consider getting themselves an access to this kind of technology, especially now it is more accessible than ever and as the market trend makes it very profitable.
In the future, the detection will be improved by assessing the liveness of faces. Checking that we
are not given a picture of a face but that it is a genuine face we have in front of the camera. This can be
made by different methods, but the most adapted to a system of those dimensions is the analysis of the
micro-behaviour of the eyes. [
This kind of system could also be used on security cameras to get frames with a higher resolution and filter them threw an artificial intelligence able to understand which frames are relevant by an analysis of the pictures. Allowing selecting only relevant frames for storage, gives the possibility to significantly augment the quality of the camera’s captors without being confronted to the problem of the storage space saturation. Emotion recognition and specifically this design can be adapted to the numerous of other uses such as home automations, alarms, research of wanted persons and many others that haven’t been mentioned in this article. It is up for everyone, on this new market, to develop their own ideas.
Of course, this paper wasn’t about a purely technical breakthrough, however I hope that the reader understands better now the face recognition market, how to use it efficiently and make it profitable, in particular with the design offered.This kind of design will make the difference between an emerging market and a fully grown and accessible one, bringing a new technology to the consumer. In other words, I want everyone to understand how face recognition systems are now in the reach of their hands.