Abstract: The research on the facial expression detection or the so-called emotion detection has been multiplying day by day. With effective judgement of feelings, we could get instant feedback from clients, increase better comprehension of the human conduct while utilizing the information technologies and in this way make the frameworks and UIs progressively emphatic and intelligent. A human-PC connection framework for an automatic face recognition or outward appearance recognition has pulled in expanding consideration from specialists in psychology, software engineering, etymology, neuroscience, and related orders. People have consistently had the intrinsic capacity to recognize faces and recognize expressions. Our challenge is to make a computer to do the same. In other words, to make a computer behave and understand like a human. Sounds interesting right. This opens huge amounts of utilizations. Facial Expression detection and Recognition can be utilized to improve access and security like the most recent Apple iPhone does, enable instalments to be handled without physical cards — iPhone does this as well! empower criminal identification and permit personalized healthcare and different administrations. Facial expressions detection and recognition is a vigorously explored point and there are huge amounts of assets on the web. We have attempted different open source activities to locate the ones that are least difficult to actualize while being precise.

Keywords: Automatic face recognition, Human-PC connection framework, Lung Cancer, Personalized healthcare.

clinical psychology, feeling detection and torment appraisal. There are at most three significant strides in an Expression Recognition System 1. To distinguish the face from the given info picture or video, 2. To remove the facial features like eyes, nose, mouth from the distinguished face and 3. To group the facial expressions into various classes like Happy, Angry, Sad, Fear, Disgust, and Surprise. Face detection is an exceptional instance of object detection. In the proposed system, face detection is executed utilizing skin color detection and division. Additionally, it involves predefined packages in python like Keras, OpenCV and Face Detection module. Let’s talk about these modules in detail.

A. Keras

Keras is like an inside of an software. Keras is an API inside python used to develop neural networks. Its main application is the creation of the convolution neural networks. A neural network is a process which learns and updates itself. This type of self-updating processes is useful in decreasing the execution time and increasing the performance of the program. A model is comprehended as an arrangement or a chart of standalone, completely configurable modules that can be stopped together with a couple of confinements as could be allowed.

I. INTRODUCTION

Face assumes a significant job in non-verbal or physical communication. Face identification or detection itself is utilized in numerous applications like measurable, security and other business applications. Thus, facial emotions or expressions are the quickest methods for communication, while passing on a data. In 1978, Ekman and Frisen announced that, Happy, Sad, Anger, Fear, Disgust and Surprise are the six fundamental expressions which are promptly perceived crosswise over altogether different societies. A system which recognizes the expressions on its own is called Automatic Facial Expression Recognition System (AFERS). The powerful Expression Recognition system can be used in many regions of science, for example

Revised Manuscript Received on January 15, 2020

D. Ram Kiran, Department of ECE, Koneru Lakshmaiah Education Foundation, Guntur, Email: ramkiran55.devireddy@gmail.com

K .VinayKumar, Department of ECE, Koneru Lakshmaiah Education Foundation, Guntur, Email: vinaykumarkarusala@gmail.com

T. Kalyan, Department of ECE, Koneru Lakshmaiah Education Foundation, Guntur, Email: kalyan6166@gmail.com

Dr. K. Ch. Kavya, Department of ECE, Koneru Lakshmaiah Education Foundation, Guntur, Email: kavya@kluniversity.in

Dr. K. SARAT KUMAR, Department of ECE, Koneru Lakshmaiah Education Foundation, Guntur, Email: kksarat@kluniversity.in

B. OpenCV

OpenCV or the Open Source Computer Vision Library is like an eye for a machine. OpenCV is nothing but the vision to a computer. This library is used for machine-learning modules and artificial-intelligence. OpenCV’s typical framework was built to give computer vision applications and to quicken machine’s recognition capabilities in the business products. OpenCV is very easy to use framework.

C. Face detection

Face detection is a computer technology being utilized in an assortment of uses that identifies human faces in computerized pictures. Face identification likewise alludes to the psychological process by which humans find and take care of faces in a visual scene.

II. PREVIOUS WORKS

There are tons of works done in this domain. We took few methodologies as reference for this paper.

A. RGB Model

One of those methodologies is emotion detection using RGB model. In this model firstly, the video capturing is used to capture frames.

Once the image is captured, face detection module is applied to extract face from the image. Then comes the major part, the extracted face is divided into segments using RGB processing. Eyes, lips and eyebrows are the major segments considered. From these segments, emotion is classified and using a data write the emotion which is classified is written in the form of a text. Pros for this model is the pre-processing. The face extraction from the image is done quite easily and we have added this to our implementation. Cons for this model is the emotion classification. It takes a lot of time to process the image and classify the emotion

detected is compared with the images under these folders and the percentage of matching for each folder is stored in a list. The index of the value which is highest among the list is retrieved. If this highest value is less than the threshold (40%), neutral is given as output. It the highest value is greater than the threshold, its index is taken. If the index value is 0, happy is given as output, if the index is 1, sad is given and so on. The output which is collected is then printed over the image using the data write command. The advantage of this model which we have developed is quite accurate and take’s moderate time for processing. But the disadvantage for this model is, more expressions and feature adding, delays the processing.

B. Comparison Model

The other methodology is a comparison model. In this model, frames capturing is done based on OpenCV package. Once the frames capturing is done, face is separated from the frames of images using any face recognition model. The extracted face is then compared with a dataset. A dataset is collection of images. Collection of these images contain almost all the categories of these images. Once any one of the images is matched with the face with the matching percentage of about 40%, the output is given. The output is nothing but the category under which the image is matched. Advantage for this model is the emotion classification. The comparison is one of the accurate and finest models. But the comparison is not done for the full extent. That is the disadvantage of this model. We have modified this model to be a full comparison model.

III. IMPLEMENTATION

Considering the second model above as the primary model, we have developed a model for the classification of emotion. In this model comparison is done for full extent. In this model firstly, video capturing is done in order to capture frames.

Fig-1: Flow Diagram of Emotion Detection

From the captured frames the face is detected and extracted using face recognition modules in python (like the first model). From here the tricky part starts.

The comparison model is used from here. The detected face is then compared to the categories of images. Prior to this implementation a dataset is downloaded and categorized under happy, sad, surprise, angry and fear. The face which is

IV. KEY STEPS

Firstly, import all the pre required libraries. Libraries which are available in python. This is most important and the toughest part. Some of them has to be installed using a pip installer in python. The libraries are OpenCV, NumPy for image processing. Now we need other libraries for face recognition, for giving the output. Hence, we have to import detect faces and draw text packages in python. There after we need to get the required inputs from a dataset. The dataset we choose to be is fer2013 dataset. We need to get all the labels from the dataset into our program. Now fix the window size and face boundaries and import draw_boundingbox. We are now going to start our processing. Firstly, capture images using OpenCV or cv2. Using video capture command read images continuously. After reading these images convert them into grey scale and RGB models for further image processing. Further processing includes applying offsets on the face coordinates and emotion offsets for which we need to import apply offsets package from python. Complete all the logical processing in order to find the emotion or facial expression. Get the emotion text and append it on the face using draw text which have imported. Ensure that the emotion window size should be less than emotion window size. Methodology is shown in the below Figure.

V. DATASET

The fer2013 collection consists of the grey scale images of size 48 x 48. The images have been automatically saved such that the face is at the middle and takes about the same percent of space in every image. The task is to separate every face based on the expression mentioned in to one of seven categories (0=Happy, 1=Sad, 2=Surprise, 3=fear, 4=angry, 5=Neutral). This collection of data set contains the images under every category and can be able to import in to the programming like python. This helps in the separation of images based on their expressions. The set of images under these folders can be compared in order to get the matching percentage. Matching percentage is the amount of overlapping over the image. This measure of overlapping gives values precisely which are then used to estimate the expression. The list which is used to store these percentages has two columns. They are the "expression" column and the "matching percentage" column.

The prediction of emotion is done based on the comparison with the dataset. The dataset has almost 3589 examples and the training set consists of the 28709 examples. Pierre-Luc Carrier and Aaron Courville were the ones who prepared this collection.

VI. RESULTS AND DISCUSSION

E. Surprise

Table- V: Tabular values for emotion Surprise

The index ‘2’ is highest and is greater than 40%. Thus, Surprise is given as output.

F. Fear

Table- VI: Tabular values for emotion Fear

The index ‘3’ is highest and is greater than 40%. Thus, Fear is given as output.

Fig-2. Results

A. Happy

Table- I: Tabular values for emotion Happy

The index ‘0’ is highest and is greater than 40%. Thus, Happy is given as output.

B. Sad

Table- II: Tabular values for emotion Sad

VI. FUTURE SCOPE

The future scope for this paper is obviously going to be accuracy which we can never compromise on. The accuracy can be increased by increasing the samples in the dataset. But this also increases the time and complexity. We also need to add more expressions like normally happy, excited, tremendously happy, very sad, bold, cold etc. But this again increases the time of processing and the complexity. Thus, for this model must be altered. One of the modifications that can be done is adding a neural network. A new neural network which alters the dataset after every execution. The neural network must take the image of the user and it must add the image to one of the categories in the dataset. This can achieve high reduction in processing time. Because the users who comes again will be given output immediately without any execution or processing.

The index ‘1’ is highest and is greater than 40%. Thus, Sad is given as the output.

C. Neutral

Table- III: Tabular values for emotion Neutral

The index ‘4’ is highest and but is less than 40%. Thus, Neutral is given as the output.

D. Angry

Table- IV: Tabular values for emotion Angry

The index ‘4’ is maximum and is greater than 40%. Thus, Angry is given as output.

VII. APPLICATIONS

1)One of the major applications is going to be security.

2)This can be added to face unlock, face with only certain emotion will unlock the mobile.

3)The other application is to give a driving alert.

4)While feeling drowsy and driving a vehicle the emotion like sleepy can be recognized and intimidated to the user which ensures safety.

5)With perfect judgment or classification of emotions, we could get instant and genuine feedback from clients which a big factor in marketing products

VIII. CONCLUSION

Therefore, one the best thing about this project is its simplicity. It can be implemented by using python. One of the major applications is going to be security. This can be added to face unlock, face with only certain emotion will unlock the mobile. In other words, with simple emotion detection module there are a lot of advantages. We can get instant feedback which will be helpful

for marketing.