Empa (FOSS Project)

Empa (Emotional Intelligence Partner) is a software tool that can be used to detect emotions from facial expressions.

Empa is a full-stack web application that uses computer vision and machine learning to detect emotions in real time using facial expressions. It is built to support individuals with communication disorders, those on the autism spectrum, people navigating cross-cultural communication, and more. Empa supports users in making their interactions more thoughtful and inclusive.

Emotional intelligence is key to how we connect with others, but not everyone has equal access to tools that help them understand and/or express emotion. Empa was created to fill this gap by using facial data. It gives users real-time insight into emotional states, helping them respond more empathetically in conversation.

Empa relies on open datasets and open-source tools:

• Facial expressions are analyzed through MediaPipe landmark extraction
• Emotion classification is performed using models trained on the FER2013 dataset
• Real-time emotion recognition achieved via webcam using a custom-trained, low latency model
• Radar chart displaying emotion confidence levels
• The system outputs a live visual breakdown of predicted emotions based on speech transcription + emotion detected, offering suggestions on how to respond with empathy

Empa can be adapted or repurposed for:

• Assistive technology for autistic or neurodivergent users
• Educational platforms teaching emotional literacy
• Cross-cultural communication tools
• Mental health research interfaces
• Empathy-aware systems in customer service, therapy, etc.

Watch the demo video here!

• Frontend: React + Tailwind CSS
• Backend: Flask (Python)
• Model Training: TensorFlow + Scikit-learn -- Jupyter Notebook
• Facial landmarks extracted with MediaPipe from the FER2013 dataset

1. landmarking.ipynb: Processes FER2013 dataset with MediaPipe to extract ~463 facial landmarks per image
2. landmarking_model: Trains a facial emotion classifier using TensorFlow and Scikit-learn
3. Backend receives webcam input, processes landmarks, and outputs emotion predictions in real-time
4. Frontend displays radar chart and suggested phrases based on both speech transcription and detected emotion

🔗 GitHub: mlekhi/empa

MIT License