Computer Vision on Edge Devices

Client Background

The client is a mid-sized retail company in the retail industry with over 100 employees. They have a well-established analytics system to track business performance and make data-driven decisions. They sought assistance from SmartCat to explore machine learning solutions and leverage computer vision technology to gain insights into customer demographics, emotions and behavior, with the potential to improve customer experience and optimize marketing strategies.

Problem Description

The client faced the challenge of accurately tracking and profiling individuals visiting their retail locations, while maintaining optimal performance within resource and computational constraints. They aimed to gather demographic information such as age, gender, and emotions at a sufficient distance (around 8 – 10 meters). However, they encountered several pain points and obstacles. Firstly, implementing complex computer vision models within their resource constraints posed a significant challenge. They needed models that were performant, real-time, and capable of maintaining accuracy.

Solution

To address the client’s challenges, the SmartCat team employed a series of strategies and tactics. We began by selecting and testing state-of-the-art models suitable for the retail industry. These models are coming from various sides and some of the technologies used are Ultralytics, YOLOv8, insightface, DeepFace, OpenCV, SixDRepNet, TensorRT, Tens of Flow, and Pytorch. Various edge devices, such as the NVIDIA Jetson TX2, Raspberry Pi, and Orange Pi, were utilized to experiment with different models and check their performance on resource-constrained devices, as well as potential optimization techniques. Leveraging edge devices offered on-site predictions, eliminating the need for sensitive data transmission over the network and minimizing network latency.

One crucial aspect of the solution also was estimating facial pose relative to the camera. This allowed the client to determine the direction of a customer’s gaze, presenting opportunities for targeted advertising and customer behavior profiling. The SmartCat team focused on achieving a higher accuracy and frame rate, camera quality, accurate distance measurement, and optimal lighting conditions to ensure better performance compared to the client’s existing system.

Results

The implementation of the SmartCat team’s solution yielded positive outcomes. We achieved better performance in terms of increased frame rate, precise distance measurement, and than the client’s previous system. Moreover, the team emphasized privacy and ensured compliance with GDPR regulations throughout the project, addressing any concerns related to facial data.

Key Metrics and Quantifiable Results:

Higher frame rate and better accuracy and performance compared to the client’s existing system
Improved accuracy in demographic profiling
Real-time processing and predictions on edge devices
Enhanced privacy and security compliance with GDPR regulations

Smart Tip

When dealing with computer vision use cases, it is essential to define the appropriate frames per second (FPS) based on the specific use case requirements. For security applications, a higher number of frames ensures real-time response and accurate detection. In contrast, in retail scenarios, a lower number of frames may be acceptable for faster processing, while still providing relevant information. Furthermore, adjusting the resolution can contribute to gaining more detailed insights.

Smart Fact

The obtained results from this project can be leveraged to build recommender systems based on demographic profiles or emotions. For instance, personalized recommendations such as offering chocolate to those feeling sad can be derived from the gathered insights. Furthermore, retail industries can utilize demographic information for targeted marketing campaigns based on age group and gender preferences.