Gauging age outside the lab: tricky business

31 May 2016

A webcam on the door of a club that can automatically tell whether a young adult is old enough to enter and buy a beer. A webcam that sees what a woman is looking at in a shop window. If it is up to UvA postdoc Fares Alnajar, it will soon be possible.

Earlier research in the field of human-computer interaction focused more on the active individual; that is, what are people clicking on? Currently, more and more attention is being paid to the passive signals they send. What is someone looking at? What is he or she feeling? How old are they?

Alnajar, who was born in Syria, is happy to have landed in Amsterdam for his doctorate. He likes the Dutch efficiency, directness and open-mindedness. ’A smart culture’, he calls it. ‘People here are raised to be independent. They think for themselves, which makes them cleverer.’

Alnajar has spent the past four years researching ways to automatically estimate age and gaze based on faces. A difference with previous research is that it focuses on the actual conditions encountered outside the lab. Alnajar: ‘In a lab, you create the ideal conditions for research. If you want a computer to determine a person’s age, you make sure their face is positioned perfectly forward, with a neutral expression, and that there are no shadows on the face. The computer subsequently analyses the face by first determining where the eyes, nose and mouth are located. To estimate someone’s age, the primary focus is wrinkles caused by ageing and skin texture.’

Quality

This research has already made considerable strides in the lab. For companies, however, the technology will only be truly interesting once it works in real life. But such ideal conditions are never found in reality, of course. ‘People stand in different postures on the street or in a shopping centre’, explains Alnajar. ‘They’ll have a certain facial expression. The light is different, and they wear accessories such as glasses. Plus, when analysing photos the quality is not always consistently good.’ Alnajar has tackled all of these obstacles in various ways.

For starters, take the poor quality of photos. In the lab, analysing skin texture is the most precise way to estimate a person’s age. As people grow older, the amount of collagen in their skin decreases, which causes the skin to become thinner, darker and more leathery. By analysing the skin’s leather-like and rough qualities, the computer can estimate someone’s age. However, when the light and pixels in a photo are poor, which is the case with many photos online, this method loses a great deal of its precision.

To solve the problem, Alnajar designed a program that can automatically predict the quality of a photo. When a person looks at a photo, he or she immediately sees how sharp it is, whether it is properly lit and how to assess it. A computer normally does not undertake this step. Alnajar’s program performs this in-between stage on the computer’s behalf. It assesses the quality of a photo and indicates which details of the face can be best used in combination with the quality of the photo, and also whether it would be better for the computer to analyse the skin texture or age-related wrinkles.

Dynamic parts face

The dynamic parts of the face (left) and the underlying muscle structure (right). Image: F. Alnajar

False wrinkles

Yet another problem arises with wrinkle analysis. For a computer, it is difficult to distinguish between wrinkles caused by ageing and those caused by facial expressions. Also, in real life people seldom have a neutral expression. When a person smiles, age-related wrinkles change shape and laugh lines appear. Alnajar taught a graphic model how to read laugh lines in combination wrinkles resulting from age. This enables the program to record the differences that belong to age and facial expression, and then differentiate between the two more accurately.

Another approach to the wrinkle problem was to use the dynamic parts of the face in a facial expression. Alnajar unleashed a machine learning algorithm on the eyelids, corners of the mouth and cheeks of smiling people in video footage. As a result, he was able to slightly refine his age estimation.

Patent

Alnajar’s greatest source of pride is his research on gaze, which he performed in cooperation with Sightcorp, a company that specialises in automatic facial analysis. It led to a number of international publications and a patent. In the past, researchers used calibration to estimate someone’s gaze. Calibration refers to recording the exact location of the pupils, the point of gaze and the camera. Based on this information it could be determined what a person was looking at. Obviously, this cannot be done outside the lab. Alnajar and his fellow researchers were the first to propose a new approach. When people look at something, they have a similar gaze patterns. Alnajar used these gaze patterns as calibration to estimate people’s points of gaze. Using a webcam, viewers had to look at stimulus for only three seconds in order to be able to estimate where they were looking. Perfect for real-life applications, in other words.

Pipeline

Image: F. Alnajar

Predicting interest

Using information about what people look at combined with age, gender and facial expression, Sightcorp tries to perform analyses. ‘By now, we have advanced to the point where direct interactions are taking place’, says Alnajar. ‘For example, if someone is looking in a shop window, a computer can make a suggestion that suits the individual in question. So an older woman would not be recommended a PlayStation. Of course, as a female you might be interested in it, but statistically speaking it is less likely than another product.’ The more factors that are factored in – gaze, emotion – the more accurately a person’s interest can be predicted.

His research is interesting for more than just marketing purposes, however. It can be used in every application that involves human-computer interaction. Surveillance, for example. A webcam for a nightclub can automatically screen people for age or aggression. Or security. You can analyse motorists’ behaviour by installing a camera in front of them to monitor whether they are paying attention or getting tired. Gaze and posture are being used increasingly in the gaming world, too, as a way to foster a more natural and fun interaction.

These concrete applications appeal to Alnajar. Although he really enjoys conducting research, ultimately he would like to work for a company where the results and products are somewhat more tangible compared to academic research.

Social mentor

Alnajar was born in a refugee camp in the Syrian city of Daraa after his family fled Palestine. He went to school there and later attended university. He was able to obtain his Master’s in Amsterdam on a scholarship from the UvA, followed by a PhD. Given the recent situation in Syria, it was difficult at times to be so far away from his family.

His supervisor Theo Gevers was extremely important to him: ‘I was allowed to stay at his house when I was homeless for two months. I know his family well. He taught me to keep a positive attitude, to think in terms of solving problems and to set priorities for my work and my life. I will remember him more as a social mentor than as a supervisor.’

Fares Alnajar

Photo: Fares Alnajar

Fares Alnajar

Fares Alnajar obtained his PhD from the UvA for his doctoral thesis Automatic Age and Gaze Estimation under Uncontrolled Conditions.

He is currently working as a postdoc at the UvA’s Intelligent Systems Lab Amsterdam. His PhD research was funded by Commit as part of the VIEWW project.

COMMIT/ tekst: Reineke Maschhaupt

Published by  Faculty of Science