A deep learning segmentation approach to calories and weight estimation of food images

Abstract

Today’s generation is very aware of what they are eating and the amountof calories in their food. Eating too many calories can lead to increasedweight, which has become a big health issue. A study from 2016 states thatmore than 1,9 billion adults are overweight where almost one third of theseare obese. Statistics from Norway show that 1 of 4 men and 1 of 5 womenare obese.Artificial Intelligence in general and deep learning in particular can be usedto help understand the content of eaten food. In this master thesis, wepropose a network to estimate the weight of food from a single image. Thisis done in three main parts: (1) image classification to classify what kindof food it is, (2) segmentation to segment out the different food from theimage and (3) estimate weight of the food. This is then compared againsta food database to get the calories. Both for the classification and theweight estimation is an inception network used, while a YOLO network isused for the segmentation. The solution is the first example of a workingestimation of grams from a single image. The results for weight estimationgive a standard error of 8.95 for all categories and 2.40 for bread which isthe best category.Keywords :Calorie estimation, Deep learning, Food classification,Image classification, Inception networks, Segmentation