Food Constituent Estimation for Lifestyle Disease Prevention by Multi-Task CNN

Unbalanced nutrition due to an unhealthy diet may increase the risk of developing lifestyle diseases. Many mobile applications have been released to record everyday meals for the health-conscious to enable them to improve their dietary habits. Most of these applications only base their food classification on an image of the food, requiring the user to manually input information about the ingredients such as the calories and salinity. To address this problem, food ingredient estimation from food images has been attracting increasing attention. Automatic ingredient estimation could possibly strongly alleviate the process of food-intake estimation and dietary assessment. In this paper, we propose an automatic food ingredient estimation method from food images by using multi-task CNN. We focus on classification of the food category and estimation of the calorie content and salinity for lifestyle disease prevention. Two-stage transfer learning using a large number of food category recognition image databases is applied to train our multi-task CNN for improved ingredient estimation. We experimentally analyze the relationship between the food category and salinity by using multi-task CNN.