When Segment Anything Model Meets Food Instance Segmentation

University of Chinese Academy of Sciences
Example samples in FoodInsSeg

Example samples in FoodInsSeg.

Download_Google Download_Baidu Evaluation Tool

Dataset Download Agreement

The FoodInsSeg dataset is available only for non-commercial research purposes. To ensure responsible utilization of the dataset, researchers accessing it should adhere to the following guiding principles:


  1. The FoodInsSeg dataset is available for academic research only, commercial and any profit purposes are strictly prohibited
  2. We retain the right to terminate access at anytime. Users should voluntarily comply with the agreement.
  3. Please cite our work based on the BibTex information we have provided.

We sincerely hope that FoodInsSeg will contribute to the progress of computer vision and food recognition research. We kindly request that you follow the agreement and properly cite the data source in your work.

Abstract

The Segment Anything Model (SAM) has significantly advanced image segmentation through its exceptional zero-shot performance. However, it faces a significant limitation due to the lack of class-specific information. To address this problem, we propose to leverage the category information given by semantic datasets for SAM. In this paper, we develop a powerful tool called InsSAM-Tool, which can adequately utilize class-specific information and SAM's segmentation capabilities to efficiently label instance segmentation. Furthermore, we leverage this tool to establish a groundbreaking dataset called FoodInsSeg, which is the first food instance dataset at ingredient level. It contains a total of 7,118 food images, annotated with 103 categories and 119,048 instance masks. FoodInsSeg provides significant potential to catalyze further research into fine-grained food instance segmentation.

The distribution of food categories

The distribution of food categories. The detailed statistic data can be found at the excel file.