ConvNeXt-Food-CLF-75

Multi‑label food ingredient classifier built on ConvNeXt‑Tiny with CBAM attention and GeM pooling.
Trained on a cleaned subset of MM‑Food‑100K for 75 ingredients.

📊 Key Metrics: Macro F1 = 0.6546 | mAP = 0.7142

🚀 Ready‑to‑use model and quick start: Hugging Face Model Card

👉 Try it live on Spaces - no installation required!

---

📖 What This Model Does

ConvNeXt-Food-CLF-75 takes an RGB food image, resizes it to 640×640, and predicts which of the 75 known ingredients are present. It performs multi‑label classification – a single dish can contain multiple ingredients (e.g., “rice”, “sauce”).

Internally, the model produces a compact feature vector (embedding) that can be reused for downstream tasks. In our pipeline, this embedding is fed into a segmentation network to guide pixel‑wise mask prediction, forming part of a calorie‑estimation system.

• Input: RGB image (any size, automatically resized to 640×640 and normalized with ImageNet statistics)
• Output:
  • A list of ingredient names (threshold 0.5 by default)
  • Raw probability scores for each of the 75 classes
  • Optional: feature embedding from the penultimate layer (for transfer learning)

Supported 75 Classes (in alphabetical order)

almond	cheese butter	grape	onion	sauce
apple	chicken duck	ice cream	orange	sausage
asparagus	chocolate	juice	pasta	seaweed
avocado	cilantro mint	kiwi	peach	shellfish
banana	coffee	lamb	peanut	shrimp
beans	corn	lemon	pear	soup
biscuit	crab	lettuce	peas	spring onion
bread	cucumber	mango	pepper	sprouts
broccoli	date	meat	pineapple	steak
cabbage	egg	melon	pizza	strawberry
cake	eggplant	milk	pork	tea
carrot	fish	mushroom	potato	tofu
cashew	french fries	noodle	pumpkin	tomato
cauliflower	garlic	okra	rice	watermelon
celery stick	ginger	olives	salad	wonton dumplings

---

🧹 Dataset Challenges

The original MM‑Food‑100K dataset is extremely noisy:

• Over 4,000 raw ingredient labels, many of which are visually indistinguishable (e.g., “enoki mushrooms” vs. “shiitake mushrooms”)
• Severe class imbalance (some classes have fewer than 10 images)
• Duplicates and inconsistent annotations

Designed a multi‑step cleaning pipeline that reduces the chaos to a clean, balanced 75‑class dataset suitable for training.

The full pipeline is documented in the dataset_preparation/ folder.
It includes:

• Manual curation of target classes
• Automated extraction of matching ingredients
• Manual merging and removal of irrelevant entries
• Filtering classes with fewer than 100 images
• Building a multi‑hot label dataset and vocabulary

If you want to understand every detail or adapt the pipeline to your own food dataset, start with the README inside that folder.

---

📊 Model Performance

Final metrics after a 4‑stage training process:

Metric	Value
F1 (macro)	0.6546
mAP	0.7142
Val Loss	0.0489

Complete training logs, hyperparameters, and stage‑by‑stage results are documented in the model/ folder.

---

💻 Hardware & Environment

All training and inference were performed on a single consumer GPU:

Component	Detail
GPU	NVIDIA RTX 2060 (6 GB VRAM)
CPU	Intel Xeon E5‑2660 v3
OS	CachyOS (Arch‑based)
Python	3.10+
PyTorch	2.5
timm	1.0.9
Mixed precision	AMP (torch.amp)

Despite the limited 6 GB VRAM, the model trains comfortably with a batch size of 10 at 640×640 resolution.

---

🚀 Quick Inference

For ready‑to‑run commands, see the Hugging Face Model Card or the model/ folder in this repository or Try it live on Spaces - no installation required!**

---

🧀 Prediction Examples

Correct predictions

---

⚠️ The model can occasionally hallucinate ingredients or fail to detect all ingredients present.

---

🙏 Acknowledgements

MM‑Food‑100K for the original dataset.

timm for the ConvNeXt backbone.

Albumentations for image augmentations.

---

Citation

If you use this model or the cleaned dataset in your research or project, please cite:

@misc{alasv2026convnextfoodclf75,
  author       = {Artem V.},
  title        = {ConvNeXt-Food-CLF-75: A Multi-Label Food Ingredient Classifier},
  year         = {2026},
  howpublished = {\url{https://github.com/Alas-V/ConvNeXt-Food-CLF-75}},
  note         = {Model weights available at \url{https://huggingface.co/Alas-V/ConvNeXt-Food-CLF-75}}
}

---

📄 License

This project is licensed under the Apache 2.0 License. See the LICENSE file for details.