The A-LIST app makes health easy. Just say what you're doing and what you're eating, and you'll always know what to do next.
The hard part isn't knowing what to do next. That's actually easy to decide once you know how many calories you've burned today and how much food you've eaten. The challenge has always been translating an activity like "30 minutes of running" into a good estimate of how many calories you burned. Even harder is translating "I ate a chicken sandwich" into an accurate estimate of your macronutrients.
Until now, especially for the food case, we've had to use API-based LLMs to interpret these inputs. It's slow. It's inaccurate. It's expensive.
Fine-tuning a small LLM is ideal for this. It's free to run. It's fast. It's accurate. It's private. And importantly, it gives us control over the intelligence our product depends on.
We now have a small LLM that we plan to ship in the next version of the A-LIST app. This is the story of how we did it and where we're going.
Fine-Tuning a Small LLM
Here's a quick rundown of the process we used.
1. Data
We used a combination of internal and external datasets for inputs, and a frontier teacher model to validate and generate outputs.
2. Base Model Selection
We started with LFM2.5 as an arbitrary starting point. It's a small LLM that we had heard good things about.
Next, we tried using an embedding model and training our own neural net on top of it. That would have been great for speed, memory, and disk usage. This approach did actually work for extracting exercise information, but food is much more varied, and macro estimation requires more internal consistency. It was better than LFM on its own, but still not good enough to ship.
We settled on using LoRA with Qwen3.5-2B. That gave us reasonable accuracy, around a ~1GB disk footprint, and performance similar to the API-based LLM.
3. Training and Eval
We used chariots.sh to manage the training and evals. Nobody on the team is a trained ML engineer. Chariot is ideal for this because you can easily spin up multiple highly intelligent agents with whatever system configuration you need, including GPUs. The agents are sandboxed from one another so they don't cross-contaminate training and eval data.
4. Results
Through this process, we climbed to a high degree of accuracy.
| architecture | calories |
|---|---|
| LFM2.5 | 0.397 |
| encoder e2e | 0.591 |
| Qwen LoRA r32 | 0.701 |
| Qwen LoRA r64/3ep | 0.761 ✓ (bar 0.75; carbs 0.775 ✓ 0.70) |
Smaller, Faster, Stronger
We're on the lookout every day for smaller, faster, and stronger local models to use across a variety of our features. Recently, Meta announced MobileMoE. This is exactly the kind of thing we're interested in because it opens up new possibilities. Running the same process on these new models produced some interesting results.
| MobileMoE-L (922M active) | Shipped Qwen 2B | MobileMoE-S (272M active) | Ship bar | |
|---|---|---|---|---|
| calories | 0.758 ✅ | 0.761 | 0.694 | 0.75 |
| carbs | 0.756 ✅ | 0.775 | 0.666 | 0.70 |
| protein | 0.838 | 0.856 | 0.775 | — |
| water | 0.919 | 0.892 | 0.868 | — |
| glycemic index | 0.913 | 0.892 | 0.901 | — |
| micros mean | 0.852 | 0.862 | 0.830 | — |
| parse failures | 0 | 0 | 0 | — |
🎊 Wow! 🎊
After our fine-tuning process:
- MoE-L clears our ship bar with half the active parameter count!
- MoE-S is VERY close to clearing the ship bar with a smaller overall model and <300M active parameters!
These are very promising results. Although MoE-L, with what we've been able to do today, is currently larger than Qwen on disk, the options provided by MobileMoE are nonetheless exciting and indicative of what we can expect to come.
With MobileMoE, we can explore methods we've already used as well as new approaches like expert pruning to provide the best possible experience for users locally.
Thanks to the team at Meta that is pushing the frontier on mobile LLMs: Yanbei Chen, Hanxian Huang, Ernie Chang, Jacob Szwejbka, Digant Desai, Zechun Liu, Vikas Chandra, and Raghuraman Krishnamoorthi!