Gemma Lara Savill
From Notebook to Mobile: Testing My Fine-Tuned News Classifier in the Real World
By Gemma Lara Savill
Published at August 31, 2025
I recently completed the first project for my Udacity Generative AI Nanodegree: fine-tuning a Foundation Model. With Parameter-Efficient Fine-Tuning (PEFT), I boosted the accuracy of DistilBERT on the AG News dataset from about 20% to 90%.
That was exciting, but then came the real question: what does 90% accuracy actually mean outside a Python notebook? Numbers are great, but I wanted to hold my model in my hand, test it against real people, and see what it could really do. That's when the idea of a "Headline Duel" was born: what if I put the model head-to-head with a human? Why not build an Android app and challenge myself and my friends to a "Headline Duel"?
Deploying a Model on a Mobile App
The first decision was how to get my model into the app. I considered two options:
- Local deployment: Converting the model into a format like a TFLite file and embedding it directly in the Android app.
- Cloud deployment: Uploading the model to a cloud service and calling it via a REST API.
I chose the cloud approach. It's a much more versatile, real-world solution. This way, the model can be updated and improved without forcing users to download a new version of the app. It's also a great way to handle larger, more complex models that would be too big for a mobile device.
To make this happen, I first had to merge the fine-tuned LoRA weights with the original DistilBERT model. I added this code to my Python notebook:
# Merge LoRA into base
merged_model = model.merge_and_unload()
# Save merged model
out_dir = "distilbert-ag-news-lora-merged"
merged_model.save_pretrained(out_dir)
AutoTokenizer.from_pretrained(base_model).save_pretrained(out_dir)
print("✅ Merged model saved to", out_dir)
Then, I uploaded the merged model to Hugging Face and created a Space to deploy it as a REST API, also using the Python notebook:
# Upload model to Hugging Face
from huggingface_hub import HfApi, create_repo, upload_folder
repo_id = "gemmalarasav/distilbert-ag-news-lora-merged"
# Create repo (skip if already exists)
api = HfApi()
try:
create_repo(repo_id, private=False) # private=True if you prefer
except Exception as e:
print("Repo may already exist:", e)
# Upload the merged model folder
upload_folder(
repo_id=repo_id,
folder_path="distilbert-ag-news-lora-merged",
commit_message="Add merged DistilBERT + LoRA AG News classifier"
)
print("✅ Model pushed to:", f"https://huggingface.co/{repo_id}")
This makes it super easy for my Android app to send a headline and receive a prediction.
The Duel Begins
To make the app work, I needed a source of headlines. I kept it simple and extracted some examples from the AG News test dataset, embedding them directly into the app. Thanks to Android Clean Architecture, it would be a simple matter to swap this out for a remote data source later.
With the headlines ready, the "duel" was simple and fun:
- The app displays a random news headline.
- The user is shown four categories: World, Sports, Business, or Sci/Tech.
- The user makes their choice.
- Once the user has committed, the app calls the model via the REST API to get its prediction.
- Finally, the app reveals who got it right: the user or the model!
A New Perspective
Going head-to-head with my model gave me a newfound appreciation for what I had created. It’s surprisingly tricky to classify some of those headlines! Seeing the model consistently make correct predictions was a great validation of the fine-tuning process.
This experience showed me the true power of fine-tuning and deployment. My trained model was no longer just a percentage on a screen; it was a tangible, working tool. This project made me think of so many other real-world use cases for this type of model, such as:
- Content tagging: Automatically classifying blog posts or articles to improve search and organization.
- Customer support routing: Analyzing a user's query and directing it to the right department.
- Educational apps: Creating a "Study Buddy" that can classify questions and provide relevant answers.
- Data filtering: Automatically sorting emails or messages into categories like "promotional" or "important."
This project turned a percentage on a screen into something tangible and interactive. It gave me a chance to see how my model performs against real people and made me appreciate the accuracy it reaches.
More importantly, it was a full-circle moment: taking what I've learned about AI fine-tuning and blending it with my Android experience to create a tool that lives beyond the notebook.
If you want to check out the project, you can find the code on my GitHub.
- Headline Duel App: https://github.com/GemmaLaraSavill/headline-duel
- Hugging Face Model: https://huggingface.co/gemmalarasav/distilbert-ag-news-lora-merged
- Hugging Face Space API: https://huggingface.co/spaces/gemmalarasav/news-classifier-space
