Turn your datasets into production-ready models — right from the platform.
We’re thrilled to announce the biggest update yet to Distillery. Starting with this release, Distillery is no longer just a model-serving engine — it’s a complete training-to-deployment platform. You can now take your datasets, fine-tune open-weight LLMs on GPU, export them as GGUF, and deploy them back through Distillery for serving. End to end. All in one place.
Here’s everything new in this release.
1️⃣ The Full Fine-Tuning Pipeline
We’ve built a complete fine-tuning workflow that connects every stage of the ML lifecycle:
- Prepare your dataset — upload in a familiar format (see below).
- Configure training — set epochs, learning rate, and model target.
- Run GPU-accelerated training — the new local trainer handles the heavy lifting.
- Export — automatically export your trained model to production-ready GGUF format.
- Deploy — push the fine-tuned model to a deployment and start serving.

JSONL Dataset Support
Alongside existing dataset formats, we now fully support the popular JSONL format — one JSON object per line, ideal for instruction-tuning and chat datasets. Just upload your file, and the platform will parse and validate it automatically.

GPU-Accelerated Training
Training runs on your GPU, with configurable fine-tuning parameters and live progress tracking. You can monitor loss and progress in real time as the job trains.

GGUF Export, Done Right
We’ve invested heavily in making GGUF export reliable. Our new export engine handles tensor mapping, head dimensions, and vocabulary sizes correctly, so your exported model loads cleanly in llama.cpp-compatible runtimes. We’ve also added a Qwen3 RoPE repair utility for users working with Qwen3-family models.

2️⃣ A Dedicated Model Store
Your models now have a proper home. The new Model Store lets you:
- Upload and download models
- List and delete stored models
- Transfer models between local and remote storage
- Pick a target model when launching training or deployments

3️⃣ Smarter Agents & Tools
The agent layer gets a major upgrade in this release:
- Coding agent — a new agent specialized in code tasks, with a pluggable LLM-provider abstraction.
- Reflection agent — an agent that iterates over its own output to improve quality.
- Typed tool registry — an extensible, typed registry for building richer agent behaviors.
- Code parsing utilities — better source-level analysis for the agents.

4️⃣ Improved Web UI
We’ve refreshed the front end to make the new capabilities easy to reach:
- New agent configuration page
- A new models management page
- Enhanced dashboard experience
Behind the Scenes
We didn’t just add features — we hardened the platform:
- Significantly expanded and improved linter configuration and code-quality tooling
- Updated Dockerfile and Makefile for the new components
- New data schema covering agents, fine-tunes, model store, deployments, tasks, and training
- A
pull_models.pyscript to automate model downloads - +29,000 lines of test coverage across domain, use-case, delivery, and infrastructure layers
What to Know Before Upgrading
As this is a pre-1.0 release, please note:
- The data store schema has grown significantly — existing data files may need migration or re-initialization.
- Makefile targets and build commands have changed — check
make help. - HTTP API routes were reorganized — update any external integrations (new handlers for agents, models, and fine-tunes).
Getting Started
git pull origin feature/fine-tuning
make build
make run
For GPU training, see the new trainer/ directory and updated README.md for setup instructions.
[SCREENSHOT: Running application main page after successful build / launch]
What’s Next
- Multi-GPU distributed training
- Cloud storage integration (S3-compatible)
- Hyperparameter sweeps / auto-tuning
- Dataset versioning and lineage tracking
We can’t wait to see what you build with Distillery. Happy training!
Follow Distillery on GitHub – https://github.com/AIHelpers/distillery
Leave a Reply