Module 10 · Lesson 2 of 2
What fine-tuning takes
Tune, test apart, roll out small
Trust a tuned model only after unseen tickets score it well, and even then give it a small share of traffic first.
Scene 1 of 5 · Gather marked examples
The team collects 3,500 tickets, each tagged by a person with the label it should get.
- Your examples — healthy — 3,500 tagged tickets
- Split — healthy
- Fine-tune — healthy
- Test set check — healthy
- Small rollout — healthy
- Gather marked examples. The team collects 3,500 tickets, each tagged by a person with the label it should get.
- Set 500 aside. 500 tickets go to the test check and never take part in training.
- Tune on the rest. The other 3,000 train the model further, often through a small set of added numbers.
- Score unseen tickets. The tuned model labels the 500 it never saw, and that score is the honest one.
- Roll out small. Next, one user in twenty is served by it for a week, and it scores 87 there.
- Now you try. Pick how the tuned model was tested, and compare its test score with real use. For example: Test set: 99 in 100. Real use: 87 in 100. The model had seen the answers, so the test measured memory: a form of data leakage.
Scene 1 of 5: Gather marked examples. The team collects 3,500 tickets, each tagged by a person with the label it should get.
Key points
- Split the examples before training, and never test on training data.
- An honest test score sits close to the score in real use.
- A test that overlaps training data reports a score nobody will see.
- Release a tuned model to a small share of users first.
(Training an already trained model a little more on your own examples, to change how it behaves on your task.) starts from a trained model and trains it a little more on your own examples: pairs of an input and the output you want. Clean, checked examples matter more than a large pile of noisy ones.
Keep some examples out of (The long, costly process in which a model learns from data. It happens before the model is used, not during each request.), as a test set. If the same examples are in both, the test score looks great and means nothing. This is one form of (When information from the test set reaches training, so the test score is higher than real use will be.).
Parameter-efficient methods, such as (Low-rank adaptation: a way to fine-tune that trains a small set of added numbers instead of every parameter.), train a small set of added numbers instead of every (One of the numbers inside a model that training adjusts. Large language models have billions of them.). They need less (A graphics processing unit: a chip that does many simple sums at the same time. Language models run on GPUs because they need huge numbers of sums.) memory, and the result is a small file used together with its base model.
Treat each tuned model as a release. Record which data and settings made it, and compare it with the current model on the same test set. Then send it a small share of traffic first.
In system design: A tuned model is tied to its base model. When a better base model comes out, you tune again and run the same tests.