Training

A training job fine-tunes a base model on your dataset and produces a new model you can deploy or continue training from.

Start a job

job = model.train(
    dataset,
    hyperparameters={
        "METHOD": "lora",
        "NUM_EPOCHS": 3,
        "LEARNING_RATE": 2e-5,
        "LORA_R": 16,
    },
)
job.monitor()

Fleet auto-selects the most cost-effective GPU that fits your model. To target a specific tier, pass hardware:

job = model.train(dataset, hardware="fleet:pro", hyperparameters={...})

The dataset argument accepts a Dataset object, a dataset ID string, or a path to a local .jsonl file — Fleet uploads it automatically if needed. All hyperparameters are optional; defaults are listed below.

Training methods

Set the method with "METHOD": "lora" in hyperparameters. Defaults to full.

DPO requires a dataset with prompt, chosen, and rejected fields per row:

{"prompt": "What is the capital of France?", "chosen": "Paris.", "rejected": "London."}

Hyperparameters

Hardware tiers

Omit hardware entirely and Fleet picks the most cost-effective GPU that fits your model automatically. Specifying a tier constrains selection to that tier — useful if you need a specific memory size or want predictable placement.

Training is billed by the hour at the actual GPU rate. Pricing varies within a tier depending on which GPU is assigned. Check live rates:

client.hardware()

Monitoring progress

job.monitor() blocks until the job finishes and streams a live progress bar with loss and learning rate:

job.monitor()

To also print every log line as it arrives:

job.monitor(verbose=True)

To fetch logs manually:

for line in job.logs():
    print(line)

Training metrics

Fleet records loss and learning rate during training. Metrics are posted every 10 steps (configurable via the METRICS_STEPS hyperparameter) and stored server-side so you can query them at any time, even after the job finishes.

data = job.metrics()
print(data["latest"])
# {'step': 100, 'loss': 2.34, 'lr': 1.38e-05, 'val_loss': None}

for point in data["metrics"]:
    print(point["step"], point["loss"])

The metrics array is the full time-series ordered by step; feed it directly into a chart. val_loss is populated when validation runs (same step number as the corresponding training point).

Control how often metrics are posted:

job = model.train(dataset, hyperparameters={"METRICS_STEPS": 5})

From the CLI:

fleet jobs metrics <job_id>

Recovering a job

If your script exits while a job is running, reconnect by job ID:

job = client.jobs.get("job_abc123")
job.monitor()  # reattaches and continues watching

Retrieving the output model

Once complete, register the fine-tuned model and get a Model object back:

fine_tuned = job.model()
print(fine_tuned.id)

Checkpoints

If SAVE_STEPS is set, Fleet saves intermediate checkpoints during training. List them:

fleet checkpoints <job_id>

Each checkpoint is a model you can retrieve, deploy, or continue training from.

Fleet enforces a rolling checkpoint limit to control storage costs. The default is 1. When a new checkpoint is saved, the oldest beyond the limit is deleted automatically from both storage and your model list. Adjust the limit:

fleet settings set max_checkpoints 5

Or via the SDK:

client.update_settings(max_checkpoints=5)

By default, all checkpoints are deleted automatically when a job completes successfully. The final output model is the canonical artifact. To retain checkpoints after completion:

fleet settings set prune_checkpoints_on_completion false

client.update_settings(prune_checkpoints_on_completion=False)

Evaluating a model

Run a forward-pass evaluation on a held-out dataset to measure how well your model performs after training. No weights are updated — Fleet loads the model, runs every row through a forward pass, and returns loss and perplexity.

job = model.evaluate("./held-out.jsonl")
job.monitor()

results = job.eval_results()
print(f"Loss:        {results['loss']:.4f}")
print(f"Perplexity:  {results['perplexity']:.4f}")
print(f"Samples:     {results['num_samples']}")

The dataset must be a .jsonl file in the same format as your training data. Use a held-out split that was not included in training for a meaningful result.

Evaluation jobs use the same GPU selection and billing as training — you are charged only for the time the forward pass takes, typically 10–20% of the equivalent training run.

Cancelling a job

job.cancel()

fleet jobs cancel <job_id>

Cancellation terminates the GPU immediately. You are charged only for time used up to that point.