Loading…
gathering yarn balls…
Loading Cat LLM training data...
← Back to Bench

CAT LLM TRAINING REPORT

GRU-L3 Pretraining — Multi-Run Character-Level Language Model Telemetry

Last updated: — Data points: 0 Epochs: 0 Loss: Softmax CE (label smoothing)

Run Status

Per-run health at a glance. Runs whose telemetry ends mid-epoch are marked IN PROGRESS; resume seams count process restarts detected via elapsed-time resets.

Summary

X-AXIS
SMOOTHING 0.60

Loss Curves

Running Avg is cumulative from step 1 — it lags Batch Loss by including early high-loss steps. Window Loss tracks the recent moving average. Dashed vertical lines mark resume seams (run-colored) and epoch boundaries (grey, Epoch Bands mode).

Training Loss Over Time

Smoothed Loss (EMA overlay)

Eval Loss

Best Eval Progression

Train–Eval Loss Gap

Bits per Byte (BPB)

Loss Derivatives

Central differences of Window Loss per contiguous epoch segment, scaled per 100 steps. Velocity near zero signals a plateau; acceleration spikes flag turbulence or resume seams.

Loss Velocity ΔL/Δstep

Loss Acceleration Δ²L/Δstep²

Accuracy

Eval metrics come from held-out eval rows only. ★ star markers flag evals that set a new all-time best loss.

Top-1 / Top-5 Accuracy (%)

Per-Category Eval Accuracy (%)

Gradient & Learning Rate

Gradient Norm (Pre/Post Clip + RMS)

Gradient Norm — Log Scale (spike view)

Learning Rate Schedule (WSD)

Teacher Forcing Schedule

Clip Scale Over Training

Training Throughput

Sequences per Second

Tokens per Second

Total Tokens Trained

Batch Time (ms)

Throughput vs. Batch Time

Forward / Backward / Clip / Optimizer Timing (ms)

Memory & Weight Norms

Weight norms interpolate between real checkpoint measurements where GPU telemetry gaps exist.

Memory RSS Over Training

GRU Layer Weight Norms

Aux Weight Norms (RMSNorm / Classifier / Vocab)

Best Checkpoints

Top checkpoints across all runs by eval Top-1, then lowest eval loss. ★ marks evals that set a new record at the time.

Eval History

Last 8 held-out evaluations per run. Δ is the change vs the same run's previous eval.

Epoch Summary

Run Comparison