Skip to content

Issue #62: Loss-curve validation on held-out data #59

Description

@gHashTag

Task

Validate loss curve on held-out data (90% train / 10% val) to ensure no overfitting before Parameter Golf submission.

Acceptance Criteria

  • Split TinyShakespeare: 90% train / 10% val
  • Run 1000 steps on train set
  • Compute BPB on held-out val set
  • Check: val_bpb < 2 * train_bpb (no severe overfitting)
  • Plot loss curves for train vs val

Deliverables

  • experiments/loss_curve/train_loss.csv
  • experiments/loss_curve/val_loss.csv
  • Plot: experiments/loss_curve/loss_curve.png
  • Overfitting diagnosis

Configuration

Refs

Go/Pivot Gate

Only proceed to Parameter Golf if val_bpb is plausible (not severely overfit).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions