Skip to content

Issue #64: simple_backward fix - split in/out offsets #519

Closed

Description

@gHashTag

Task

Fix simple_backward to use separate in/out offsets (not shared) for correct gradient flow.

Acceptance Criteria

  • Split offsets_in and offsets_out in simple_backward
  • Correct gradient accumulation: W.grad += grad_out - grad_in
  • No gradient sharing between input and output

Test

  • overfit-100: 500 steps, BPB < 0.5
  • Run in experiments/backward/overfit_100/

Refs

  • EXP-001: Trinity 3³ physics
  • EXP-010: Ternary block alignment

Implementation

Current: shared offsets cause gradient mixing
Fix: separate offsets_in (from tokens) and offsets_out (to tokens)

Commit

"fix(backward): split in/out offsets in simple_backward"
Refs: EXP-001, EXP-010

Activity

  1. gHashTag commented on Apr 20, 2026

    @gHashTag
    OwnerAuthor
  2. added a commit that references this issue on Apr 30, 2026
    e7fe63a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions