Skip to content

Fix #14: k_fold covers every row in exactly one test fold - #15

Open
tanujb03 wants to merge 1 commit into
techcsispit:mainfrom
tanujb03:fix/14-kfold-cover-all-rows
Open

tanujb03 wants to merge 1 commit into
techcsispit:mainfrom
tanujb03:fix/14-kfold-cover-all-rows

Conversation

@tanujb03

Copy link
Copy Markdown
Contributor

Closes #14

What I changed

  • mess_mood/evaluate.py: k_fold no longer drops the last len(rows) % k rows. The remainder is spread over the first folds, so fold sizes are len // k or one more. The seeded shuffle and the train/test partition (train is every row outside the test slice) are unchanged.
  • tests/test_mess_mood.py: two coverage tests. One uses n % 5 != 0 (21, 23, 24, 7) and checks every row is in exactly one test fold and that train and test partition the data. The other does the same for the 113 shipped reviews and checks fold sizes differ by at most one. Both fail on main.
  • Side effect: the "5-fold cross-validation accuracy" printed by python -m mess_mood evaluate was computed over 110 of the 113 reviews. It now uses all 113, so the number moves slightly: 83% before, 82% after. Split accuracy (91%) is unchanged.

How it works

k_fold used fold_size = len(rows) // k and sliced rows[i*fold_size:(i+1)*fold_size], so the last len(rows) % k rows (3 of 113) were never in any test slice. Now I take divmod(len(rows), k) and make the first extra folds one row bigger, walking a running start/end index through the shuffled list. The k slices therefore tile the whole list exactly once, which gives the "every review is in exactly one test fold" promise from the README. When the length divides evenly nothing changes.

Testing

pytest: 15 passed (13 existing + 2 new). The new tests fail on main (2 failed) and pass on this branch.

Spread the len % k remainder over the first folds instead of dropping it, and add coverage tests for n % k != 0 and the shipped dataset.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

k_fold: 3 of the 113 reviews are never in any 5-fold test fold

1 participant