You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Categorical distribution and Choose(items, weights) return zero-weight outcomes: weights [1e308, 1e308, 0] always sample index 2 #78
CategoricalDistributionProvider (Essentials.DistributionProviders.Categorical/CategoricalDistributionProvider.cs, around lines 50–82) and the weighted IRandomProvider.Choose (Essentials/RandomHelpers.csSumWeights, around lines 111–128, and Essentials/IRandomProvider.cs, around lines 276–290) have two flaws:
The weight total can overflow. Each weight is checked to be finite, but the sum is not. Weights [1e308, 1e308, 0] give total = +∞, so every weights[i] / total is 0.
cumulative[^1] = 1.0 gives the last outcome all the remaining mass, even when its weight is 0. After (1), the CDF is 0 everywhere except at the last index, so every sample lands there. Without any overflow, ordinary rounding can also leave cumulative[n-2] just below 1, which gives a zero-weight last outcome a small but real chance of being drawn.
Reproduction (tested)
new CategoricalDistributionProvider([1e308, 1e308, 0]):
Pmf(0) = Pmf(1) = Pmf(2) = 0 and Mean = 0, so the masses sum to 0.
10,000 samples are all index 2, the outcome with weight zero.
rng.Choose([0, 1], [1e308, 1e308]) returns 1 on 10,000 of 10,000 calls (expected about 50/50).
Quantile(1 - 2^-53) then returns 7, while Pmf(7) = 0.
Why it matters
The output is silently wrong rather than an exception. An outcome the caller marked impossible gets picked. The overflow case also breaks the "masses sum to one" property that the test suite asserts.
Suggested fix / acceptance criteria
Make overflow impossible: divide every weight by the largest weight before summing, or throw when the total is not finite. Apply this to both the distribution and SumWeights.
Instead of pinning only the last entry, set the cumulative value to 1.0 from the last non-zero weight onward, so trailing zero weights get no mass.
Tests:
Neither repro ever returns a zero-weight index.
For [1e308, 1e308, 0], the masses sum to 1, and the first two outcomes are each about 0.5.
Different from #68, which is about tail accuracy of the Categorical SurvivalFunction.
Next step: scale by the largest weight before summing (or throw when the total isn't finite), and pin the CDF to 1.0 from the last non-zero weight. Fix both call sites together.
Activity
matt-edmondson commentedon Oct 5, 2026
Triage
Essentials.DistributionProviders.CategoricalandRandomHelpers.SumWeights/IRandomProvider.ChooseNext step: scale by the largest weight before summing (or throw when the total isn't finite), and pin the CDF to 1.0 from the last non-zero weight. Fix both call sites together.
Generated by Claude Code