Skip to content

[BUG] NN Descent resulting in negative and NaN distances #991

Closed

Description

@jinsolp

Describe the bug
For certain datasets, using L2Expanded metric results in negative distances. For L2SqrtExpanded metric results in NaN distances

Cause
This happens because NN Descent internally calculates distances in fp16.
To calculate L2Expanded, we have to do this operation;

norm(x) + norm(y) - 2*dot(x, y)

norm(x) and norm(y) is calcualted in fp32, while dot(x,y) is calculated in fp16.

For almost identical vectors, the distance should be 0. However due to precision issues the calculation of the equation above becomes a negative number (e.g. -0.00006).

Calculation L2SqrtExpanded sqrt this negative distance, resulting in NaN values.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions