Repository navigation
Linalg stacking tests should not test exact equality #44
Description
The following two failures show that the exact equality test in _test_stacks should be relaxed, as the results agree within some tolerance that's both hardware and software dependent:
__________________________________________________________________ test_eigh __________________________________________________________________
> ???
test_linalg.py:217:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_linalg.py:232: in test_eigh
_test_stacks(lambda x: linalg.eigh(x).eigenvalues, x,
test_linalg.py:61: in _test_stacks
assert_exactly_equal(res_stack, decomp_res_stack)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
x = Array([0. , 1.9999999], dtype=float32), y = Array([0., 2.], dtype=float32)
def assert_exactly_equal(x, y):
"""
Test that the arrays x and y are exactly equal.
If x and y do not have the same shape and dtype, they are not considered
equal.
"""
assert x.shape == y.shape, f"The input arrays do not have the same shapes ({x.shape} != {y.shape})"
assert x.dtype == y.dtype, f"The input arrays do not have the same dtype ({x.dtype} != {y.dtype})"
> assert all(exactly_equal(x, y)), "The input arrays have different values"
E AssertionError: The input arrays have different values
array_helpers.py:181: AssertionError
----------------------------------------------------------------- Hypothesis ------------------------------------------------------------------
Falsifying example: test_eigh(
x=Array([[[1., 1.],
[1., 1.]]], dtype=float32),
)
________________________________________________________________ test_eigvalsh ________________________________________________________________
> ???
test_linalg.py:241:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_linalg.py:248: in test_eigvalsh
_test_stacks(linalg.eigvalsh, x, res=res, dims=1)
test_linalg.py:61: in _test_stacks
assert_exactly_equal(res_stack, decomp_res_stack)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
x = Array([0. , 1.9999999], dtype=float32), y = Array([0., 2.], dtype=float32)
def assert_exactly_equal(x, y):
"""
Test that the arrays x and y are exactly equal.
If x and y do not have the same shape and dtype, they are not considered
equal.
"""
assert x.shape == y.shape, f"The input arrays do not have the same shapes ({x.shape} != {y.shape})"
assert x.dtype == y.dtype, f"The input arrays do not have the same dtype ({x.dtype} != {y.dtype})"
> assert all(exactly_equal(x, y)), "The input arrays have different values"
E AssertionError: The input arrays have different values
array_helpers.py:181: AssertionError
----------------------------------------------------------------- Hypothesis ------------------------------------------------------------------
Falsifying example: test_eigvalsh(
x=Array([[[1., 1.],
[1., 1.]]], dtype=float32),
)
Activity
asmeurer commentedon Nov 25, 2021
The idea with the stacking tests is that a function that operates on stacks of matrices should be the same on a stacked array as if you do it on the individual arrays independently. Is there any reason this shouldn't hold? The computations are independent of one another for different matrices.
To be fair the spec doesn't really specify this. Maybe it should clarify whether it should be the same or just "close".
leofang commentedon Dec 1, 2021
Sorry @asmeurer I missed your reply. What I meant is that the comparison of actual & expected outputs like this
should return
True. In other words, we can't expect an exact comparison between floating points that is currently done here:array-api-tests/array_api_tests/test_linalg.py
Line 61 in df5f6f5
leofang commentedon Dec 5, 2021
cc: @kmaehashi for vis
[-]Test condition too stringent [/-][+]Linalg stacking tests should not test exact equality[/+]Allow specifying the equality check in the linalg _test_stacks
Use assert_allclose() in the linalg tests for float inputs
Allow specifying the equality check in the linalg _test_stacks