DeadlineExceeded failures in test_eye/test_linspace #214

Activity

1. oleksandr-pavlyk

   changed the title [-]DeadlineExceeded failures in test_eye/linspace[/-] [+]DeadlineExceeded failures in test_eye/test_linspace[/+] on Nov 19, 2023
2. honno

   added

   performance

   bugSomething isn't workingSomething isn't working

   on Nov 20, 2023
3. oleksandr-pavlyk commented on Dec 12, 2023

   

   oleksandr-pavlyk

   on Dec 12, 2023

   ContributorAuthor

   More actions

   This is the change I applied locally:

   diff --git a/array_api_tests/test_creation_functions.py b/array_api_tests/test_creation_functions.py
   index 94b6b0e..af86542 100644
   --- a/array_api_tests/test_creation_functions.py
   +++ b/array_api_tests/test_creation_functions.py
   @@ -354,14 +354,25 @@ def test_eye(n_rows, n_cols, kw):
            ph.assert_kw_dtype("eye", kw_dtype=kw["dtype"], out_dtype=out.dtype)
        _n_cols = n_rows if n_cols is None else n_cols
        ph.assert_shape("eye", out_shape=out.shape, expected=(n_rows, _n_cols), kw=dict(n_rows=n_rows, n_cols=n_cols))
   -    f_func = f"[eye({n_rows=}, {n_cols=})]"
   +    expected_seq = []
        for i in range(n_rows):
   +        row = []
            for j in range(_n_cols):
   -            f_indexed_out = f"out[{i}, {j}]={out[i, j]}"
   -            if j - i == kw.get("k", 0):
   -                assert out[i, j] == 1, f"{f_indexed_out}, should be 1 {f_func}"
   -            else:
   -                assert out[i, j] == 0, f"{f_indexed_out}, should be 0 {f_func}"
   +            value = 1 if j - i == kw.get("k", 0) else 0
   +            row.append(value)
   +        expected_seq.append(row)
   +    expected_array = xp.asarray(expected_seq, dtype=out.dtype)
   +    test = expected_array != out
   +    if xp.any(test):
   +        i_nz, j_nz, = xp.nonzero(test)
   +        f_func = f"[eye({n_rows=}, {n_cols=})]"
   +        for i in i_nz:
   +            for j in j_nz:
   +                f_indexed_out = f"out[{i}, {j}]={out[i, j]}"
   +                if j - i == kw.get("k", 0):
   +                    assert out[i, j] == 1, f"{f_indexed_out}, should be 1 {f_func}"
   +                else:
   +                    assert out[i, j] == 0, f"{f_indexed_out}, should be 0 {f_func}"

   and it has helped with the timeout. I think a similar change needs to be applied to pytest_helpers.assert_array_elements.

   If this looks OK, I will open a PR

   👀React with 👀1Reacted by Matthew Barber
4. honno commented on Jan 16, 2024

   

   honno

   on Jan 16, 2024

   Member

   More actions

   Just to say, what we (i.e. me) intend to do after resolving #221 is to review the test suites method of array comparison, because yes it's awfully slow right now.

   Historical reasons (signed zero/NaN comparisons, masking being optional, and a mistrust of adopter's implementations of comparison functions) meant we've just iterated over arrays and casted elements one-by-one. But we probably can use vectorization everywhere at this point, with some clever use of xp.where, and some more trust/sanity-checking of adopter's own comparison functions.

   Related discussions:

   • Tests are too slow for libraries with eager-mode dispatch overhead #197
   • Make test_eye more efficient #200 (comment)
5. honno commented on Apr 16, 2024

   

   honno

   on Apr 16, 2024

   Member

   More actions

   @oleksandr-pavlyk I wonder if this is fixed already? #236 and #246 might of solved this already.

   I had trouble getting dpctl to work locally, although will have a serious go at it down the line as obviously it'll be useful. IntelPython/dpctl#1619 did help.