Description
py/test/selenium/webdriver/common/_bidi/input_tests.py fails intermittently on Firefox, with a different test failing on each run. Observed over repeated runs on macOS arm64 (Apple Silicon), pinned browsers, headless:
test_key_input_with_value_attribute[firefox]test_key_input_special_characters[firefox]test_combined_input_actions[firefox]
Each failed in some runs and passed in others. The target failed in 3 out of 3 runs before an unrelated fix and 2 out of 3 after, with the failing test name varying between runs — so this is flakiness in the file rather than a single broken assertion.
These are distinct from test_wheel_scroll_with_duration, which failed in this same file for a different, understood reason (reading pageYOffset mid-animation) and has been fixed separately by waiting for the scroll to settle.
The rotating failure set suggests a shared timing assumption in the keyboard/action tests — most likely asserting on input state before Firefox has finished dispatching the synthesised events. Chrome was not observed failing these.
Reproducible Code
bazel test //py:test/selenium/webdriver/common/_bidi/input_tests-firefox-bidi \
--pin_browsers=true --headless=true --local_test_jobs 1 \
--cache_test_results=no --runs_per_test=5
Debugging Logs
=================== 1 failed, 25 passed, 1 xfailed in 16.61s ===================
FAILED py/test/selenium/webdriver/common/_bidi/input_tests.py::test_key_input_special_characters[firefox]
=================== 1 failed, 25 passed, 1 xfailed in 16.61s ===================
FAILED py/test/selenium/webdriver/common/_bidi/input_tests.py::test_combined_input_actions[firefox]
(baseline, before the wheel-scroll fix - failed 3/3 runs)
FAILED py/test/selenium/webdriver/common/_bidi/input_tests.py::test_wheel_scroll_with_duration[firefox]
FAILED py/test/selenium/webdriver/common/_bidi/input_tests.py::test_key_input_with_value_attribute[firefox]
Selenium version
trunk (Python bindings, 4.49.0-SNAPSHOT)
What are the browser(s) and version(s) where you see this issue?
Firefox (pinned build used by the Bazel test suites)
Description
py/test/selenium/webdriver/common/_bidi/input_tests.pyfails intermittently on Firefox, with a different test failing on each run. Observed over repeated runs on macOS arm64 (Apple Silicon), pinned browsers, headless:test_key_input_with_value_attribute[firefox]test_key_input_special_characters[firefox]test_combined_input_actions[firefox]Each failed in some runs and passed in others. The target failed in 3 out of 3 runs before an unrelated fix and 2 out of 3 after, with the failing test name varying between runs — so this is flakiness in the file rather than a single broken assertion.
These are distinct from
test_wheel_scroll_with_duration, which failed in this same file for a different, understood reason (readingpageYOffsetmid-animation) and has been fixed separately by waiting for the scroll to settle.The rotating failure set suggests a shared timing assumption in the keyboard/action tests — most likely asserting on input state before Firefox has finished dispatching the synthesised events. Chrome was not observed failing these.
Reproducible Code
bazel test //py:test/selenium/webdriver/common/_bidi/input_tests-firefox-bidi \ --pin_browsers=true --headless=true --local_test_jobs 1 \ --cache_test_results=no --runs_per_test=5Debugging Logs
Selenium version
trunk (Python bindings, 4.49.0-SNAPSHOT)
What are the browser(s) and version(s) where you see this issue?
Firefox (pinned build used by the Bazel test suites)