[CML][TGLH][ICL] suspend/resume TIMEOUT, no test starting log

[aiChaoSONG]

This issue is transferred from https://gitlab.devtools.intel.com/sof-ci/sofcitest/issues/670 and https://gitlab.devtools.intel.com/sof-ci/sofcitest/issues/667

TIMEOUT errors without any test log.
Found in jf-cml-hel-rt5682-01 and jf-cml-hel-rt5682-02. Looks this started in JF's Helios after replacing USB hub.
cmd: TPLG=sof-cml-rt1011-rt5682.tplg ~/sof-test/test-case/check-suspend-resume.sh -l 5

framework load: ssh ubuntu@jf-cml-hel-rt5682-01.jf.intel.com 'nohup bash -c "TPLG=sof-cml-rt1011-rt5682.tplg ~/sof-test/test-case/check-suspend-resume.sh -l 5 2>&1 | nc -N 10.239.156.45 56947" >/dev/null &'

framework run: 'netcat -l -k 56947'

The latest update is that we found this issue only happens on JF DUTs. We suspect this issue is network or usb hub related.

[marc-hb]

Still happening even after the sleep 1 of #651

• https://sof-ci.sh.intel.com/#/result/planresultdetail/3401?model=TGLH_RVP_HDA&testcase=check-suspend-resume-50 and the next test experienced a timeout
• So did https://sof-ci.01.org/sofpr/PR4084/build8802/devicetest/?model=TGLH_RVP_HDA&testcase=check-suspend-resume-with-playback

[marc-hb]

We suspect this issue is network or usb hub related.

Thanks to @keqiaozhang's large testing efforts, we also know old enough kernels don't reproduce this (while keeping everything the same)

[mengdonglin]

Duplicate of https://github.com/intel-innersource/drivers.audio.ci.sof-framework/issues/9. Fixed and closed.

[marc-hb]

This is less frequent but still happening regularly, see for instance all the links above automatically created by github.

[marc-hb]

I just spotted this, mentioning just in case:

https://sof-ci.01.org/sofpr/PR4105/build8867/devicetest/?model=APL_UP2_NOCODEC&testcase=check-suspend-resume-with-capture

rtcwake: assuming RTC uses UTC ...
rtcwake: wakeup from "mem" using /dev/rtc0 at Wed Apr 28 22:14:48 2021
2021-04-28 22:14:50 UTC Sub-Test: [REMOTE_COMMAND] sleep for 5
2021-04-28 22:14:55 UTC Sub-Test: [REMOTE_INFO] Check for the kernel log status
declare -- cmd="journalctl_cmd --since=@1619648077"
2021-04-28 22:14:55 UTC [ERROR] Caught kernel log error
===========================>>
[ 1218.444372] kernel: rtc_cmos 00:02: Alarms can be up to one month in the future

EDIT: thesofproject/linux#3011

[aiChaoSONG]

This issue is fixed already. close.

[marc-hb]

This is the start of the only known "fix" reference above: #651

After some kernel update, we start to observe a timeout issue on several JF devices. As we are not going to fix the kernel, this workaround is applied.

If there is some other, actual fix then please link to it and let's remove the workaround.

Inserting a delay without any understanding of why it makes any difference is not a "fix".