Repository navigation
[BUG] dsp not suspended after 15s when runtime suspend on TGLH_RVP_NOCODEC_ZEPHYR #4507
Description
Describe the bug
TGLH_RVP_NOCODEC_ZEPHYR dsp not suspended after 15s when check-kmod-load-unload
inner daily 5356 model=TGLH_RVP_NOCODEC_ZEPHYR testcase=check-runtime-pm-status-15
To Reproduce
TPLG=/lib/firmware/intel/sof-tplg/sof-tgl-h-nocodec.tplg ~/sof-test/test-case/check-runtime-pm-status.sh -l 15
Environment
Kernel Branch: topic/sof-dev
Kernel Commit: 8877846e storage: 8877846e
SOF Branch: main
SOF Commit: f96e25f storage: f96e25f
Topology:sof-tgl-h-nocodec.tplg
Platform: TGLH_RVP_NOCODEC_ZEPHYR
Screenshots or console output
[console]
2021-07-19 22:47:06 UTC [REMOTE_INFO] ===== Iteration 1 of 15 for DMIC =====
2021-07-19 22:47:06 UTC [REMOTE_COMMAND] arecord -D hw:0,10 -r 48000 -c 4 -f S16_LE /dev/null -q
2021-07-19 22:47:08 UTC [REMOTE_INFO] runtime status: active
2021-07-19 22:47:08 UTC [REMOTE_COMMAND] kill process: kill -9 56918
2021-07-19 22:47:08 UTC [REMOTE_INFO] arecord killed
2021-07-19 22:47:08 UTC [REMOTE_INFO] wait dsp power status to become suspended
2021-07-19 22:47:25 UTC [REMOTE_ERROR] dsp is not suspended after 15s, end test
2021-07-19 22:47:25 UTC [REMOTE_ERROR] Starting func_exit_handler(), exit status=1, FUNCNAME stack:
2021-07-19 22:47:25 UTC [REMOTE_ERROR] die() @ /home/ubuntu/sof-test/test-case/../case-lib/lib.sh
2021-07-19 22:47:25 UTC [REMOTE_ERROR] func_check_dsp_status() @ /home/ubuntu/sof-test/test-case/check-runtime-pm-status.sh:46
2021-07-19 22:47:25 UTC [REMOTE_ERROR] main() @ /home/ubuntu/sof-test/test-case/check-runtime-pm-status.sh:112
2021-07-19 22:47:25 UTC [REMOTE_INFO] Test Result: FAIL!
Activity
kv2019i commentedon Jul 20, 2021
I could reproduce this on a TGL-U platform as well. This looks the same as #4400 . The log I get is this:
[ 3247.161405] sof-audio-pci-intel-tgl 0000:00:1f.3: ipc tx: 0x40010000: GLB_PM_MSG: CTX_SAVE
[ 3247.663096] sof-audio-pci-intel-tgl 0000:00:1f.3: error: ipc timed out for 0x40010000 size 76
[ 3247.663111] sof-audio-pci-intel-tgl 0000:00:1f.3: preventing DSP entering D3 state to preserve context
[ 3247.663114] sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ IPC dump start ]------------
[ 3247.663145] sof-audio-pci-intel-tgl 0000:00:1f.3: error: hda irq intsts 0x00000000 intlctl 0xc0000000 rirb 00
[ 3247.663151] sof-audio-pci-intel-tgl 0000:00:1f.3: error: dsp irq ppsts 0x00000000 adspis 0x00000000
[ 3247.663176] sof-audio-pci-intel-tgl 0000:00:1f.3: error: host status 0x00000000 dsp status 0x00000000 mask 0x00000003
[ 3247.663180] sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ IPC dump end ]------------
[ 3247.663184] sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump start ]------------
[ 3247.663202] sof-audio-pci-intel-tgl 0000:00:1f.3: unknown ROM status value ffffffff
[ 3247.663393] sof-audio-pci-intel-tgl 0000:00:1f.3: invalid header size 0xffffffff. FW oops is bogus
[ 3247.663397] sof-audio-pci-intel-tgl 0000:00:1f.3: error: unexpected fault 0xffffffff trace 0xffffffff
[ 3247.663402] sof-audio-pci-intel-tgl 0000:00:1f.3: ------------[ DSP dump end ]------------
[ 3247.663407] sof-audio-pci-intel-tgl 0000:00:1f.3: ctx_save ipc error -110, proceeding with suspend
... in most cases system will recover from this (driver will ignore the error as DSP is power-cycled anyway). Needs to be fixed still of course.
lgirdwood commentedon Jul 21, 2021
@XiaoyunWu6666 this is the same bug as the APL version, it's on @kv2019i task list. @dcpleung fyi.
zephyr: unify CTX_SAVE handling for Zephyr
[-][BUG] dsp not suspended after 15s when check-runtime-pm-status-15 on TGLH_RVP_NOCODEC_ZEPHYR [/-][+][BUG] dsp not suspended after 15s when runtime suspend on TGLH_RVP_NOCODEC_ZEPHYR [/+]kv2019i commentedon Aug 18, 2021
Hmm, the CTX_SAVE bug is now fixed, but it seems in latest daily tests the test is still failing. When looking at the log, it seems the failure reason is now something else:
http://sof-ci.sh.intel.com/#/result/planresultdetail/5931?model=TGLH_RVP_NOCODEC_ZEPHYR&testcase=check-runtime-pm-status-15
[ 603.150526] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: ipc tx: 0x30200000: GLB_TPLG_MSG: BUFFER_NEW
[ 603.151011] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: error: ipc error for 0x30200000 size 20
[ 603.151015] kernel: sof-audio-pci-intel-tgl 0000:00:1f.3: error: failed to load widget BUF20.0
I'll debug this further.
kv2019i commentedon Aug 18, 2021
I could not reproduce this locally (tried both GCC and XCC builds). This could be related to the other failures earlier in the test plan. Let's see when we have the XCC built binaries in CI, whether it helps with this as well.
lgirdwood commentedon Aug 27, 2021
@XiaoyunWu6666 any update, can we close ?
XiaoyunWu6666 commentedon Aug 30, 2021
still exist @lgirdwood , but may be affected by the previous testcase .
I just added reboot after previous testcase (which brought ipc timed out) , to see whether pure runtime suspend testcase can reproduce this or not
XiaoyunWu6666 commentedon Aug 31, 2021
from inner daily 6200 , after excluding affects from previous testcases , check-runtime-pm-status and kmod-load-unload worked well without 'dsp not suspended'
I'll close this issue if next daily works well too , thanks