Hanging workflows

Dear Github-Community,

I am experiencing hanging workflows on CentOS 7 and Ubuntu 18.04/20.04 [1-3].
I tried to debug them using a tmate session, but unfortunately, also tmate freezes.

The freeze happens in a python script (biodynamo run) which calls a cmake, make, run binary workflow.
Previously, I have also seen this behavior during valgrind tests, and during startup of an xvfb server.
Based on the latter one I found the following stackoverflow post which suggests to use tini as init process [4].
The workflow output shows defunct processes [1] (line 403), which atm I am unsure where they are coming from.
Do you think these defunct processes could cause these freezes?

Many thanks in advance for your help!
Lukas

PS As a new user I am not allowed to include more than two links.

[1] https://github.com/BioDynaMo/biodynamo/pull/107/checks?check_run_id=927027315#step:7:430
[2] github DOT com/BioDynaMo/biodynamo/pull/107/checks?check_run_id=927027188#step:5:2321
[3] github DOT com/BioDynaMo/biodynamo/pull/107/checks?check_run_id=927027213#step:5:2079
[4] https://stackoverflow.com/questions/50634819/xvfb-run-hangs-in-container

Hi @breitwieserCern,

Glad to see you in Github Community Forum!

With timestamp enabled, the job is cancelled after the issue command(takes about 2hours).
Github will check if there any exit code returned to STDOUT to dertermine the next step. If the command executed internally and has not any exit code, it will hang(doc here). Hence, the defunct process could be the reason.

You check the appearance on local client to confirm whether it hangs as well.

Thanks.