I recently got access to GitHub Actions, and noticed two small UX issues.
- The default template for a Python CI action either has a bug in it, or there’s a bug in the way that step names are rendered.
Consider this example workflow run: https://github.com/freakboy3742/pytest-tldr/runs/196270114 The third step should read “Set up Python 3.6”, but instead the raw format string is exposed. This is an step definition reproduced verbatim from the Python test template provided by Github when setting up Actions on this repository. I’m not sure if the problem is with the rendering or the template, but I couldn’t spot anything obviously wrong with the template.
Secondly, consider the following sequence of actions:
User has an open pull request in a repo with Actions CI configured, and has the PR page open in their browser
From their own computer, user pushes an update to the PR
Github adds the commit to the list displayed on the PR
Github starts the CI run, and marks the merge status as pending test results.
Github completes the CI run, and marks the merge state as good/bad depending on success.
The problem: for a brief period, during state 3, the PR is marked as “Ok to merge”, regardless of the previous status of the PR. It’s only a couple of seconds; but the window of opportunity is definitely long enough for me (as an end user) to think “oh, the tests passed” when they haven’t run at all. This is a especially problematic if the push was designed to fix a previous test failure, but doesn’t fix the problem; the sequence of merge statuses on the PR is displayed as being “FAIL -> OK -> PENDING-> FAIL”, rather than “FAIL -> PENDING -> FAIL”.
This might be exacerbated by distance: I’m based in Perth, Australia, so the simple rules of physics mean that “instantaneous” actions that occur in a data centre in US take 0.3-0.5 seconds to reach my UI. It’s also not completely clear if there might be a queue processing delay on some occasions - I’ve had a couple of occasions where it’s been more than a couple of seconds.
I’m also not sure if this is entirely a Github Actions issue - other CI systems might have similar problems - but I’d expect a system tightly integrated with Github’s UI to avoid any such problems.
UX suggestion: The default merge status of a new commit on a PR should be “PENDING” if there are any previous commits on the PR that have CI markers (regardless of whether that previous status is good/bad/in progress). Once the CI run starts. At the very least, if the previous known status is bad, it shouldn’t be displayed as *good*.