Github Actions CI: 2 small UX issues

I recently got access to GitHub Actions, and noticed two small UX issues.

  1. The default template for a Python CI action either has a bug in it, or there’s a bug in the way that step names are rendered.

Consider this example workflow run: https://github.com/freakboy3742/pytest-tldr/runs/196270114 The third step should read “Set up Python 3.6”, but instead the raw format string is exposed. This is an step definition reproduced verbatim from the Python test template provided by Github when setting up Actions on this repository. I’m not sure if the problem is with the rendering or the template, but I couldn’t spot anything obviously wrong with the template.

  1. Secondly, consider the following sequence of actions:

  2. User has an open pull request in a repo with Actions CI configured, and has the PR page open in their browser

  3. From their own computer, user pushes an update to the PR

  4. Github adds the commit to the list displayed on the PR

  5. Github starts the CI run, and marks the merge status as pending test results.

  6. Github completes the CI run, and marks the merge state as good/bad depending on success.

The problem: for a brief period, during state 3, the PR is marked as “Ok to merge”, regardless of the previous status of the PR. It’s only a couple of seconds; but the window of opportunity is definitely long enough for me (as an end user) to think “oh, the tests passed” when they haven’t run at all. This is a especially problematic if the push was designed to fix a previous test failure, but doesn’t fix the problem; the sequence of merge statuses on the PR is displayed as being “FAIL -> OK -> PENDING-> FAIL”, rather than “FAIL -> PENDING -> FAIL”.

This might be exacerbated by distance: I’m based in Perth, Australia, so the simple rules of physics mean that “instantaneous” actions that occur in a data centre in US take 0.3-0.5 seconds to reach my UI. It’s also not completely clear if there might be a queue processing delay on some occasions - I’ve had a couple of occasions where it’s been more than a couple of seconds.

I’m also not sure if this is entirely a Github Actions issue - other CI systems might have similar problems - but I’d expect a system tightly integrated with Github’s UI to avoid any such problems.

UX suggestion: The default merge status of a new commit on a PR should be “PENDING” if there are any previous commits on the PR that have CI markers (regardless of whether that previous status is good/bad/in progress). Once the CI run starts. At the very least, if the previous known status is bad, it shouldn’t be displayed as *good*.

1 Like

Hey @freakboy3742,

Thanks for being here! We appreciate you taking the time to write this feedback, 

I have passed your concerns along to the engineering team. Though I can’t guarantee anything or share a timeline for this, I can tell you that it’s been shared with the appropriate teams for consideration.

A follow up on this idea/issue:

I’ve been experimenting with more complex workflows, and found a related problem.

Consider a workflow definition with multiple jobs A through E, where E depends on D, D depends on C, and so on. When the workflow is triggered, the status indicator shows job A as the only outstanding task. When A completes, the PR is shown as mergeable for 1-5 seconds… and then the status indicator is moved back to pending job B, which eventually completes, and the PR is again shown as mergeable for 1-5 seconds… and then job C is queued and the status shows as pending… and so on.

The entire workflow is known at the time it is loaded, so the full list of jobs is known at the time the workflow is triggered. Setting a “pending” status for *all* the outstanding jobs (not just the ones dependent on the currently executing jobs) would avoid the “interim” mergeable status, and give a better indicator of overall workflow progress.