Automatic commenting on a commit with results from a script

Hi folks!

I am trying to create a comment on a commit automatically by using results from my runner program. Let’s consider this run. I want to be able to take the highlighted URL and include that in the comment:

Screen Shot 2020-08-28 at 9.02.08 PM

Programmatically, I am using Weights and Biases to keep track of my ML models. So, once, I run wandb.init(project="wandb-github-actions", entity="sayakpaul") from my Python program I am able to access that URL via wandb.run.url within the program.

Thanks for your feedback.
From your train.py file(Line 51~55), did you want to write the run_url into the metrics.txt file, and then allow this data to be accessed by subsequent Actions?
If so, you could add reading the metrics.txt file step to check if this run_url is correct and then use git command to push this commit.
You can also add git command directly in your train.py file, such as:

print("\n======================= Push changes to remote =======================")
import subprocess
print("Set git config...")
subprocess.Popen(["git", "config", "--global", "user.name", "github-actions[bot]"])
subprocess.Popen(["git", "config", "--global", "user.email", "41898282+github-actions[bot]@users.noreply.github.com"])

print("Push to remote...")
subprocess.Popen(["git", "add", "-A"])
subprocess.Popen(["git", "commit", "-m", "Your run url"])
subprocess.Popen(["git", "push"])

Hope it helps and if I misunderstood you, please correct me. Thanks

Hi @niconbw yes, you understood the requirement correctly. I wanted to understand where the comment would go?

Hi @sayakpaul

Did you mean to know where the metrics.txt file is generated? If so, the file will be saved into the path specified by the default environment variable github.workspace. Then you can directly access this default working directory.

@niconbwI was meaning where the comment would go? Because my requirement is to be able to log the run URL as a comment upon each new commit.

Hi @sayakpaul
I can’t run your workflow because there is no wandb environment. Generally speaking, if you can get the value of the run_url in your train.py file, you can use this run_url as the comment when submitting the commit.
A simple example :


Result:

If it still doesn’t meet your needs, some screenshots for describing your request will be appreciated.
By the way, there is an existing wandb action. You could check it. Hope it helps.

@niconbwappreciate your help here. I am aware of that repository but I found it kind of complicated to get started with. Hence I preferred something simpler.

Your example does meet the requirement. Do you think it would be better to run this action on a PR and have the run URL as a comment?

Also, if you set your wandb API Key as a GitHub secret my workflow should be able to fetch that and utilize it.

Again, thank you so much for being so helpful :slight_smile:

Thanks for your update.
The action above will save a csv file with more information. You can decide whether to use it according to your needs.

Thanks, @niconbw. So, if I were to extend this use-case to support commenting capabilities upon a new PR what would be the solutions?

Hi @sayakpaul
Not sure but you could try to use Create Pull Request REST API in your train.py file.
More about How to use an API with Python. Hope it helps.

@sayakpaul have you seen the weights and biases GitHub Action? https://github.com/marketplace/actions/get-runs-from-weights-biases

To comment on a PR, its best to use this Action https://github.com/actions/github-script

@hamelsmu thanks for your response. I am aware of that action but my purpose with the action I mentioned in this thread is a bit more reduced. I want to be able to (upon a new commit) -

  • Train a toy ML model
  • Log experiments to wandb
  • Generate artifacts file and provide them as an action artifact download
  • Cache pip dependencies

It might seem very trivial to you but I am doing this for self-learning :smiley:

Then upon a PR, I want to be able to fetch the latest run URL and comment that out on the PR. I have been able to realize these so far - https://github.com/sayakpaul/wine/blob/a49b0d8386c6ef926bca45fd1cd2e8cd474a52b0/.github/workflows with a caveat that my action is picking up on an older run instead of the latest one.