Unable to grep output of command

I have a step that runs a script to deploy a revision to google cloud endpoints and then get the configuration ID from that. The script works perfectly in the local environment but fails to grab the configuration ID in the github actions runner. The relevant lines in the script follow:

# Deploy updated endpoints configuration
gcloud endpoints services deploy openapi.json --project ${PROJECT}

# Get latest configuration
services=$(gcloud endpoints configs list --service=${SERVICE} --limit 1)
# Extract the configuration ID from the services list - The format is YYYY-MM-DDrXX
config_id=$(echo ${services} | grep -ioE '\d+\-\d+\-\d+[a-z]\d+')
echo "ConfigID: ${config_id}"

On the last line, the config_id should look similar to 2020-07-21r21. This works as expected in the local environment, but not in the action runner. If I echo services, I see CONFIG_ID SERVICE_NAME 2020-07-21r20 *** (the *** is the service being passed in as a command to the script via a secret env variable)

The step is:

- name: Get Config ID
  run: ./config_id -p ${{ secrets.PROJECT }} -s ${{ secrets.SERVICE }} -n ${{ secrets.NAME }}

@therealkris,

In GitHub Actions workflow, environment variables are case-sensitive.

From your comments,

If I echo services, I see CONFIG_ID SERVICE_NAME 2020-07-21r20 ***

I notice the variable name “CONFIG_ID” is completely uppercase.

But in the script,

echo "ConfigID: ${config_id}"

you are using the variable name “config_id” that is completely lowercase.

Please change the variable name to uppercase (echo “ConfigID: ${CONFIG_ID}”) in the script to see if the problem can be solved.

@brightran That’s the header text in the command output, not an actual variable. The variable config_id comes from this line: config_id=$(echo ${services} | grep -ioE '\d+\-\d+\-\d+[a-z]\d+'). It takes that output, finds the text that matches the regex (in this case 2020-07-21r20), and sets that value as config_id. Remember, this script works perfectly when run locally. When run as part of the actions workflow, though, this step fails.

As a note, though, I updated the whole thing from a shell script to a python script and it is now working fine, so my immediate issue is resolved. The python script does not have the same issue with the regular expression matching. What I’d be interested in knowing, however, is what the issue is in the workflow run that causes the grep to return nothing. I suspect it has something to do with the *** in the gcloud command output, which corresponds to a service url that is stored in a secret, but I don’t know.

@therealkris,

The variable config_id comes from this line: config_id=$(echo ${services} | grep -ioE '\d+\-\d+\-\d+[a-z]\d+') .

Ah, yes, I missed it.
Now I seem find the reason that way the command fails to get the configuration ID.

In the grep command, it seems does not recognize the metacharacter ‘\d’ in the regular expression “\d+\-\d+\-\d+[a-z]\d+”.
I know, generally many people (include me) are used to using the metacharacter ‘\d’ instead of '[0-9]‘ to represent the digital characters in the regular expression.
However, I find that some command line interfaces do not recognize the metacharacter ‘\d’, and they seem to prefer the metacharacter '[0-9]‘.
Maybe in some environments, the two metacharacters are supported in the commands.

I tested on my local environment (Windows 10, Bash), and only when I used the metacharacter ‘[0-9]’, the configuration ID can be fetched. When using '\d‘, it returned nothing.

  • [Test 1]
    script:
services='CONFIG_ID SERVICE_NAME 2020-07-21r20 this the service name'
# using the metacharacter '[0-9]'
config_id=$(echo ${services} | grep -ioE '[0-9]+\-[0-9]+\-[0-9]+[a-z][0-9]+')
echo "config_id = ${config_id}"

result:
0-9

  • [Test 2]
    script:
services='CONFIG_ID SERVICE_NAME 2020-07-21r20 this the service name'
# using the metacharacter '\d'
config_id=$(echo ${services} | grep -ioE '\d+\-\d+\-\d+[a-z]\d+')
echo "config_id = ${config_id}"

result:
-d

Ah that’s interesting. I didn’t consider that the regex would be supported differently across environments. After some testing, I’ve found that on my that macOS command line, \d works, but in the ubuntu command line, it does not and only [0-9] works.

Thanks for the tip!

The issue is the use of -E, which means to use Unix extended regular expressions. Use -P instead for Perl-compatible regular expressions, and \d works. Or alternatively use [[:digit:]] for the equivalent character class in extended regular expressions. :wink:

I guess grep on MacOS has different defaults there.

1 Like

@airtower-luna,

Thanks for your reminder.
Yes, when using the option ‘-P’ (–perl-regexp), both the ‘\d’ and ‘[0-9]’ can work as expected.

@therealkris,

I just remembered that there are many regular expression flavors. The popular regular expression flavors include Perl, PCRE, PHP, .NET, Java, JavaScript, XRegExp, VBScript, Python, Ruby, Delphi, R, Tcl, POSIX, etc…
Each regular expression flavor is the syntax and behavior of a particular engine.

When using the option ‘-E’ (–extended-regexp), it is specified by POSIX.
One key syntactic difference is that the backslash (\) is NOT a metacharacter in a POSIX bracket expression. So in POSIX, the regular expression ‘[\d]’ matches a ‘\’ or a ‘d’.

@airtower-luna @brightran Yep, I see that now. That you guys for the help! I’m all set.