set-output Truncates Multiline Strings

Hi,

I am using this code to create a release for one of our repositories:

- shell: bash
        id: release_description
        run: |
          description=$(./resources/get_release_description.sh ${{ steps.versioning.outputs.cli_version }})
          echo $description
          echo "::set-output name=description::$description"
        if: startsWith(steps.commit_message.outputs.commit_message, 'Version change')
      - shell: bash
        run: |
          echo ${{ steps.release_description.outputs.description }}
        if: startsWith(steps.commit_message.outputs.commit_message, 'Version change')
      - uses: csexton/create-release@add-body
        id: create_release
        with:
          tag_name: ${{ steps.versioning.outputs.cli_version }}
          release_name: safe-cli
          draft: false
          prerelease: false
          body: ${{ steps.release_description.outputs.description }}

This works without any errors. However, the problem is, the description (the body parameter of the create_release action) is basically a small markdown document, so it’s a multiline string. The problem is that ::set-output is truncating everything except the first line. I can confirm that by echoing the description in the same action where it’s assigned and then referencing the description output variable in the next action.

Could someone please advise how I can get this to work with a multiline string?

Thanks. 

2 Likes

I have reported your question to the appropriate engineering team for further evaluation. The team will review the feedback and notify me about the next steps. I will update here in time. Thank you for your understanding.

1 Like

I got response from the team. 

% and \n and \r can be escaped like below, the runner will unescape in reverse.

content="${content//'%'/'%25'}"
content="${content//$'\n'/'%0A'}"
content="${content//$'\r'/'%0D'}"

Please try to add next three red lines to your yml, kindly let me know whether this could help.

- shell: bash
        id: release_description
        run: |
          description=$(./resources/get_release_description.sh ${{ steps.versioning.outputs.cli_version }})
          echo $description
<font color="#FF0000"> description="${description//'%'/'%25'}"
          description="${description//$'\n'/'%0A'}"
          description="${description//$'\r'/'%0D'}"</font>
echo "::set-output name=description::$description"
if: startsWith(steps.commit_message.outputs.commit_message, 'Version change')

 I tested in my side, after adding these lines, I can use echo " ${{ steps.release_description.outputs.description1 }} " to output multiple line value. Please pay attention to “”

multiline.png

13 Likes

Sorry, I just noticed your reply!

Thanks very much for this, it resolves the issue.

It took me an hour to realize that multi-line strings were the issue for me, and then a few more hours to find this solution.

It would be great if multi-line strings as values could be better supported, and documented.

Here are a few observation. First, it is important to suppress word-splitting upon expansion in bash. This is easiest done by enclosing with double quotes:

REPORT="$(cat logfile)"

Similarly, word splitting also has to be suppressed when reading back:

echo "$REPORT"

The double quotes are necessary.

However, Workflows gets confused/truncates when dealing with an expanded multi-line value:

echo "::set-env name=REPORT::$REPORT"

This does not work for multi-line variables.

The solution above uses a standard bash feature to expand and substitute characters:

REPORT="${REPORT//'%'/'%25'}"
REPORT="${REPORT//$'\n'/'%0A'}"
REPORT="${REPORT//$'\r'/'%0D'}"

This essentially makes the value into a single line string.

The github engineers then also knew that when reading back the variable in the workflow these escape values get substituted back to the actual characters:

${{ env.REPORT }}

This results actually in a multi-line string. Unfortunately, this is not documented anywhere and comes across a bit as magic. If there are other substitutions it would be good to also document those.

1 Like

Thanks so much posting this. I came across the same issue and found this rather obscure solution, after a lot of trial and error.

A couple observation dealing with multiline values, mostly as a reminder on how this works.

If using bash, it is important to suppress word splitting upon parameter expansion, to keep the line breaks. It is easiest with double quotes:

REPORT="$(cat log.out)"

Accessing the variable also invokes word splitting, which needs to be suppressed:

echo "$REPORT"

However, set-output or set-env does not work with expanded multiline values:

echo "::set-env name=REPORT::$REPORT"

This does does not work, as Workflows somehow truncates or ignores the value.

The solution above is to escape the newlines and other apparently unhandled characters using bash expansion and substitution. This makes the value effectively single line.

The thing to notice is that Workflows substitutes the escaped characters back when the parameters is used in ${{ }}:

body: |
  eslint and diff report:
  ```
  ${{ env.REPORT }}
  ```

Here the final value includes actual newlines.

It would be great if this behaviour cold be documented somewhere. For example, are there other substitutions happening ?

@andreasplesch  Thank you for your further investigation. I would recommand you to create an issue in the action toolkit repo . You could ask for adding an example for set-env and set-output with multiline values in this document. 

Thanks. Good to know that there is more in depth documentation available in the toolkit repo. I think a link from the main help page at https://help.github.com/en/actions to this documentation could be helpful.

Hm, it looks like newlines should already be escaped automatically:

https://github.com/actions/toolkit/blob/master/packages/core/src/command.ts#L76

Anyways, here is the new issue in the repo:

https://github.com/actions/toolkit/issues/403

and a related issue:

https://github.com/actions/toolkit/issues/193

Escaping of special characters already happened for javascript actions but does not for bash scripts. There is a plan to update documentation.