CodeQL with submodules

Looking for best practices on setting up CodeQL analysis with submodules. Currently we are running the analysis from the top repo, but for any alerts in the submodules it reports “Preview unavailable”, “Sorry, we couldn’t find this file in the repository.”

Is there a way to tell CodeQL to look in the submodule repo?

Also we’d really like to run the analysis on submodule pull requests, is there a preferred method? My thought is to checkout the top repo recursively, then checkout the pull request reference and run the analysis and hopefully the report will then actually show the code with the alert? Any better way to tell CodeQL we really only care about alerts from the specific submodule?

EDIT - I did just find the configuration capability to specify directories… so one part solved

Ok, so I’m checking out the main repo with submodules (gets default submodule), then checking out the submodule (gets the reference submodule) into the expected directory… so I think I’m in business, and should be able to build the code and run analysis on just the submodule (using the path option)

I’m happy to hear that! Let us know if you need help with anything.

The only remaining issue is in reviewing code scanning alerts I still get “Preview unavailable Sorry, we couldn’t find this file in the repository.” Since the submodule is down 1 directory (builds from main_repo_dir):

main_repo_dir
|_ submodule_repo_dir

So all code references look like submodule_repo_dir/code.c#L52, but to resolve from within the submodule it would need to be code.c#L52 (without the submodule_repo_dir). Is there any way to strip leading directory from the reported reference? It’s not all that hard to figure out things by hand, but it would be nice if there was a way to make the links work. The simple example here isn’t that bad, but imagine 10+ submodules all at different locations in the directory structure…

Hi, I checked in with the engineering team for this, and the missing context for annotations within submodules is a known limitation.

However, stripping the directory alone would not be enough here, as we (at that point) do not look through the submodule reference at all.
This requires more work from our side, and we’re tracking that issue internally.

Thanks for the feedback, looking forward to improvements! Running CodeQL analysis in our workflow has been very beneficial!

I’m actually trying it two ways, one is running the analysis as an action from the top repo (which makes sense that it’s extra work to get to the submodule reference), but I’m also running the analysis from the submodule to analyze PRs at that level. I’d think when run from the submodule the path just needs to be correct? Example of the submodule analysis at osal/codeql-build.yml at 4de3def9cd60a72486616f682800449d2953dbfe · skliper/osal · GitHub (nasa/cFS is the main repo, osal is the submodule of interest).

Hi,

I agree that the second way should work.
I can’t see any runs of your codeql action in Actions · skliper/osal · GitHub though?

I would expect that if you run your workflow on the osal repo, it should do an analysis of that, and also report all findings with preview et al.

Thanks for looking! Just updated the workflow to run on all pushes. See Actions · skliper/osal · GitHub, and the reference issue in https://github.com/skliper/osal/security/code-scanning/93?query=ref%3Arefs%2Fheads%2Ffix775-add_codeql (I think the osal/src/tests/time-base-api-test/ time-base-api-test.c #L223 link just needs “osal/” removed"

Sorry for the really late reply, somehow this thread somehow slipped through the cracks.

I am in conversations now with the engineering teams - two workarounds I had in my mind already don’t work, I’ll get back to you if I find something that does.

1 Like

a) We are tracking a bug in the codeql-action that we expose a setting that you could use to set the source root of the analysis to osal, that would fix all your problems.
Unfortunately, there’s no ETA for that.
b) In the meantime, you can do the following:
Under the hood, the codeql action uses the codeql CLI.
If you use the codeql CLI directly (instead of the action) you have access to the relevant setting.
You need to (approximately) do the following:

  1. Run your setup steps for the build
  2. Run codeql database create with flags for cpp, and (that’s the thing the action can’t do) --source-root=osal. As build command, put in your make invocation that builds all targets you’re interested in.
  3. Run codeql database analyze on the newly created database from step two, with the suites you are interested in. Use --format=sarif-latest to get a sarif file out of that.
  4. Run codeql github upload-results with that sarif file to get the analysis results into code scanning. The values for the required parameters should all be supplied by the various Actions env vars.

See also CodeQL Code Scanning: improvements for users analyzing codebases on 3rd party CI/CD systems - GitHub Changelog for this workflow. That blog post also links to the codeql CLI documentation.
In general, I find that codeql <subcommand> --help is also of high quality.

I believe that you will not need to install codeql on the actions workers, as that should come pre-installed.

1 Like

In theory, to use the codeql commands in a GitHub actions workflow, codeql-actions will need to be initialized using github/codeql-action/init@v1? I tried this here, and was given an error that the codeql command was not found.

I was also looking into possibly downloading the zip CodeQL Bundle, but I am having issues doing so on the command line.

Hi @ArielSAdamsNASA ,
Using github/codeql-action/init will run some commands in the background that will clash with using the CLI directly.
You can see here where the codeql-bundle will be installed. If you add that directory (or its codeql subdirectory) to the PATH, you should be good to go.
The env variable AGENT_TOOLSDIRECTORY is available during job execution.

Otherwise, you should be able to install the latest CLI release from https://github.com/github/codeql-cli-binaries/releases/latest and use that. Follow the instructions at https://codeql.github.com/docs/codeql-cli/getting-started-with-the-codeql-cli/to also get the necessary queries.

Let me know if you need any more help!

Thank you! I was able to use the latest CLI release in a GitHub Actions workflow, however, the preview is still unavailable.

Can you take a screenshot? This page is not publicly accessible.

Workflow:

name: "CodeQL Analysis"

on:
  push:
  pull_request:

env:
  SIMULATION: native
  ENABLE_UNIT_TESTS: true
  OMIT_DEPRECATED: true
  BUILDTYPE: release

jobs:

  CodeQL-Build:
    runs-on: ubuntu-18.04
    timeout-minutes: 15
    
    steps:
    
      - name: Checkout bundle
        uses: actions/checkout@v2
        with:
          repository: nasa/cFS
          submodules: true

      - name: Checkout submodule
        uses: actions/checkout@v2
        with:
          path: osal 

      - name: Check versions
        run: git submodule

      - name: Check versions
        run: git submodule

      # Setup the build system
      - name: Set up for build
        run: |
          cp ./cfe/cmake/Makefile.sample Makefile
          cp -r ./cfe/cmake/sample_defs sample_defs
          make prep
   
      - name: CodeQL Download
        run: |
            mkdir codeql
            cd codeql && wget https://github.com/github/codeql-action/releases/download/codeql-bundle-20210421/codeql-bundle-linux64.tar.gz
            tar xvzf codeql-bundle-linux64.tar.gz codeql   
            readlink -f codeql
            
      #Run codeql database create
      - name: CodeQL Database
        run: |
            cd ../
            ./osal/codeql/codeql/codeql database create osal-database --language=cpp --source-root=osal --command='make -j native/default_cpu1/osal/'
            
      #Run codeql database analyze on the newly created database from step two
      - name: CodeQL Analyze 
        run: |
            cd ../
            ./osal/codeql/codeql/codeql database analyze osal-database /home/runner/work/osal/osal/codeql/codeql/qlpacks/codeql-cpp/JPL_C/ --format=sarif-latest --output=analysis.SARIF

        
      #Run codeql github upload-results with that sarif file 
      - name: CodeQL Upload
        run: |
            cd ../
            ./osal/codeql/codeql/codeql github upload-results --sarif=analysis.SARIF --ref=refs/heads/CodeQLPreview --repository=ArielSAdamsNASA/osal --commit=$GITHUB_SHA --github-auth-stdin=${{ secrets.GITHUB_TOKEN }}

Output:

That is strange! I’ve pinged the engineering team, and will get back to you once I’ve heard from them.

1 Like

Hi Ariel,

the engineering team confirmed this behavior as normal.
The reason is that we currently don’t look for files across submodules for the preview.
There is an internal feature request that we produce the snippet that’s shown there as part of the analysis of the code, and then use that for the preview window, but that feature is not being worked on right now.

Let me know if I can help you further.