[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perform CodeQL Analysis on C++ code fails with "Oops! A fatal internal error occurred." #402

Open
geoffthemedio opened this issue Feb 24, 2021 · 8 comments

Comments

@geoffthemedio
Copy link

When testing CodeQL on my project's C++ code, the build succeeded but the following analysis step failed. The error in the log has a lot of Java-related details that are perhaps too long to paste here, but it starts with:

Oops! A fatal internal error occurred.
  java.lang.RuntimeException: com.google.common.util.concurrent.ExecutionError: java.lang.InternalError: a fault occurred in a recent unsafe memory access operation in compiled Java code
  	at com.semmle.util.exception.Exceptions.asUnchecked(Exceptions.java:103)
  	at com.semmle.inmemory.trap.TrapImporter.run(TrapImporter.java:106)
  	at com.semmle.cli2.ql.dataset.ImportCommand.executeSubcommand(ImportCommand.java:86)
  	at com.semmle.cli2.picocli.PlumbingRunner.run(PlumbingRunner.java:110)
  	at 
[...]

The test can be seen here: https://github.com/freeorion/freeorion/pull/3325/checks?check_run_id=1965808793

Googling the error lead to an issue that suggested making a report such as this.

@alexet
Copy link
Contributor
alexet commented Feb 24, 2021

Thanks for the report. This is a badly reported out of disk space error.

Sometimes clearing the build artefacts after the build but before analysis will get you enough space if your build artefacts take up a lot of space. Building less can also help although it looks like you aren't building any third party things. Otherwise you may have to use a machine with more disk space.

@geoffthemedio
Copy link
Author

Sometimes clearing the build artefacts after the build but before analysis will get you enough space if your build artefacts take up a lot of space.

Could you point me to instructions for how to do that?

Building less can also help although it looks like you aren't building any third party things.

It might be the template-heavy Boost-dependent code.

Otherwise you may have to use a machine with more disk space.

Is that something I can control with GitHub Actions CodeQL? Or do you mean it must be run offline instead?

@alexet
Copy link
Contributor
alexet commented Feb 24, 2021

Looking into this a bit further I think the issue is just the template heavy code. We record data about all template instatiations which takes up a lot of space. It looks like you need about 20-25GB of space to run codeql on your codebase.

Sadly I don't think this is going to work on the standard runners as they only have 17GB. You could run the analysis offline.

@geoffthemedio
Copy link
Author

Does it need to link the final libraries and binaries to be able to run the analysis?

If not, I could exclude some subprojects or source files, such as the very template-heavy resource-intensive parsers, and still hopefully get useful results about the rest of the codebase.

@alexet
Copy link
Contributor
alexet commented Feb 24, 2021

We can get better results if we see the linker execution as we can disambiguate calls where linking means only one target is available but if everything is linked together anyway it is usually less important as no disambiguation is needed.

@geoffthemedio
Copy link
Author

I modified the .yml to set some defines in the CMake call to exclude building several projects. There's no pre-made define to exclude the template-heavy parsers, but nonetheless this seems to have allowed the analysis to run partially. However, it still eventually runs out of disk space, see: https://github.com/freeorion/freeorion/runs/1974640186?check_suite_focus=true#step:7:129

I'd like to try the suggestion to "clearing the build artefacts after the build", but I'm not sure how to do that. Googling leads to various scripts for actions for cleaning up stale artifacts, which seem to be intended to be run periodically as a separate action, but I presumably need to do this during the CodeQL action itself. Could you point me to instructions or an example of what to add to the .yml to make that happen?

Also, is there a way to view the results of the analysis that ran partially, or perhaps were necessary subsequent steps not run after the out of disk failures?

@alexet
Copy link
Contributor
alexet commented Feb 25, 2021

The "clearing the build artefacts after the build" suggestion was something like doing rm -rf build at the end of the of the build as we don't care about the build results. However I hadn't looked in detail and hadn't noticed that you only had about 200MB to spare so it might not gain you much.

Unfortunately there isn't really a way of seeing the partial results as we need to post process the query results.

@geoffthemedio
Copy link
Author

rm -rf build didn't help, but I was able to get it to run through and generate a single useful warning from the analysis by excluding some source files from the build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants