Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: support continuously running fuzzing with OSS-Fuzz #50192

Closed
katiehockman opened this issue Dec 15, 2021 · 10 comments
Closed

cmd/go: support continuously running fuzzing with OSS-Fuzz #50192

katiehockman opened this issue Dec 15, 2021 · 10 comments
Labels
FeatureRequest fuzz Issues related to native fuzzing support NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@katiehockman
Copy link
Contributor

katiehockman commented Dec 15, 2021

Motivation

Currently, OSS-Fuzz supports continuously fuzzing go-fuzz targets in its libfuzzer build mode. There is no OSS-Fuzz support for native fuzz targets, meaning that a developer must fully manage the execution of a fuzz test, directly from the command line. In many cases, this will mean running fuzzing as a background process, or on a separate machine.

This is insufficient for many use cases. It can often take several days to several months for fuzzing to discover a crash. Even if a crash is found quickly, OSS-Fuzz can do a better job of promptly reporting the crash to the interested parties.

We should make the necessary changes in the standard library to support the integration of native fuzzing with OSS-Fuzz, and partner with the OSS-Fuzz team to hook it up.

Two main approaches come to mind for how to support this:

Option 1: Go native fuzzing supports a libFuzzer mode.

This could be available through a new -fuzzengine flag, e.g.:

$ go test -fuzz=Fuzz -fuzzengine=libfuzzer

This will create an ELF file which will be linkable by libFuzzer for native fuzz targets.
This can likely be heavily based on go114-fuzz-build, which performs a similar task, just for go-fuzz targets instead of native targets.

Using this with -c could look something like this:

$ go test -c -fuzz=FuzzPng -fuzzengine=libfuzzer
$ clang FuzzPng.a -o FuzzPng -fsanitize=fuzzer
$ ./FuzzPng corpus

We will need to create a template, much like the template in go114-fuzz-build, which will export a LLVMFuzzerTestOneInput function which can be used by libFuzzer for each mutation.

The template in go114-fuzz-build is fairly straightforward, since a go-fuzz target can take the []byte provided by libFuzzer directly. This will be more involved for the native support, since it’s exported targets take a *testing.F, and the fuzz target is not defined until f.Fuzz is called.

This will be more complicated for structured data as well, however it should still be possible since libFuzzer can be turned into a structure-aware fuzzing engine.

Option 2: OSS-Fuzz integrates with native fuzzing directly

Native support could be integrated into OSS-Fuzz directly. This would likely not be straightforward from the OSS-Fuzz side, and it’s unclear how much effort would be required for this from their side.

OSS-Fuzz can either run the fuzz targets directly using go test -fuzz=FuzzTargetName, or they can build the binary using go test -c and execute it that way.

From the Go side, fuzzing will need to support the following features which are not currently supported:

  • cmd/go: support corpus minimization #49290
  • Fuzzing metrics, including the maximum number of seconds it takes to execute the slowest input during fuzzing and the maximum memory used during fuzzing, which are not currently implemented. We need to figure out how these metrics are shared with OSS-Fuzz.

Conclusion

It's going to be difficult to make a decision right now. We need to do more investigation into what would be involved from the OSS-Fuzz side, and outline the changes would need to be made in the standard library for each option. For example, there is still ambiguity around how native fuzzing could be run in a -libfuzzer mode (e.g. how is the seed corpus provided?).

My inclination is towards (2), as it will provide product parity between running fuzz targets locally, and continuously with OSS-Fuzz. But we'll see.

/cc @golang/fuzzing

@katiehockman katiehockman added NeedsFix The path to resolution is known, but the work has not been done. FeatureRequest fuzz Issues related to native fuzzing support labels Dec 15, 2021
@katiehockman katiehockman added this to the Go1.19 milestone Dec 15, 2021
@naveensrinivasan
Copy link

On option 2, I recently asked the question google/oss-fuzz#7020 and it looks like oss-fuzz is likely to support. cc @inferno-chromium

@inferno-chromium
Copy link

Adalogics and @oliverchang plan to evaluate these options and pick one of them in 2022. See google/oss-fuzz#7020 (comment)

@DavidKorczynski
Copy link

We have a working PoC for option 1 here: google/oss-fuzz#7055

@prestonvanloon
Copy link

I'd like to express interest in option 1. It is desirable to run alternative fuzzing engines, i.e. libFuzzer, AFL, etc.

@ianlancetaylor
Copy link
Contributor

CC @fuzz

Looks like this didn't make 1.19. Moving to Backlog. Please recategorize as appropriate. Thanks.

@ianlancetaylor ianlancetaylor modified the milestones: Go1.19, Backlog Jun 24, 2022
@firelizzard18
Copy link
Contributor

Option 1 is far more flexible, as it would work with any libfuzzer-based tools. Specifically, I want to use GitLab's continuous fuzzing support.

@synenka
Copy link

synenka commented Mar 21, 2024

Can this issue be marked as "closed" now that #7055 was merged?

@rolandshoemaker
Copy link
Member

I think we'd still like to ideally provide native support for this. The vast majority of people using the Go fuzzer are unlikely to be integrating with libFuzzer, but instead just using the builtin fuzzing engine. Requiring users to use a third party system to get this basic functionality seems like a pretty poor UX.

@firelizzard18
Copy link
Contributor

@rolandshoemaker The title of this issue is support continuous fuzzing with OSS-Fuzz and google/oss-fuzz#7055 appears to do that. What is there left to do?

@rolandshoemaker
Copy link
Member

Oh bah, my bad, thought this was a different issue. Seems fine to close this. I think we'd still like a better integration with oss-fuzz, that uses the native engine rather than the libFuzzer mode, which was being discussed at one point, but no one is working on that for now as far as I know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FeatureRequest fuzz Issues related to native fuzzing support NeedsFix The path to resolution is known, but the work has not been done.
Projects
Status: Done
Development

No branches or pull requests

9 participants