Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add nix to projects (using nix) #6317

Closed
wants to merge 2 commits into from

Conversation

Pamplemousse
Copy link

@Pamplemousse Pamplemousse commented Aug 26, 2021

⚠️ This is a Work-In-Progress that does not even work (yet?) ⚠️

As I am struggling to integrate nix to OSS-fuzz, these PR (with #6338) helps to keep track of the progress, and help gather feedback on approaches taken.

Context

As part of a fellowship, I work on fuzzing nix; More details on https://discourse.nixos.org/t/tweag-fellowship-fuzzing-nix-0 .
One of the "end-goal" of the project is to integrate nix to OSS-fuzz; This PR is a step in that direction.

Approach

Unlike #6338;

This approach aims to stay "close" to the build system used upstream to avoid maintenance burden:

  • use nix to:
    • manage build reproducibility (i.e. if it works upstream, then it works in the Dockerfile; no extra management of dependencies)
    • ease run time reproducibility (copy the pertinent content from the store in $OUT)
  • leverage upstream's autotools system:

(Another pretty similar approach would be to compile statically using pkgsStatic packages from nixpkgs in upstream's flake.nix, but this is not well supported, and many such packages currently fail to build...)

Disadvantage: does not work (yet).

TODO

cc @regnat

@google-cla
Copy link

google-cla bot commented Aug 26, 2021

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@Pamplemousse Pamplemousse changed the title [WIP] Add nix to projects [WIP] Add nix to the projects (using nix) Aug 27, 2021
@Pamplemousse Pamplemousse changed the title [WIP] Add nix to the projects (using nix) [WIP] Add nix to projects (using nix) Aug 27, 2021
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
@google-cla
Copy link

google-cla bot commented Sep 2, 2021

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@Pamplemousse
Copy link
Author

I am having a hard time building for coverage, see #6385 .

@Pamplemousse
Copy link
Author

@googlebot I signed it!

@Pamplemousse
Copy link
Author

About integrating the fuzz targets, and related local.mk (makefile) upstream:

  1. If a fuzz target is put upstream, then its related local.mk should be upstream too.
    The idea is to avoid bitrot; Separating those two artifacts will lead to a fuzz target being harder to maintain from upstream only;
  2. The recipes in local.mk relies on fuzzing-specific flags, that are not present in the upstream code, and hard to express in the upstream building logic (Port Nix to Meson NixOS/nix#3160 (comment)). Furthermore, OSS-fuzz opinionated values over these flags conflicts in places with nix build system (example: --stdlib=libc++);
    i.e. nicely integrating local.mk upstream is not possible, and hence would need an associated build.sh not to bitrot.
  3. a. build.sh is required by OSS-fuzz to be present here;
    b. It has a specific interface (i.e. environment variables) to deal with the abstraction OSS-fuzz wants to provide (i.e. fuzzing engine and sanitizer agnosticity).
    The second point in particular makes it annoying to introduce a build.sh-like upstream, because the abstraction expected by OSS-fuzz conflicts with upstream's.

So:
Fuzz targets, local.mk, and build.sh should live here.
(at least until it will be possible to comply with OSS-fuzz's build abstraction from upstream's build system - for example if NixOS/nix#3160 gets merged).

  * in development only
  * to be removed from the PR

Signed-off-by: Pamplemousse <xav.maso@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants