--ghc-options are cached, but shouldn't be

[konsumlamm]

Describe the bug
When passing GHC options to cabal build (via --ghc-options), those options are still applied on later invocations of cabal build (unless cabal clean has been performed in the meantime).

To Reproduce
Steps to reproduce the behavior:

$ cabal v2-build --ghc-options="-ddump-simpl"
outputs core
modify the code
$ cabal v2-build
still outputs core

Expected behavior
No core output when building the second time (i.e. don't use the GHC options from the previous build).

System information
cabal-install version: 3.6.2.0
GHC version: 8.10.7

[Mikolaj]

This may be a left-over from the v1- system where you "configured" a package first and it kept various settings in a configuration. But now cabal build should indeed cache stuff based on its arguments, not unconditionally.

Are we correct that this is a bug? Where does it reside? Cabal library, client, elsewhere?

[phadej]

We don't want to recompile packages (especially dependencies, as command line --ghc-options applies to everything), when non-output changing flags are specified. -ddump flags are such flags.

See #4247

[Mikolaj]

Hmm, arguably, -ddump are (debug, side-) output flags. How easy is it [edit: for the user, manually, as a workaround] to recompile without that flag?

[phadej]

They change the compiler debug output, but they don't change the result of compilation.

How easy is it ...

rm -rf dist-newstyle && cabal build

(I'm not sure what cabal clean does, thus i just remove the build directory).

[Mikolaj]

@konsumlamm: could you try that? Is that an acceptable workflow for you?

[Mikolaj]

@phadej: so, for my continued education, the deps compiled with the flags will remain in store/, but removing the build directory deletes the cache of flags, so the next build will not pick up the fancy deps, but will use/build vanilla ones?

[konsumlamm]

cabal clean works as a workaround (that's what I'm already doing), but it recompiles the whole package, so it's not really ideal.

We don't want to recompile packages (especially dependencies, as command line --ghc-options applies to everything), when non-output changing flags are specified. -ddump flags are such flags.

So cabal has a list of flags that are non-output changing? Then why are those flags still applied when rebuilding a single module?

[Mikolaj]

@konsumlamm: I don't understand your question. The flags remain in flag cache (whatever it is), right?

[phadej]

works as a workaround (that's what I'm already doing), but it recompiles the whole package, so it's not really ideal.

Toggling {-# OPTIONS_GHC -ddump-simpl #-} in a module you are interested is another option.

[konsumlamm]

I don't understand your question. The flags remain in flag cache (whatever it is), right?

Yes, the flags from the previous builds are reused, but not everything is recompiled (only the module that changed). So this implies to me that this only happens with certain flags (non-output changing ones, otherwise it should recompile everything). Or are all flags passed via --ghc-options considered to be non-output changing?

Toggling {-# OPTIONS_GHC -ddump-simpl #-} in a module you are interested is another option.

That works even better, I think this a good enough workaround (at least for my use case), thanks!

[Mikolaj]

Yes, the flags from the previous builds are reused, but not everything is recompiled (only the module that changed). So this implies to me that this only happens with certain flags (non-output changing ones, otherwise it should recompile everything).

Yes, that's how I undrerstands @phadej's comment above and that's what I find in the linked ticket and PRs relating to it.

[konsumlamm]

Yes, the flags from the previous builds are reused, but not everything is recompiled (only the module that changed). So this implies to me that this only happens with certain flags (non-output changing ones, otherwise it should recompile everything).

Yes, that's how I undrerstands @phadej's comment above and that's what I find in the linked ticket and PRs relating to it.

So then my question is: Why are the flags from the previous build reused instead of using the new ones (which in my case are none), when they don't change the output anyway?

[Mikolaj]

A good question. In theory, there could be flags that break the build if they are not applied consistently to all files. Not sure if that's the answer, though.

[jneira]

This has been resolved by #7973, thanks all!

[fgaz]

@jneira could you clarify whether that pr disables/fixes caching only for output-changing flags or for all flags? I agree with @phadej that non-output-changing flags should not invalidate the cache. Or does it simply fix #7816 (comment)?

@Mikolaj

so, for my continued education, the deps compiled with the flags will remain in store/, but removing the build directory deletes the cache of flags, so the next build will not pick up the fancy deps, but will use/build vanilla ones?

iirc those flags do not get into the hash inputs at all (the output will be the same anyway), so the deps will be reused

[jneira]

sorry I misunderstood the mentioned pr, it only made ghc-option be applied to local packages and did not affect its caching
the pr mentioned the caching issue but did not claim its resolution
afaiu that supposed dependencies are not rebuilt but cause they simply are not applied to them

[patrickdoc]

For this particular issue, it's easiest to ignore dependencies. #7973 should have fixed non-local packages to behave more as you would expect (re-used when possible, not rebuilt for flag changes like -ddump*, etc).

For this issue: compiler flags are cached as part of the configuration. But, that caching is blind to a specific set of flags (see

cabal/Cabal/src/Distribution/Simple/Program/GHC.hs

Line 49 in bd24cac

normaliseGhcArgs :: Maybe Version -> PackageDescription -> [String] -> [String]

). This is to avoid unnecessary recompilation.

Unfortunately, this causes the ignored flags to be "sticky". You can't add/remove an ignored flag in the cache by itself. You either have to delete the cache (rm -rf dist-newstyle or cabal clean), or overwrite the cache by adding/removing a non-ignored flag.

For example,

cabal clean
cabal build --ghc-options=-ddump-simple all
# modify a module
cabal build all                          # -ddump-simpl still applied because ["-ddump-simpl"] == [] in the cache
cabal build --ghc-options=-ddump-asm all # nothing happens because -ddump* is ignored
cabal build --ghc-options=-fcse all      # local package rebuilt because a non-ignored flag changed, -ddump-simpl removed from cache

See "Caching and Recompilation Avoidance" and bullet 3 under "Proposal" for more info: #7998