process >= 1.6.14 requires ghc-9.2.4 to be built again, which causes error

[Abastro]

Describe the bug
At some point, I added process >= 1.6.14 as that version fixed a bug.
After a while, I tried to build a test and it failed:

I use hspec as dependency, which has ghc package as a transitive dependency.
Then, it tries to build ghc-9.2.4 package instead of using what should be already available.
First it complained that alex then happy is missing. After installing those, it error-ed out while trying to build GHC modules.

To Reproduce
Steps to reproduce the behavior:

pulp-commons.cabal:

cabal-version:      2.4
name:               pulp-commons

(... version, synopsis, license, ...)

library
  hs-source-dirs:       src
  default-extensions:
    LambdaCase
  default-language:     GHC2021
  ghc-options:          -W
  build-depends:
      base >=4.14 && <5
    , vector
    , containers
    , mtl
    , text
    , filepath
    , directory
    , X11
    , unix
    , stm == 2.5.*
  exposed-modules:
    Defines

test-suite test-pulp-commons
  type:               exitcode-stdio-1.0
  main-is:            Spec.hs
  hs-source-dirs:     test

  default-extensions:
    LambdaCase
  default-language:     GHC2021

  build-depends:
      base
    , pulp-commons
    , process >= 1.6.14
    , QuickCheck >=2.14
    , hspec ^>=2.10

Ran command

cabal v2-test pulp-commons

Then, e..g this happens when alex is not installed

Resolving dependencies...
Build profile: -w ghc-9.2.4 -O1
In order, the following will be built (use -v for more details):
 - ghc-9.2.4 (lib) (requires build)
 - hspec-core-2.10.6 (lib) (requires build)
 - hspec-2.10.6 (lib) (requires build)
 - pulp-commons-0.1.0.0 (test:test-pulp-commons) (first run)
Starting     ghc-9.2.4 (lib)
Building     ghc-9.2.4 (lib)

Failed to build ghc-9.2.4.
Build log (
/home/abastro/.cabal/logs/ghc-9.2.4/ghc-9.2.4-27443687ed3d50d5763876a9d1e19a85cb41f0467f57d384fbf08b6a809af35b.log
):
Configuring library for ghc-9.2.4..
Preprocessing library for ghc-9.2.4..
Error: cabal-3.8.1.0: The program 'alex' is required but it could not be found

Error: cabal: Failed to build ghc-9.2.4 (which is required by
test:test-pulp-commons from pulp-commons-0.1.0.0). See the build log above for
details.

Expected behavior
If ghc package is never supposed to be built, proper error messages should have complained about the version bounds - it took quite some time to .
If ghc package could be built, it should build successfully without errors.

System information

• x86, Linux, Ubuntu
• cabal-3.8.1.0, ghc-9.2.4

Additional context
In the dependencies of the test, when I have process part commented out like:

  build-depends:
      base
    , pulp-commons
    -- , process >= 1.6.14
    , QuickCheck >=2.14
    , hspec ^>=2.10

then existing ghc package is used, and build proceeds normally.

[ulysses4ever]

What version of cabal do you use? With your cabal file I get:

> cabal test pulp-commons
Resolving dependencies...
Error: cabal: The test command is for running test suites, but the target
'pulp-commons' refers to the library pulp-commons from the package
pulp-commons-0.

Also, please try to post reproducers that's easy to use. E.g. a git repository that we could clone or an archive would make our life much easier than the incomplete cabal file that you posted so far.

[ulysses4ever]

I was able to reproduce it (I think) with this pulp-commons.cabal:

> cat pulp-commons.cabal                           1 (2.232s) < 20:40:54
cabal-version:      2.4
name:               pulp-commons
version: 0

library
  build-depends:
      base
    , process >= 1.6.14
    , hspec ^>=2.10

And doing

❯ cabal build pulp-commons                         1 (1.886s) < 20:40:45
Build profile: -w ghc-9.2.4 -O1
In order, the following will be built (use -v for more details):
 - ghc-9.2.4 (lib) (requires build)
 - hspec-core-2.10.5 (lib) (requires build)
 - hspec-2.10.5 (lib) (requires build)
 - pulp-commons-0 (lib) (configuration changed)
Starting     ghc-9.2.4 (lib)
Building     ghc-9.2.4 (lib)

Failed to build ghc-9.2.4.
Build log (
/home/artem/.cabal/logs/ghc-9.2.4/ghc-9.2.4-c245881187aab2525e7916080c5e7520ef9958fb1dd4936286a19e5e17c36525.log
):
Configuring library for ghc-9.2.4..
Preprocessing library for ghc-9.2.4..
happy: 1845: unrecognised directive: %shift

Error: cabal: Failed to build ghc-9.2.4 (which is required by pulp-commons-0).
See the build log above for details.

Maybe, the ghc package should hold some special status in the solver, so that cabal never tries to rebuild it. On the other hand, I'm not even sure why ghc (the package) doesn't pin all it's dependencies exactly: it has == only for a couple of ghc-* dependencies, but maybe it should have it for all of them? @bgamari?

[Abastro]

Ouch, sorry for not having a proper reproducer. I would try to give an easier-to-use reproducer next time I open an issue!

[gbaz]

So there is a list of non-upgradable packages hardwired here and i think it just lacks ghc itself!

cabal/cabal-install/src/Distribution/Client/Dependency.hs

Line 400 in 207d4ee

, pkgname <- [ mkPackageName "base"

(also i think the comment above this line in the source about needing to update a second place is now outdated and this is the only source of truth?)

[ulysses4ever]

@Abastro are you interested in submitting a patch by any chance?

[gbaz]

I think that at least all of "ghc" "ghc-boot" and "ghci" should be added fwiw.

[Abastro]

Let me try opening a PR for adding these.
By the way, what does the list in Distribution.Solver.Modular.Solver do now?
There is already a discrepancy between the two lists (Notably, former missing ghc-bignum).

[grayjay]

By the way, what does the list in Distribution.Solver.Modular.Solver do now?

That list seems redundant with the list in Distribution.Client.Dependency. It looks like cabal avoids choosing a source version for packages in either list. We should probably remove the list in Distribution.Solver.Modular.Solver, because the list in Distribution.Client.Dependency is implemented using constraints, which are more general, and it is better to pass constraints into the solver than hard code non-upgradable package names in the solver.

[andreasabel]

I think I just hit this issue in

• Build with Cabal-3.8 and GHC 9.4 hackage-server#1141 (comment)

Cabal-3.8.1.0 comes with a constraint process >= 1.6.14 (released 2022) for ghc >= 8.2 (released 2017!). This looks completely non-sensical to me. Where does this come from and what purpose does it serve? The only compiler that comes potentially with such a recent process is GHC 9.4.

[andreasabel]

Cabal-3.8.1.0 comes with a constraint process >= 1.6.14

Blame for this extra constraint goes to 95cf762. (Ping @robx.)

Sat Jul 9 14:48:25 2022 +0200
Cabal: depend on process-1.6.14.0 to fix waitForProcess error message

This is guarded to be for GHC from 8.2 only, since the process release depends on base-4.10.

This extra constraint seems to have been removed from master.

[andreasabel]

My problems would not be addressed by the linked PR, so maybe I open a new issue:

• Cabal-3.8.1.0 depends on too recent process-1.6.14, rendering it unusable with package ghc-9.2.4 and older #8554

[Mikolaj]

BTW, #8342 already fixed the underlying problem (the bound on process). But no version with this PR is released yet and also a confirmation it solves user problems would be very welcome.

[andreasabel]

@Mikolaj wrote:

BTW, #8342 already fixed the underlying problem (the bound on process). But no version with this PR is released yet and also a confirmation it solves user problems would be very welcome.

Good point, I shall try this PR in the context of hackage-server.