Elm 0.19 support

[domenkozar]

won't have time to work on this soon, but internals changed quite a bit.

[jb55]

Do you think it could be simplified now by simply looking at the direct/indirect dependencies in elm.json instead of calling out to the Elm library?

So:

1. Read elm.json for all direct/indirect dependencies (since I believe these represent the resolved dependencies)
2. Prefetch + continue like normal?

[jb55]

So I implemented the above algorithm here: jb55@faa46d4

I'm going to test it with an elm project I'm converting over to see if it works.

[domenkozar]

That looks really good! :)

[jb55]

so I got pretty close, but this doesn't work because elm expects cached build artifacts in the elm-stuff or ~/.elm cache dir. Here's the message I get back:

/build/project/fakehome/.elm/0.19.0/package/NoRedInk/elm-json-decode-pipeline/1.0.0/objs.dat
is corrupt.

Elm caches build artifacts in the following directories:

    /build/project/fakehome/.elm
    elm-stuff/

Maybe you recently installed a command line tool or editor plugin that messes
with them? They definitely should not be doing that, but you never know! So
maybe try deleting them? Everything will be rebuilt from scratch. This may help
reveal the corrupting influence.

This seems obvious in retrospect: elm install fetches and then "builds" the dependencies, storing binary build artifacts into the ~/.elm cache. Right now I'm just fetching the sources.

Not sure what to do next, it looks like we'll still need to plug into the elm compiler internals to build the fetched sources...

[domenkozar]

The whole idea is for Nix to just prefetch source and then use elm to build them. I think it does download versions.dat which is binary representation of Elm package index. Not sure how objs.dat fit into this :)

[domenkozar]

Some additional info: gdotdesign/elm-github-install#62

[domenkozar]

I think I know how to implement this, it should make it really fast as hashes are now precalculated: https://package.elm-lang.org/packages/elm/time/latest/endpoint.json

Most of the relevant code is in Deps.Website module of the Elm compiler.

https://package.elm-lang.org/all-packages is what is turned into versions.dat in Deps.Cache

[jb55]

Nice, let me know how it goes. I'm currently deploying my 0.19 app with duct tape and glue.

[domenkozar]

Yes. Also need to make a proper Nix api out of https://github.com/NixOS/nixpkgs/blob/67f58eee5ee8d21ccd914fbcde18e1be146dd490/pkgs/development/compilers/elm/default.nix#L88

I'll get on with this in a couple of weeks, not before.

[domenkozar]

Nix side is done: NixOS/nixpkgs#50653

[jerith666]

Here's the commit updating my elm project to 0.19: jerith666/elbum@04faada.

In particular note that I couldn't reuse the versions.dat that's already in the nixpkgs expression for elm itself, I had to pull down a newer one and use that.

I think @jb55's suggestion of parsing elm.json is right on track, at least for applications. Not sure it makes sense to have nix expressions for libraries ... ?

[domenkozar]

There is hope for Elm 0.20 https://discourse.elm-lang.org/t/resolving-dependencies-in-elm-0-19-package-projects/2589

[domenkozar]

@jb55 the error you got was due to caching, just do rm -rf elm-stuff before your builds.

[jb55]

@domenkozar hmm interesting, when I run my branch again after removing elm-stuff, I get this:

-- HTTP PROBLEM ----------------------------------------------------------------

The following HTTP request failed:

    <https://package.elm-lang.org/all-packages>

Here is the error message I was able to extract:

    HttpExceptionRequest Request { host = "package.elm-lang.org" port = 443
    secure = True requestHeaders =
    [("User-Agent","elm/0.19.0"),("Accept-Encoding","gzip")] path =
    "/all-packages" queryString = "" method = "GET" proxy = Nothing rawBody =
    False redirectCount = 10 responseTimeout = ResponseTimeoutDefault
    requestVersion = HTTP/1.1 } (ConnectionFailure
    Network.BSD.getProtocolByName: does not exist (no such protocol name: tcp))

getting there...

[domenkozar]

sounds like you need to update your deps, it's trying to fetch something it doesn't have. I'll make a PR how I got things to work, although it's not finished yet.

[domenkozar]

See https://github.com/domenkozar/elm2nix/pull/15

[jb55]

@domenkozar yeah weird I tried deleting everything, running elm bump, tried again, mine still doesn't work. will try your PR.

[jerith666]

if it's trying to fetch https://package.elm-lang.org/all-packages (rather than a specific package), that means it doesn't have a valid versions.dat file at ${ELM_HOME}/0.19.0/package/versions.dat. ($ELM_HOME defaults to ~/.elm, not elm-stuff; fetchElmDeps sets it to a temporary .elm/ subdir in the build dir.)

[jb55]

yup, I was able to get @domenkozar's PR working by copying ~/.elm/0.19.0/package/versions.dat into my project and using fetchElmDeps..

[domenkozar]

Merged to master!