Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move the CLI into its own crate #1351

Merged
merged 13 commits into from
Jun 16, 2023
Merged

Move the CLI into its own crate #1351

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
d91f02e
Fetch the wasm-bindgen-cli version from Cargo.lock instead of Cargo.toml
vkleen Jun 7, 2023
dde557e
Move the Nickel CLI to a separate crate
vkleen Jun 9, 2023
495595e
Fix snapshot tests
vkleen Jun 9, 2023
3c34952
Rename the nickel flake output to nickel-lang-cli for consistency
vkleen Jun 9, 2023
bdf29d8
Move the nickel-lang crate into a subdirectory
vkleen Jun 13, 2023
9f7c0e2
Update documentation for the new crate split
vkleen Jun 13, 2023
c4d0a48
Tweak snapshot test instructions
vkleen Jun 14, 2023
093839b
Use project_root instead of ad-hoc environment variables
vkleen Jun 14, 2023
82e7dec
Remove the library from nickel-lang-cli for now
vkleen Jun 14, 2023
451ec86
Rename nickel-lang to nickel-lang-lib and nickel-lang-cli to nickel-lang
vkleen Jun 14, 2023
57d1eef
Apply suggestions from code review
vkleen Jun 16, 2023
ecb4b41
Tweak documentation for nickel_lang_utilities::project_root
vkleen Jun 16, 2023
6aa84e9
Use MacOS 13 github runner because of https://github.com/cachix/insta…
vkleen Jun 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 3 additions & 2 deletions .github/workflows/continuous-integration.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,14 @@ jobs:
matrix:
os:
- ubuntu-latest
- macos-latest
# MacOS 13 because of https://github.com/cachix/install-nix-action/issues/183
- macos-13
rust_channel:
- stable
include:
- os: ubuntu-latest
system: x86_64-linux
- os: macos-latest
- os: macos-13
system: x86_64-darwin
continue-on-error: true

Expand Down
24 changes: 18 additions & 6 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

128 changes: 10 additions & 118 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,127 +1,19 @@
[package]
name = "nickel-lang"
version = "1.0.0"
authors = ["Nickel team"]
license = "MIT"
readme = "README.md"
description = "Programmable configuration files."
homepage = "https://nickel-lang.org"
repository = "https://github.com/tweag/nickel"
keywords = ["configuration", "language", "nix"]
edition = "2021"

[[bin]]
name = "nickel"
path = "src/bin/nickel.rs"
bench = false

[lib]
bench = false

[features]
default = ["markdown", "repl", "doc"]
markdown = ["termimad"]
repl = ["rustyline", "rustyline-derive", "ansi_term"]
repl-wasm = ["wasm-bindgen", "js-sys", "serde_repr"]
doc = ["comrak"]

[build-dependencies]
lalrpop = "0.19.9"

[dependencies]
lalrpop-util = "0.19.9"
regex = "1"
simple-counter = "0.1.0"
clap = { version = "4.3", features = ["derive"] }
codespan = "0.11"
codespan-reporting = "0.11"
logos = "0.12"
serde = { version = "1.0.154", features = ["derive"] }
serde_json = "1.0.94"
serde_yaml = "0.9.19"
toml = { version = "0.7.2", features = ["parse"] }
void = "1"
sha-1 = "0.10.0"
sha2 = "0.10.6"
md-5 = "0.10.5"
directories = "4.0.1"
unicode-segmentation = "1.10.1"
indoc = "2"

termimad = { version = "0.23.0", optional = true }
ansi_term = { version = "0.12", optional = true }

rustyline = { version = "11.0", optional = true}
rustyline-derive = { version = "0.8.0", optional = true }

# The `wasm-bindgen` version is pinned (`=`) because it must be a version
# available in Nixpkgs.
wasm-bindgen = { version = "=0.2.83", optional = true, features = ["serde-serialize"] }
serde-wasm-bindgen = "0.5.0"
js-sys = { version = "0.3", optional = true }
serde_repr = { version = "0.1", optional = true }
pretty = "0.11.3"

comrak = { version = "0.17.0", optional = true, features = [] }
once_cell = "1.17.1"
typed-arena = "2.0.2"
malachite = {version = "0.3.2", features = ["enable_serde"] }
malachite-q = "0.3.2"
indexmap = {version = "1.9.3", features = ["serde"] }
strip-ansi-escapes = "0.1.1"

[dev-dependencies]
pretty_assertions = "1.3.0"
assert_matches = "1.5.0"
criterion = "0.4"
pprof = { version = "0.11.1", features = ["criterion", "flamegraph"] }
nickel-lang-utilities = {path = "utilities", version = "1.0.0"}
similar = "2.2.1"
test-generator = "0.3.1"
insta = { version = "1.28.0", features = ["filters"] }

[workspace]
members = [
".",
"nickel-lang-lib",
"nickel-lang-cli",
"lsp/nls",
"lsp/lsp-harness",
"utilities",
"nickel-wasm-repl",
"pyckel",
]

# Enable this to use flamegraphs
# [profile.release]
# debug = true

[[bench]]
name = "numeric"
harness = false

[[bench]]
name = "functions"
harness = false

[[bench]]
name = "arrays"
harness = false

# [[bench]]
# name = "records"
# harness = false

[[bench]]
name = "serialization"
harness = false

[[bench]]
name = "mantis"
harness = false

[[bench]]
name = "stdlib"
harness = false

[[bench]]
name = "typecheck-nixpkgs-lib"
harness = false
[workspace.package]
version = "1.0.0"
authors = ["The Nickel Team <nickel-lang@protonmail.com>"]
license = "MIT"
edition = "2021"
keywords = ["configuration", "language", "nix", "nickel"]
repository = "https://github.com/tweag/nickel"
homepage = "https://nickel-lang.org"
95 changes: 35 additions & 60 deletions HACKING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,12 @@ long.

## Content

The Nickel repository consist in 3 crates:
The Nickel repository consist of 4 crates:

- `nickel-lang` (path: `.`). The main crate containing the interpreter as a
library as well as the `nickel` binary.
- `nickel-lang-lsp` (path: `lsp/nls/`). the Nickel Language Server (NLS), an LSP
- `nickel-lang-lib` (path: `nickel-lang-lib`). The main crate containing the interpreter
as a library.
- `nickel-lang` (path: `nickel-lang-cli`). The `nickel` binary.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, so the path is nickel-lang-cli, but the crate name will be nickel-lang ? Hmm...that sounds a bit confusing. On the other hand, I agree that nickel-lang is a bit misleading.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point, I had the CLI just under cli, like the utilities subdirectory now. I don't think there's any paritcular necessity to tie the directory name to the crate name, and just nickel-lang felt a bit misleading as you say.

- `nickel-lang-lsp` (path: `lsp/nls/`). The Nickel Language Server (NLS), an LSP
server for Nickel.
- `nickel-lang-utilities`: (path: `utilities/`). An auxiliary crate regrouping
helpers for tests and benchmarks. Not required to build `nickel` itself.
Expand Down Expand Up @@ -57,10 +58,16 @@ nickel-lang-lsp 0.1.0

### Nickel

To only build the main crate `nickel-lang`, run:
To only build the main crate `nickel-lang-lib`, run:

```console
cargo build -p nickel-lang-lib
```

To build the interpreter CLI, run:

```shell
$ cargo build
$ cargo build -p nickel-lang
$ ./target/debug/nickel --version
nickel-lang 0.1.0
```
Expand All @@ -82,7 +89,7 @@ nickel-lang-lsp 0.1.0
There is a WebAssembly (WASM) version of the REPL, which is used for the online
playground on [nickel-lang.org][nickel-lang.org]. To ease the build, we use the
`nickel-repl` located in `nickel-wasm-repl`, which just wraps and re-export
the `nickel-lang` with the right settings for building to WebAssembly.
the `nickel-lang-lib` with the right settings for building to WebAssembly.

The Nix flake has also an output to do the whole build, but incremental
compilation is not as good as with direct usage of `cargo`.
Expand Down Expand Up @@ -117,71 +124,39 @@ LICENSE package.json nickel_lang_bg.js nickel_lang_bg.wasm [..]
Tests are run via `cargo test`. They are two types of tests:

- Unit tests, located directly in the corresponding module.
- Integration tests, located in the dedicated crate `tests/integration`.
- Integration tests, located in the dedicated crate `nickel-lang-lib/tests/integration`.
- Snapshot tests, located in `nickel-lang-cli/tests/smapshot`.

### Happy-path testing
### Test annotations

Tests are annotated with an expected result in a comment using TOML syntax that
must be located at the very beginning of the file. See the implementation in
`utilities/src/annotated_test.rs` for details. These annotations are also used
to mark examples, in the top-level subdirectory `examples`, with an expected
failure condition if necessary.

Tests for the happy path - i.e., valid Nickel programs which do not raise errors
are generally written in standalone Nickel files in the `tests/integration/pass`
are generally written in standalone Nickel files in the `nickel-lang-lib/tests/integration/pass`
directory. All `.ncl` files in this directory are automatically converted into
Rust integration tests, which run the file and assert that no errors were
raised during evaluation.

Each of these `.ncl` files is structured as an array of `Bool` expressions, which
is ultimately passed to a `check` function defined in
`tests/integration/pass/lib/assert.ncl`. This function applies an `Assert` contract
to each value in the array, which checks that the value it is applied to evaluates
to `true`. The benefit of using a contract for this is that if a test fails we
can simply run the file directly using Nickel, which gives better error messages
than the ones we get by default from `cargo test`.

### Testing failures

Tests which are expected to fail are predominantly written in Rust, because that
makes it possible to additionally check which particular error was raised. For
example, ([`tests/integration/records_fail.rs`](./tests/integration/records_fail.rs)):

```rust
#[test]
fn non_mergeable() {
assert_matches!(
eval("({a=1} & {a=2}).a"),
Err(Error::EvalError(EvalError::MergeIncompatibleArgs(..)))
);
assert_matches!(
eval("({a | default = false} & {a | default = true}).a"),
Err(Error::EvalError(EvalError::MergeIncompatibleArgs(..)))
);
}
```
`nickel-lang-lib/tests/integration/pass/lib/assert.ncl`.
This function applies an `Assert` contract to each value in the array, which
checks that the value it is applied to evaluates to `true`. The benefit of using
a contract for this is that if a test fails we can simply run the file directly
using Nickel, which gives better error messages than the ones we get by default
from `cargo test`.

The exception to the above is the suite of tests which check that the programs in
`examples/` behave as expected. These tests are also auto-generated from the example
programs, but since not every example is expected to pass, these files are annotated
with a comment declaring the expected behaviour. For example
([`examples/simple-contracts/simple-contract-div.ncl`](./examples/simple-contracts/simple-contract-div.ncl)):

```nickel
# test: blame

let Even = fun label value =>
if builtin.is_number value && value % 2 == 0 then
value
else
contract.blame label in
let DivBy3 = fun label value =>
if builtin.is_number value && value % 3 == 0 then
value
else
contract.blame label in
# Will cause an error! 4 is not divisible by 3.
(4 | Even
| DivBy3)
```
Tests which are expected to fail may be written in Rust in `nickel-lang-lib/tests/integration`.
However, simple failure test cases can make use of the test annotation support
and are located in `nickel-lang-lib/tests/integration/fail`.

### Snapshot testing

The project also contains a suite of snapshot tests in the `tests/snapshot`
The project also contains a suite of snapshot tests in the `nickel-lang-cli/tests/snapshot`
directory. Here, `.ncl` files written in the subdirectories of the `input`
directory are run against the last-built Nickel binary, and their output is
compared to the last-known output.
Expand All @@ -190,7 +165,7 @@ Failures of these tests do not necessarily mean that anything is wrong. Rather
it should be seen as an opportunity to review the diffs and either accept
any changes, or fix any issues introduced.

See the [README] in the snapshot testing crate for more detailed guides on
See `README.md` in the snapshot testing crate for more detailed guides on
working with snapshot tests.

## Benchmarking
Expand Down
4 changes: 2 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ Please follow the getting started guide for Nickel users on the [nickel-lang
website](https://nickel-lang.org/getting-started). The instructions below are
either reproduced for this document to be self-contained or because
they are aimed toward hacking on the Nickel interpreter itself (e.g. building
the `nickel-lang` crate documentation).
the `nickel-lang-lib` crate documentation).

### Run

Expand All @@ -91,7 +91,7 @@ the `nickel-lang` crate documentation).
`nix profile add github:tweag/nickel`. The `nickel` command is then in your
`$PATH` and is available anywhere.
- Without Nix, you can use `cargo run` after [building](#build), passing
arguments with an extra `--` as in `cargo run -- -f program.ncl`.
arguments with an extra `--` as in `cargo run --bin nickel -- -f program.ncl`.

2. Run your first program:

Expand Down
Loading