Skip to content

Commit e319864

Browse files
committed
Improve test setup
1 parent bd6a0fd commit e319864

12 files changed

+109
-47
lines changed

.devcontainer/devcontainer.json

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
{
2+
"name": "Rust & Microsoft SQL Server",
3+
// "build": {
4+
// "dockerfile": "Dockerfile"
5+
// },
6+
"dockerComposeFile": "../docker-compose.yml",
7+
"service": "dev",
8+
"workspaceFolder": "/workspace",
9+
"runArgs": [ "--cap-add=SYS_PTRACE", "--security-opt", "seccomp=unconfined" ],
10+
11+
// Set *default* container specific settings.json values on container create.
12+
"settings": {
13+
"terminal.integrated.shell.linux": "/bin/bash",
14+
"lldb.executable": "/usr/bin/lldb",
15+
// VS Code don't watch files under ./target
16+
"files.watcherExclude": {
17+
"**/target/**": true
18+
}
19+
},
20+
21+
// Add the IDs of extensions you want installed when the container is created.
22+
"extensions": [
23+
"bungcip.better-toml",
24+
"vadimcn.vscode-lldb",
25+
"mutantdino.resourcemonitor",
26+
"matklad.rust-analyzer"
27+
],
28+
29+
// Use 'forwardPorts' to make a list of ports inside the container available locally.
30+
// "forwardPorts": [],
31+
32+
// Use 'postCreateCommand' to run commands after the container is created.
33+
// "postCreateCommand": "rustc --version",
34+
35+
// Comment out connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
36+
"remoteUser": "vscode"
37+
}

Contributing.md

+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Contributions
2+
3+
Wether they be in code, intersting feature suggestions, design critique or bug reports, all contributions are welocme. Please start an issue, before investing a lot of work. This helps avoid situations there I would feel the need to reject a large body of work, and a lot of your time has been wasted. `odbc-parquet` is a pet project and a work of love, which implies that I maintain it in my spare time. Please understand that I may not always react immediatly. If you contribute code to fix a Bug, please also contribute the test to fix it. Happy contributing.
4+
5+
## Local build and test setup
6+
7+
Running local tests currently requires:
8+
9+
* Docker and Docker compose.
10+
11+
### Visual Studio Code
12+
13+
Should you use Visual Studio Code with the Remote Developmont extension, it will pick up the `.devcontainer` configuration and everything should be setup for you.
14+
15+
### Not Visual Studio Code
16+
17+
With docker and the SQL Driver installed run:
18+
19+
```shell
20+
docker-compose up
21+
```
22+
23+
This starts two containers called `odbc2parquet_dev` and `odbc2parquet_mssql`. You can use the `dev` container to build your code and execute tests in case you do not want to install the required ODBC drivers and/or Rust toolchain on your local machine.
24+
25+
Otherwise you can install these requirements from here:
26+
27+
* Install Rust compiler and Cargo. Follow the instructions on [this site](https://www.rust-lang.org/en-US/install.html).
28+
* [Microsoft ODBC Driver 17 for SQL Server](https://docs.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server?view=sql-server-ver15).
29+
* <https://rustup.rs/>
30+
31+
### Workaround
32+
33+
Have not figuered out, how to get the nighly toolchain during Docker setup yet. Meanwhile you must use `rustup toolchain install nightly` and `cargo +nightly install parquet` to install the missing pieces for the development setup on the dev container.
34+
35+
36+
We now can execute the tests in Rust typical fashion using:
37+
38+
```
39+
cargo test
40+
```
41+
42+
to run all tests in the workspace, which should now succeed.

README.md

-8
Original file line numberDiff line numberDiff line change
@@ -60,11 +60,3 @@ odbc2parquet --connection-string "Driver={ODBC Driver 17 for SQL Server};Server=
6060
```
6161

6262
Use `odbc2parquet --help` to see all option.
63-
64-
## Local test setup
65-
66-
Executing `cargo test` requires
67-
68-
* the Microsoft Excel ODBC Drivers.
69-
* the MS SQL Database started with `docker-compose up`
70-
* `cargo install parquet` for the parquet binaries `parquet-read` and `parquet-schema`.

docker-compose.yml

+17-4
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,17 @@
1-
mssql:
2-
build: docker/
3-
ports:
4-
- 1433:1433
1+
services:
2+
3+
mssql:
4+
build: docker/mssql
5+
ports:
6+
- 1433:1433
7+
8+
dev:
9+
build: docker/dev
10+
volumes:
11+
- .:/workspace:cached
12+
13+
# Overrides default command so things don't shut down after the process ends.
14+
command: sleep infinity
15+
16+
# Runs app on the same network as the database container, allows "forwardPorts" in devcontainer.json function.
17+
network_mode: service:mssql

docker/dev/Dockerfile

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# See here for image contents: https://github.com/microsoft/vscode-dev-containers/tree/v0.148.0/containers/rust/.devcontainer/base.Dockerfile
2+
3+
FROM mcr.microsoft.com/vscode/devcontainers/rust:0-1
4+
5+
# Install Microsoft ODBC SQL Drivers (msodbcsql17 package) for Debian 10
6+
# https://docs.microsoft.com/de-de/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver15
7+
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
8+
RUN curl https://packages.microsoft.com/config/debian/10/prod.list > /etc/apt/sources.list.d/mssql-release.list
9+
RUN echo msodbcsql17 msodbcsql/ACCEPT_EULA boolean true | debconf-set-selections
10+
11+
# [Optional] Uncomment this section to install additional packages.
12+
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
13+
&& apt-get -y install --no-install-recommends unixodbc-dev msodbcsql17
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.

tests/lib.rs

-35
Original file line numberDiff line numberDiff line change
@@ -2,40 +2,6 @@ use assert_cmd::Command;
22
use predicates::ord::eq;
33
use tempfile::tempdir;
44

5-
#[test]
6-
fn test_xls_table() {
7-
let expected = "\
8-
{Text: \"Hello\", Real: 1.3, Boolean: true}\n\
9-
{Text: \"World\", Real: 5.4, Boolean: false}\n\
10-
";
11-
12-
// A temporary directory, to be removed at the end of the test.
13-
let out_dir = tempdir().unwrap();
14-
// The name of the output parquet file we are going to write. Since it is in a temporary
15-
// directory it will not outlive the end of the test.
16-
let out_path = out_dir.path().join("out.par");
17-
// We need to pass the output path as a string argument.
18-
let out_str = out_path.to_str().expect("Tempfile path must be utf8");
19-
20-
let mut cmd = Command::cargo_bin("odbc2parquet").unwrap();
21-
cmd.args(&[
22-
"-vvvv",
23-
"-c",
24-
// See: https://www.connectionstrings.com/microsoft-excel-odbc-driver/
25-
"Driver={Microsoft Excel Driver (*.xls, *.xlsx, *.xlsm, *.xlsb)};Dbq=tests/test-table.xlsx;",
26-
out_str,
27-
"SELECT * FROM [sheet1$]",
28-
])
29-
.assert()
30-
.success();
31-
32-
// Use the parquet-read tool to verify the output. It can be installed with
33-
// `cargo install parquet`.
34-
let mut cmd = Command::new("parquet-read");
35-
cmd.arg(out_str).assert().success().stdout(eq(expected));
36-
}
37-
38-
/// Currently this test requires the docker setup from `odbc-api` to run.
395
#[test]
406
fn nullable_parquet_buffers() {
417
let expected = "\
@@ -87,7 +53,6 @@ fn foobar_connection_string() {
8753
.code(1);
8854
}
8955

90-
/// Currently this test requires the docker setup from `odbc-api` to run.
9156
#[test]
9257
fn parameters_in_query() {
9358
let expected = "\

tests/test-table.xlsx

-4.78 KB
Binary file not shown.

tests/test.accdb

-756 KB
Binary file not shown.

0 commit comments

Comments
 (0)