chore: faster build install (#655)

* [ ] I have added tests to `tests` to cover my changes.
* [x] I have updated `docs/`, if necessary.
* [ ] I have updated the `README.md`, if necessary.

***What existing issue does this pull request close?***

No issue 🎩 👉 🐇 

***How are these changes tested?***

locally and ci

***Provide additional context.***

- build system: setuptools for consistency with other projects,
simplicity, and stability
- consolidated all? configs to pyproject.toml
- replaced bash scripts with invoke to be more pythonic 🐍

.github/workflows/docs.yaml

@@ -5,34 +5,41 @@ on:
     branches:
       - main
 
-env:
-  GALILEO_API_URL: "http://localhost:8000"
-
 jobs:
   test:
     runs-on: ubuntu-latest
+
     strategy:
       matrix:
         python-version: [3.8]
       fail-fast: false
 
     steps:
       - uses: actions/checkout@v3
+
       - name: set up python
         uses: actions/setup-python@v4
         with:
+          cache: "pip"
+          cache-dependency-path: "pyproject.toml"
           python-version: ${{ matrix.python-version }}
+
+      - name: install invoke
+        run: pip install invoke
+
       - name: install dependencies
-        run: scripts/install.sh
+        run: inv install
+
       - name: docs
-        run: scripts/docs-build.sh
+        run: inv docs-build
+
       - name: Pushes to another repository
         uses: cpina/github-action-push-to-another-repository@main
         env:
           API_TOKEN_GITHUB: ${{ secrets.GH_TOKEN }}
         with:
-          source-directory: 'docs/autodocs/_build/markdown'
-          target-directory: 'autodocs'
-          destination-github-username: 'rungalileo'
-          destination-repository-name: 'docs'
-          user-email: team@rungalileo.io
+          source-directory: "docs/autodocs/_build/markdown"
+          target-directory: "autodocs"
+          destination-github-username: "rungalileo"
+          destination-repository-name: "docs"
+          user-email: team@rungalileo.io

.github/workflows/publish.yaml

@@ -6,22 +6,37 @@ on:
       - created
 
 jobs:
-  publish:
+  main:
     runs-on: ubuntu-latest
+
+    strategy:
+      matrix:
+        python-version: ["3.9"]
+
     steps:
       - name: checkout
         uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+
+      - name: apt-get update
+        run: sudo apt-get update -y
 
       - name: set up python
         uses: actions/setup-python@v4
         with:
-          python-version: "3.9"
+          python-version: ${{ matrix.python-version }}
+          cache: "pip"
+          cache-dependency-path: "pyproject.toml"
+
+      - name: install invoke
+        run: pip install invoke
 
-      - name: install flit
-        run: pip install flit
+      - name: build
+        run: inv build
 
-      - name: publish to pypi
+      - name: publish
         env:
-          FLIT_USERNAME: ${{ secrets.FLIT_USERNAME }}
-          FLIT_PASSWORD: ${{ secrets.FLIT_PASSWORD }}
-        run: scripts/publish.sh
+          TWINE_USERNAME: ${{ secrets.FLIT_USERNAME }}
+          TWINE_PASSWORD: ${{ secrets.FLIT_PASSWORD }}
+        run: inv publish

.github/workflows/test.yaml

@@ -7,32 +7,39 @@ on:
   pull_request:
     types: [opened, synchronize]
 
-env:
-  GALILEO_API_URL: "http://localhost:8000"
-
 jobs:
   test:
     runs-on: galileo
+
     strategy:
       matrix:
-        python-version: [3.7, 3.8, 3.9]
+        python-version: ["3.7", "3.8", "3.9"]
       fail-fast: false
 
     steps:
       - uses: actions/checkout@v3
+
       - name: set up python
         uses: actions/setup-python@v4
         with:
+          cache: "pip"
+          cache-dependency-path: "pyproject.toml"
           python-version: ${{ matrix.python-version }}
+
+      - name: install invoke
+        run: pip install invoke
+
       - name: install dependencies
         run: |
           if [[ ${{ matrix.python-version }} == "3.7" ]]; then
-            scripts/install.sh --extras=py37
+            inv install --no-editable --extras dev,test,doc,py37
           else
-            scripts/install.sh
+            inv install --no-editable
           fi
+
       - name: test
-        run: scripts/test.sh
+        run: inv test
+
       - name: upload coverage
         uses: codecov/codecov-action@v1
         with:

CONTRIBUTING.md

@@ -26,35 +26,26 @@ $ which python pip
 
 [`pyenv`](https://github.com/pyenv/pyenv) is suggested for local python development.
 
-## Flit
-
-This project uses `flit` to manage our project's dependencies.
-
-After activating the environment as described above, install flit:
+## Development
 
-```sh
-$ pip install flit
-```
+This project uses `setuptools` to manage our project's dependencies and `invoke` for running tasks.
 
-Install dependencies
+After activating the environment as described above, install `invoke` and setup your local environment:
 
 ```sh
-./scripts/install.sh
+pip install --upgrade pip
+pip install invoke
+inv all
 ```
 
-## Development
-Developing with Flit is simple. Symlink this repo to your venv so you can make changes
-and test them without reinstalling the package.
+To install a development version of the package in another python environment, simply use that virtual enviroment's pip:
 
-Run the following from the root of dataquality:
 ```sh
-flit install -s
+pip install -e .
 ```
 
-You can specify which python environment to install into using the `--python` flag
-(useful for developing/testing from external venvs)
-
 ### Debugging
+
 If you're looking to debug some code in dataquality, for example with `pdb` in jupyter,
 you can do that easily:
 1. Install and symlink `dataquality` as shown above
@@ -66,7 +57,7 @@ you can do that easily:
 ## Formatting
 
 ```sh
-./scripts/format.sh
+inv format
 ```
 
 ## Tests
@@ -76,13 +67,13 @@ You will need to have a local cluster running, read our [API documentation](http
 Set some local env variables
 
 ```
-. ./scripts/set-local-env.sh
+GALILEO_CONSOLE_URL="http://localhost:3000"
 ```
 
 Now run your tests!
 
 ```sh
-./scripts/test-cov-html.sh
+inv test
 ```
 
 
@@ -91,7 +82,7 @@ Now run your tests!
 Run this from this project's root directory to boot up tests.
 
 ```
-. ./scripts/set-local-env.sh; jupyter notebook docs/
+GALILEO_CONSOLE_URL="http://localhost:3000" jupyter notebook docs/
 ```
 
 
@@ -100,7 +91,7 @@ Run this from this project's root directory to boot up tests.
 Everything is done through github actions. Make sure to bump the version of the package
 
 ```
-./scripts/bump-version.sh
+inv update-version-number
 ```
 
 commit the change and publish a new version.