Skip to content

Commit

Permalink
Merge branch 'feature/add-code-linting-#165' into feature/fix-code-li…
Browse files Browse the repository at this point in the history 
…nting-#184
  • Loading branch information
@MRichards99
MRichards99 committed Dec 8, 2020
2 parents fe88196 + 0366d08 commit 1013e82
Show file tree
Hide file tree
Showing 2 changed files with 104 additions and 46 deletions.
122 changes: 81 additions & 41 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,9 @@ machine.
### pyenv (Python Version Management)
To start, install [pyenv](https://github.com/pyenv/pyenv). There is a Windows version of
this tool ([pyenv-win](https://github.com/pyenv-win/pyenv-win)), however this is
currently untested on this repoThis is used to manage the various versions of Python
that will be used to test/lint Python during development. Install by executing the following:
currently untested on this repo. This is used to manage the various versions of Python
that will be used to test/lint Python during development. Install by executing the
following:

```bash
curl https://pyenv.run | bash
Expand Down Expand Up @@ -125,7 +126,8 @@ used to lint/format/test the code in the included `noxfile.py`. To install Nox,
as shown below. Nox is not listed as a Poetry dependency because this has the potential
to cause issues if Nox was executed inside Poetry (see
[here](https://medium.com/@cjolowicz/nox-is-a-part-of-your-global-developer-environment-like-poetry-pre-commit-pyenv-or-pipx-1cdeba9198bd)
for more detailed reasoning):
for more detailed reasoning). If you do choose to install these packages within a
virtual environment, you do not need the `--user` option:

```bash
pip install --user --upgrade nox
Expand Down Expand Up @@ -153,6 +155,21 @@ Currently, the following Nox sessions have been created:
dependencies (pulled directly from Poetry) for any known vulnerabilities. This session
gives the output in a full ASCII style report.

Each Nox session builds an environment using the repo's dependencies (defined using
Poetry) using `install_with_constraints()`. This stores the dependencies in a
`requirements.txt`-like format temporarily during this process, using the OS' default
temporary location. This could result in permissions issues (this has been seen by a
colleague on Windows), so adding the `--tmpdir [DIRECTORY PATH]` allows the user to
define where this file should be stored. Due to Nox session being initiated in the
command line, this argument needs to be a positional argument (denoted by the `--` in
the Nox command). This argument is optional, but **must** be the final argument avoid
interference with Nox's argument parsing. An example:

```bash
nox -s lint -- util datagateway_api --tmpdir /root
```


### Pre Commit (Automated Checks during Git Commit)
To make use of Git's ability to run custom hooks, [pre-commit](https://pre-commit.com/)
is used. Like Nox, Pip is used to install this tool:
Expand Down Expand Up @@ -278,45 +295,68 @@ This is illustrated below.


`````
─── datagateway_api
├── common
│ ├── database
│ │ ├── backend.py
│ │ ├── filters.py
│ │ ├── helpers.py
│ │ ├── models.py
│ │ └── session_manager.py
│ ├── icat
│ ├── backends.py
│ ├── constants.py
│ ├── exceptions.py
│ ├── filters.py
│ └── helpers.py
├── src
│ ├── resources
│ │ ├── entities
│ │ │ ├── entity_endpoint.py
│ │ │ └── entity_map.py
│ │ └── non_entities
│ │ └── <non_entity>_endpoints.py
│ ├── swagger
│ │ ├── openapi.yaml
│ │ └── swagger_generator.py
│ └── main.py
├── test
│ ├── resources
│ │ ├── entities
│ │ │ └──test_<entity>.py
│ │ └── non_entities
│ │ └── test_<non_entity>.py
│ └── test_base
│ ├── constants.py
│ └── rest_test.py
├── util
│ └── icat_db_generator.py
├── logs.log
└── config.json
.
├── .flake8
├── .gitignore
├── .pre-commit-config.yaml
├── LICENSE
├── README.md
├── config.json.example
├── datagateway_api
│ ├── common
│ │ ├── backend.py
│ │ ├── backends.py
│ │ ├── config.py
│ │ ├── constants.py
│ │ ├── database
│ │ │ ├── backend.py
│ │ │ ├── filters.py
│ │ │ ├── helpers.py
│ │ │ ├── models.py
│ │ │ └── session_manager.py
│ │ ├── date_handler.py
│ │ ├── exceptions.py
│ │ ├── filter_order_handler.py
│ │ ├── filters.py
│ │ ├── helpers.py
│ │ ├── icat
│ │ │ ├── backend.py
│ │ │ ├── filters.py
│ │ │ ├── helpers.py
│ │ │ └── query.py
│ │ └── logger_setup.py
│ └── src
│ ├── main.py
│ ├── resources
│ │ ├── entities
│ │ │ ├── entity_endpoint.py
│ │ │ └── entity_map.py
│ │ ├── non_entities
│ │ │ └── sessions_endpoints.py
│ │ └── table_endpoints
│ │ └── table_endpoints.py
│ └── swagger
│ ├── apispec_flask_restful.py
│ ├── initialise_spec.py
│ └── openapi.yaml
├── noxfile.py
├── poetry.lock
├── postman_collection_icat.json
├── pyproject.toml
├── test
│ ├── test_base.py
│ ├── test_database_helpers.py
│ ├── test_entityHelper.py
│ └── test_helpers.py
└── util
└── icat_db_generator.py
`````

The directory tree can be generated using the following command:

`git ls-tree -r --name-only HEAD | grep -v __init__.py | tree --fromfile`


#### Main
`main.py` is where the flask_restful api is set up. This is where each endpoint resource
class is generated and mapped to an endpoint.
Expand Down
28 changes: 23 additions & 5 deletions noxfile.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
code_locations = "datagateway_api", "test", "util", "noxfile.py"


def install_with_constraints(session, *args, **kwargs):
with tempfile.NamedTemporaryFile() as requirements:
def install_with_constraints(session, req_dir=None, *args, **kwargs):
with tempfile.NamedTemporaryFile(dir=req_dir) as requirements:
session.run(
"poetry",
"export",
Expand All @@ -20,18 +20,35 @@ def install_with_constraints(session, *args, **kwargs):
session.install(f"--constraint={requirements.name}", *args, **kwargs)


def get_tmp_dir(session):
tmp_dir = None

try:
if session.posargs[-2] == "--tmpdir":
tmp_dir = session.posargs.pop(-1)
session.posargs.remove("--tmpdir")
except IndexError:
session.log("Info: No --tmpdir option given")

return tmp_dir


@nox.session(python="3.6", reuse_venv=True)
def black(session):
tmp_dir = get_tmp_dir(session)
args = session.posargs or code_locations
install_with_constraints(session, "black")

install_with_constraints(session, tmp_dir, "black")
session.run("black", *args, external=True)


@nox.session(python="3.6", reuse_venv=True)
def lint(session):
tmp_dir = get_tmp_dir(session)
args = session.posargs or code_locations
install_with_constraints(
session,
tmp_dir,
"flake8",
"flake8-bandit",
"flake8-black",
Expand All @@ -49,8 +66,9 @@ def lint(session):

@nox.session(python="3.6", reuse_venv=True)
def safety(session):
install_with_constraints(session, "safety")
with tempfile.NamedTemporaryFile() as requirements:
tmp_dir = get_tmp_dir(session)
install_with_constraints(session, tmp_dir, "safety")
with tempfile.NamedTemporaryFile(dir=tmp_dir) as requirements:
session.run(
"poetry",
"export",
Expand Down

0 comments on commit 1013e82

Please sign in to comment.