Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reworking the general flow of the README #71

Merged
merged 51 commits into from
Jun 10, 2020
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
4710ea4
initial stab at reworking the general flow of the README
jspaaks Jun 8, 2020
aab0c4e
added outline for README
jspaaks Jun 8, 2020
8e18b30
Merge remote-tracking branch 'origin/master' into 66-readme-flow
jspaaks Jun 8, 2020
6d03ab6
Merge remote-tracking branch 'origin/master' into 66-readme-flow
jspaaks Jun 9, 2020
67136cf
moved files to new cli/ directory; updated references to it
jspaaks Jun 9, 2020
0d2c606
updated Makefile/INSTALL
jspaaks Jun 9, 2020
e176c10
updated gitignore
jspaaks Jun 9, 2020
905cada
split workflow job cpp into cli and cgi
jspaaks Jun 9, 2020
f7e6109
moved src/py things to openapi/ and flask/
jspaaks Jun 9, 2020
8cad0e1
split the python gh action into openapi and flask
jspaaks Jun 9, 2020
765daf1
clearer gh action job names
jspaaks Jun 9, 2020
add333d
removed openapi stuff from flask job in gh action
jspaaks Jun 9, 2020
8509ba7
new directory structure
jspaaks Jun 9, 2020
aa46350
updated Makefile
jspaaks Jun 9, 2020
7f2d891
added sleeps to gh actions to allow services to be up in time
jspaaks Jun 9, 2020
a823277
test workflow
jspaaks Jun 9, 2020
8aeecfc
linking to shared library is now a makefile task
jspaaks Jun 9, 2020
79b0782
added serving from flask directory for testing
jspaaks Jun 9, 2020
7a8968e
change makefile task name
jspaaks Jun 9, 2020
b5259aa
python webapp tasks now depend on linked .so file, while web service …
jspaaks Jun 9, 2020
6af82f1
Update README.outline.md
jspaaks Jun 9, 2020
4bab7ab
Update README.outline.md
jspaaks Jun 9, 2020
771f4df
Update README.outline.md
jspaaks Jun 9, 2020
919e425
Update README.outline.md
jspaaks Jun 9, 2020
3c24dbe
forgot to update the workflow
jspaaks Jun 9, 2020
b7a8347
now using python extension suffix in the softlink
jspaaks Jun 9, 2020
b53c9b0
updated paths in a few spec files
jspaaks Jun 9, 2020
cddd550
updated tests to point to the react/ directory
jspaaks Jun 9, 2020
68b5bb1
updated tests to point to the react/ directory
jspaaks Jun 9, 2020
cb1ec4c
removed duplicate files from src
jspaaks Jun 9, 2020
2ab618e
updated host-files task
jspaaks Jun 9, 2020
1b93061
added linking of wasm file
jspaaks Jun 9, 2020
e553fc9
webassembly should work; react almost works
jspaaks Jun 9, 2020
9548718
added missing file but is now duplicated
jspaaks Jun 9, 2020
765e968
Dont ignore wasm files, they are needed to online examples
sverhoeven Jun 10, 2020
df77d2f
Break connection between tangled files and compiled files
sverhoeven Jun 10, 2020
32c63f8
Shorten js test names + update phony make targets + host-files make t…
sverhoeven Jun 10, 2020
5e63e9a
Merge remote-tracking branch 'origin/master' into 66-readme-flow
sverhoeven Jun 10, 2020
532ca80
celery is needed in flask chapter
sverhoeven Jun 10, 2020
14de6f1
One more src/js -> webassembly
sverhoeven Jun 10, 2020
c8e3007
Update sonarcloud exclude
sverhoeven Jun 10, 2020
d93a2c7
Unique names for artifacts of each job
sverhoeven Jun 10, 2020
872a3a2
Document react/worker.js
sverhoeven Jun 10, 2020
12e5e7d
Dont check react/worker.js for dups
sverhoeven Jun 10, 2020
4283086
Updated CHANGELOG + also clean flask/newtonraphsonpy.*.so on `make cl…
sverhoeven Jun 10, 2020
5e181db
Correct apache command
sverhoeven Jun 10, 2020
67e855f
Updated CONTRIBUTING.md
sverhoeven Jun 10, 2020
9637562
Remove `react/newtonraphsonwasm*` on `make clean-compiled`
sverhoeven Jun 10, 2020
e4b1096
Use chapter dir name in anchors
sverhoeven Jun 10, 2020
88fe8ac
Updated README.outline.md
sverhoeven Jun 10, 2020
c81f7ca
Updated CHANGELOG
sverhoeven Jun 10, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 90 additions & 18 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
- uses: actions/checkout@v2
- name: Check all entangled files are in sync with Markdown
run: make check
cpp:
cli:
# The type of runner that the job will run on
runs-on: ubuntu-latest

Expand All @@ -23,11 +23,62 @@ jobs:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2

- name: Run C++ examples
run: make test-cli test-cgi
python:
- name: Run C++ test for CLI
run: make test-cli
cgi:
# The type of runner that the job will run on
name: Python ${{ matrix.python-version }}
runs-on: ubuntu-latest

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2

- name: Run C++ test for CGI
run: make test-cgi
openapi:
# The type of runner that the job will run on
name: openapi / python-${{ matrix.python-version }}
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.6, 3.7, 3.8]
fail-fast: true
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python-version }}
architecture: "x64"

- name: Which Python
run: which python

- name: Install Python dependencies
run: make pip-pybind11 && make pip-connexion

- name: Run Python example
run: make test-py

- name: Start web service in background
run: |
make run-webservice 2>&1 | tee ./run-webservice.log &
sleep 2
- name: Test web service
run: make test-webservice

- name: Upload log of services
if: ${{ always() }}
uses: actions/upload-artifact@v2
with:
name: service-logs
path: ./run-*.log
flask:
# The type of runner that the job will run on
name: flask / python-${{ matrix.python-version }}
runs-on: ubuntu-latest
strategy:
matrix:
Expand All @@ -53,29 +104,27 @@ jobs:
run: which python

- name: Install Python dependencies
run: make py-deps && pip install httpie
run: make pip-pybind11 && make pip-flask && make pip-celery && pip install httpie

- name: Run Python example
run: make test-py
- name: Build shared library and link to it
run: |
make flask-link-newtonraphsonpy

- name: Start web application in background
run: make run-webapp 2>&1 | tee ./run-webapp.log &
run: |
make run-webapp 2>&1 | tee ./run-webapp.log &
sleep 2

- name: Test web application
run: http --ignore-stdin -f localhost:5001 epsilon=0.001 guess=-20

- name: Start web service in background
run: make run-webservice 2>&1 | tee ./run-webservice.log &

- name: Test web service
run: make test-webservice

- name: Start Celery web app in background
run: make run-celery-webapp 2>&1 | tee ./run-celery-webapp.log &
run: |
make run-celery-webapp 2>&1 | tee ./run-celery-webapp.log &

- name: Start Celery worker in background
run: |
cd src/py
cd flask
PYTHONPATH=$PWD/../.. celery -A tasks worker 2>&1 | tee ./run-celery-worker.log &
cd ../..

Expand All @@ -93,7 +142,30 @@ jobs:
with:
name: service-logs
path: ./run-*.log
wasm:
webassembly:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2

- name: Install emscripten
uses: mymindstorm/setup-emsdk@v4

- name: Build WebAssembly module
run: make build-wasm

- name: Start web server for hosting files in background
run: make host-files 2>&1 | tee ./web-server.log &

- name: Run tests
run: make test-wasm

- name: Upload log of web server
if: ${{ always() }}
uses: actions/upload-artifact@v2
with:
name: web-server-log
path: ./web-server.log
react:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
Expand Down
11 changes: 8 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,19 @@ __pycache__/
cypress/plugins
cypress/support
cypress/videos
/cypress/screenshots

# Ignore compiled files
bin/newtonraphson.exe
src/py/newtonraphsonpy.*.so
apache2/cgi-bin/newtonraphson
/cli/newtonraphson.exe
/cgi/apache2/cgi-bin/newtonraphson
/openapi/newtonraphsonpy.*.so
/flask/newtonraphsonpy.*.so
/webassembly/newtonraphsonwasm.js
/webassembly/newtonraphsonwasm.wasm

# Ignore entangled db
.entangled/db.sqlite

# other ignores
node_modules

29 changes: 16 additions & 13 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ All the commands in the [README.md](README.md) and [CONTRIBUTING.md](CONTRIBUTIN
UID := $(shell id -u)
# Prevent suicide by excluding Makefile
ENTANGLED := $(shell perl -ne 'print $$1,"\n" if /^```\{.*file=(.*)\}/' *.md | grep -v Makefile | sort -u)
COMPILED := bin/newtonraphson.exe src/py/newtonraphsonpy.*.so apache2/cgi-bin/newtonraphson src/js/newtonraphsonwasm.js src/js/newtonraphsonwasm.wasm
COMPILED := cli/newtonraphson.exe openapi/newtonraphsonpy.*.so cgi/apache2/cgi-bin/newtonraphson webassembly/newtonraphsonwasm.js webassembly/newtonraphsonwasm.wasm

entangle: *.md
<<pandoc-tangle>>
Expand All @@ -46,22 +46,25 @@ pip-celery:
pip-connexion:
<<pip-connexion>>

bin/newtonraphson.exe: src/cli-newtonraphson.cpp
cli/newtonraphson.exe: cli/cli-newtonraphson.cpp
<<build-cli>>

test-cli: bin/newtonraphson.exe
test-cli: cli/newtonraphson.exe
<<test-cli>>

apache2/cgi-bin/newtonraphson: src/cgi-newtonraphson.cpp
cgi/apache2/cgi-bin/newtonraphson: cgi/cgi-newtonraphson.cpp
<<build-cgi>>

test-cgi: apache2/cgi-bin/newtonraphson
test-cgi: cgi/apache2/cgi-bin/newtonraphson
<<test-cgi>>

src/py/newtonraphsonpy.*.so: src/py-newtonraphson.cpp
openapi/newtonraphsonpy.*.so: openapi/py-newtonraphson.cpp
<<build-py>>

test-py: src/py/example.py src/py/newtonraphsonpy.*.so
flask-link-newtonraphsonpy: openapi/newtonraphsonpy.*.so
<<flask-link-newtonraphsonpy>>

test-py: openapi/example.py openapi/newtonraphsonpy.*.so
<<test-py>>

test: test-cli test-cgi test-py test-webservice
Expand All @@ -84,24 +87,24 @@ start-redis:
stop-redis:
<<stop-redis>>

run-webapp: src/py/newtonraphsonpy.*.so
run-webapp: openapi/newtonraphsonpy.*.so
<<run-webapp>>

run-webservice: src/py/newtonraphsonpy.*.so
run-webservice: openapi/newtonraphsonpy.*.so
<<run-webservice>>

test-webservice:
<<test-webservice>>

run-celery-worker: src/py/newtonraphsonpy.*.so
run-celery-worker: openapi/newtonraphsonpy.*.so
<<run-celery-worker>>

run-celery-webapp: src/py/newtonraphsonpy.*.so
run-celery-webapp: openapi/newtonraphsonpy.*.so
<<run-celery-webapp>>

build-wasm: src/js/newtonraphsonwasm.js src/js/newtonraphsonwasm.wasm
build-wasm: webassembly/newtonraphsonwasm.js webassembly/newtonraphsonwasm.wasm

src/js/newtonraphsonwasm.js src/js/newtonraphsonwasm.wasm: src/wasm-newtonraphson.cpp
webassembly/newtonraphsonwasm.js webassembly/newtonraphsonwasm.wasm: webassembly/wasm-newtonraphson.cpp
<<build-wasm>>

host-files: build-wasm
Expand Down
55 changes: 29 additions & 26 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
UID := $(shell id -u)
# Prevent suicide by excluding Makefile
ENTANGLED := $(shell perl -ne 'print $$1,"\n" if /^```\{.*file=(.*)\}/' *.md | grep -v Makefile | sort -u)
COMPILED := bin/newtonraphson.exe src/py/newtonraphsonpy.*.so apache2/cgi-bin/newtonraphson src/js/newtonraphsonwasm.js src/js/newtonraphsonwasm.wasm
COMPILED := cli/newtonraphson.exe openapi/newtonraphsonpy.*.so cgi/apache2/cgi-bin/newtonraphson webassembly/newtonraphsonwasm.js webassembly/newtonraphsonwasm.wasm

entangle: *.md
docker run --rm --user ${UID} -v ${PWD}:/data nlesc/pandoc-tangle:0.5.0 --preserve-tabs *.md
Expand All @@ -28,24 +28,27 @@ pip-celery:
pip-connexion:
pip install connexion[swagger-ui]

bin/newtonraphson.exe: src/cli-newtonraphson.cpp
g++ src/cli-newtonraphson.cpp -o bin/newtonraphson.exe
cli/newtonraphson.exe: cli/cli-newtonraphson.cpp
g++ cli/cli-newtonraphson.cpp -o cli/newtonraphson.exe

test-cli: bin/newtonraphson.exe
./bin/newtonraphson.exe
test-cli: cli/newtonraphson.exe
./cli/newtonraphson.exe

apache2/cgi-bin/newtonraphson: src/cgi-newtonraphson.cpp
g++ -Ideps src/cgi-newtonraphson.cpp -o apache2/cgi-bin/newtonraphson
cgi/apache2/cgi-bin/newtonraphson: cgi/cgi-newtonraphson.cpp
g++ -Icgi/deps/ -Icli/ cgi/cgi-newtonraphson.cpp -o cgi/apache2/cgi-bin/newtonraphson

test-cgi: apache2/cgi-bin/newtonraphson
echo '{"guess":-20, "epsilon":0.001}' | apache2/cgi-bin/newtonraphson
test-cgi: cgi/apache2/cgi-bin/newtonraphson
echo '{"guess":-20, "epsilon":0.001}' | cgi/apache2/cgi-bin/newtonraphson

src/py/newtonraphsonpy.*.so: src/py-newtonraphson.cpp
g++ -O3 -Wall -shared -std=c++14 -fPIC `python3 -m pybind11 --includes` \
src/py-newtonraphson.cpp -o src/py/newtonraphsonpy`python3-config --extension-suffix`
openapi/newtonraphsonpy.*.so: openapi/py-newtonraphson.cpp
g++ -O3 -Wall -shared -std=c++14 -fPIC -Icli/ `python3 -m pybind11 --includes` \
openapi/py-newtonraphson.cpp -o openapi/newtonraphsonpy`python3-config --extension-suffix`

test-py: src/py/example.py src/py/newtonraphsonpy.*.so
python src/py/example.py
flask-link-newtonraphsonpy: openapi/newtonraphsonpy.*.so
cd flask && ln -s openapi/newtonraphsonpy.*.so && cd -

test-py: openapi/example.py openapi/newtonraphsonpy.*.so
python openapi/example.py

test: test-cli test-cgi test-py test-webservice

Expand All @@ -67,28 +70,28 @@ start-redis:
stop-redis:
docker stop some-redis

run-webapp: src/py/newtonraphsonpy.*.so
python src/py/webapp.py
run-webapp: openapi/newtonraphsonpy.*.so
python flask/webapp.py

run-webservice: src/py/newtonraphsonpy.*.so
python src/py/webservice.py
run-webservice: openapi/newtonraphsonpy.*.so
python openapi/webservice.py

test-webservice:
curl -X POST "http://localhost:8080/api/newtonraphson" -H "accept: application/json" -H "Content-Type: application/json" -d "{\"epsilon\":0.001,\"guess\":-20}"

run-celery-worker: src/py/newtonraphsonpy.*.so
PYTHONPATH=src/py celery worker -A tasks
run-celery-worker: openapi/newtonraphsonpy.*.so
PYTHONPATH=openapi celery worker -A tasks

run-celery-webapp: src/py/newtonraphsonpy.*.so
python src/py/webapp-celery.py
run-celery-webapp: openapi/newtonraphsonpy.*.so
python flask/webapp-celery.py

build-wasm: src/js/newtonraphsonwasm.js src/js/newtonraphsonwasm.wasm
build-wasm: webassembly/newtonraphsonwasm.js webassembly/newtonraphsonwasm.wasm

src/js/newtonraphsonwasm.js src/js/newtonraphsonwasm.wasm: src/wasm-newtonraphson.cpp
emcc --bind -o src/js/newtonraphsonwasm.js -s MODULARIZE=1 -s EXPORT_NAME=createModule src/wasm-newtonraphson.cpp
webassembly/newtonraphsonwasm.js webassembly/newtonraphsonwasm.wasm: webassembly/wasm-newtonraphson.cpp
emcc -Icli/ --bind -o webassembly/newtonraphsonwasm.js -s MODULARIZE=1 -s EXPORT_NAME=createModule webassembly/wasm-newtonraphson.cpp

host-files: build-wasm
python3 -m http.server 8000
cd flask && python3 -m http.server 8000 && cd -

test-wasm:
npx cypress run --config-file false
Expand Down
Loading