Skip to content

Commit

Permalink
update master (#57)
Browse files Browse the repository at this point in the history
* [Task] #6 provide fallback index.html

* [Task] #6 production ready code (m)

move httpdocs folder to dist
have compile without sourcemaps for faster speed

* [Task] #6 create github action for upload when main is updated (#21)

* [change] #6 new ftp upload action

* [Fix] #6 replace host with server in ftp action

* [Task] #6 basic log (#26)

* [CHANGE] #6 revert back to require output for production

* [Task] #6 add ability to manually upload to prod

* [Task] #9 enable manual start of codechecks

* 10 webhook for writing (#36)

* [Change] #3 clean up npm scripts, to have clean folder before build

* [Task] #10 created data types in typescript

* [Temp] #10 created subroute for writing, and folder structure

* [Change] #3 include to use relative paths from src folder in ts and node

https://stackoverflow.com/questions/43281741/how-can-i-use-paths-in-tsconfig-json
See comment from Remo H. Hansen with at least 100 upvoted

* [Change] Update VSCode to keep files open

* [Task] #18 setup dotenv for secret variables

* [Temp, Task] #10 Validate inputs using express-validator and custom functions

* [Task] #18 prevent parameter pollution

* [Task] #10 validating incoming parameter and logging errors

* [Task] #7 add basic cache to express

* [Changes] #7 Error Handling, to include basic custom Error Handling

* [Task] #10 enhanced validation to only allow known parameters

* [Change] #35 added Jest, tests for helper functions when writing

* [Task] #10 better error Handling

* [Task] #35 add tests for writing webhook validation

* [TASK] #18 protect Webhook using KEY

* [Fix] #35 test know import path structure now

* [Task] #35 add test for protected webhook

* [Task] #35 refactor build to run jest tests

* [Task] #10 switched to crypto instead of bcrypt for dependency issue

see synk inflight

* [Fix] #36 PRQ Feedback

* [Task] #3 improve error handling, logger and added chalk

to colorize console output.
Had to use chalk version 4 because of typescript converting to require, and chalk5 do want import syntax.

* [Change] #3 nodemon to clear console when in dev mode

* [!Task] #32 webhook creates folder and file based on date

* [Change] #35 relocated tests and refactor write, also added file check

* [Task] #18, installed helmet, configured self as CSP origin

* [Fix] moved chalk out of dev dependency

* [Task] #32 error logging and text output improvement, log string instead of "object"

* [Task] #18 CSP Update to allow localhost for testing

* [Fix] #3 debugging setup improvments

* [FIX] #10 Error Handling

* [Task] #10 writing basic non calculated data to file

* [Fix] #10 avoid Header Modification after sending the request

* [Task] #10 JSON Data pretty output

* [Task] #32 update types to reflect subobjects of entry

* [Task] #10 write time

* [Task] #32 added logging for time edgecases

* [Task] #10 output seconds

* [Task] #10 calculate distance based on lat and lon

* [Task] #32 writing tests for time and distance

* [Task] #32 change distance calculation to use pythagoras

* [Task] #38 add favicon

* [Task] #32 time converted to seconds

* [Taskk] #32 speed calculation and output and tests

* [Task] #32 speed tests

* [Task] #33 add ignore

* [Task] #32 test finetuning

* [Task] #32 add angle between entries

* [Task] #32 test for angle, extracted getData function

* [change] #32 test to include optional leading 0 for days

* [!!!Task] #18 add uncaughtExeption handler as last resort

* [Task] #7 enhance static options to include common filetypes;  index file

start is used as index file to avoid collisions with host provider

* [change] #32 validation to be used more explictly

* [change] #32 add index to log while writing

* [Task] #32 test if 1000 calls can be made with randomized data

* [!!! Task] #32 limit JSON Data to be 1000 lines: replace last line with most recent entry

* [Change, Task] #32 if 1000 entries exceeded, only replace last if hdop is good

* [Change] build action enable button to on manually

* [temp] test y tests fail

* Create node.js.yml

* Create main.yml

* [!!!Fix] Created new workflow to build / test node, commented tests back in.

Increased time between server calls in test, to check difference time more accurately

* [Task] #33 moved ignore to its own file since it creates data rather than validating it

* 42 output json (#44)

* [Task] #42, created route to output json

* [Task] #42 added tests for read json

* 41 add rate limiter (#45)

* [Task] #18, limit request size for security reasons

* [Task] #43, introduce gzip to transfer data

* [Task] #34 improve error handling, log server shutdowns

* [Task] #34 installed and integrated tooBusy to send 503 when load is high

* [Task] #34 improved tooBusy, improved formatting

* [Task, Temp] #41 installed ratelimiter and slowDown

* [Task] #42 cleanup ipv6 addresses

* [Change] #10 error handling for better gitBash and txt output, also reduced stack in case of validation errors

* [Task] #41 prepare Log for RateLImit errors

* [Temp] #41 write route rateLImited

temp: see Todos

* [Task] #34 colorize prefix in console

* [Task] #42 extract middlewares and move to folder

* [Task] #41 ratelimiter cleaning up periodicly

* [Task] #41 skip tests in rateLimiting

* Bump follow-redirects from 1.15.5 to 1.15.6 (#47)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.5 to 1.15.6.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.15.5...v1.15.6)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* 43 secure output route (#46)

* [Task] #43 create color pallette

via atmos

* [Task] #43 create color pallette

via atmos

* [Task] #43 cleanup colors and svg

* [Task] #41 remove test code

* [CHANGE] #3 reconfigured nodemon to copy static files

* [Task] #18 replaced getRawBody with builtIn express urlEncoded

* [Temp, Task] #43 basic login page, not yet used as middleware

* [Temp] #43, create and validate json web token

* [Task] #43, add slowDown and RateLimit for failed login attempts

* [Task] #43, ratelimit for login page

* [Task] #43, add global ratelimiter

* [fix] #7, improve error handeling for express errors

* [Task] #43 rework body limitations to be checked only appropiate methods

* [Task] #43 added check for data before using it

* [Task] #43 check that body is ignored for GET in request

* [Task] #43 login test

* [Task] #43 create tests for login

* [Task] #43 fine tune error handling

* [Task] #43, finished login and jwt related tests

* [Change] #34, no further need for test logging

* [Task] #43, fine tune jwt, middleware process improved

* [CHANGE] #43 created new esLint to have clientside js without ts

* [Temp] #43 test to see new linter configuration

* [Change] #43 switched to bcrypt for passwords

* [Task] #43 read return json in all cases

* [Task] #43 introduced color classes

* [Task] #43, prq feedback

* [Temp} #43 figuring out why tests dont run on github

* [Task] #43 code cleanup

* 48 move login to seperate controller (#49)

* [Task] #43, add label to form

* [Task] #48 login controller

* 50 integrate csrf protection for login form (#53)

* [Task] #50, create CSRF Validation for login form

* [Task] #43, added icon to repository for later use

* [Task] #50, cleanup cetntralized; rename token functions

* [Task] #50, reduced token length and improved error handling

* [Task] #50 csrf tests added to login

* [Task] #50, added test case for csrf, repaired integration

* fix: upgrade express-rate-limit from 7.1.5 to 7.2.0 (#52)

Snyk has created this PR to upgrade express-rate-limit from 7.1.5 to 7.2.0.

See this package in npm:
https://www.npmjs.com/package/express-rate-limit

See this project in Snyk:
https://app.snyk.io/org/type-style/project/e2bcd002-cb74-409c-ba55-fb6349df1cbc?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <snyk-bot@snyk.io>

* fix: package.json & package-lock.json to reduce vulnerabilities (#54)

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-EXPRESS-6474509

Co-authored-by: snyk-bot <snyk-bot@snyk.io>

* [Snyk] Upgrade express from 4.18.2 to 4.18.3 (#51)

* fix: upgrade express from 4.18.2 to 4.18.3

Snyk has created this PR to upgrade express from 4.18.2 to 4.18.3.

See this package in npm:
https://www.npmjs.com/package/express

See this project in Snyk:
https://app.snyk.io/org/type-style/project/e2bcd002-cb74-409c-ba55-fb6349df1cbc?utm_source=github&utm_medium=referral&page=upgrade-pr

* 50 integrate csrf protection for login form (#53)

* [Task] #50, create CSRF Validation for login form

* [Task] #43, added icon to repository for later use

* [Task] #50, cleanup cetntralized; rename token functions

* [Task] #50, reduced token length and improved error handling

* [Task] #50 csrf tests added to login

* [Task] #50, added test case for csrf, repaired integration

* fix: upgrade express-rate-limit from 7.1.5 to 7.2.0 (#52)

Snyk has created this PR to upgrade express-rate-limit from 7.1.5 to 7.2.0.

See this package in npm:
https://www.npmjs.com/package/express-rate-limit

See this project in Snyk:
https://app.snyk.io/org/type-style/project/e2bcd002-cb74-409c-ba55-fb6349df1cbc?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <snyk-bot@snyk.io>

---------

Co-authored-by: snyk-bot <snyk-bot@snyk.io>

* [Task] update dev after main merge

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: snyk-bot <snyk-bot@snyk.io>
  • Loading branch information
3 people authored Mar 28, 2024
1 parent b292d6c commit 1f7bb45
Show file tree
Hide file tree
Showing 47 changed files with 5,464 additions and 451 deletions.
2 changes: 1 addition & 1 deletion .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@
//"@typescript-eslint/no-unused-vars": "warn"
"jest/no-conditional-expect": "off"
},
"ignorePatterns": ["dist", "jest.config.js"]
"ignorePatterns": ["dist", "jest.config.js", "httpdocs"]

}
25 changes: 0 additions & 25 deletions .github/workflows/build.yml

This file was deleted.

9 changes: 4 additions & 5 deletions .github/workflows/eslint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ jobs:
with:
node-version: 16
- run: npm ci # or yarn install
- uses: sibiraj-s/action-eslint@v3
with:
eslint-args: '--ignore-path=.gitignore --quiet'
extensions: 'js,jsx,ts,tsx'
annotations: true
- name: Lint server-side code
run: npx eslint src/ --fix
- name: Lint client-side code
run: npx eslint httpdocs/js/ --fix
48 changes: 48 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Tests

on:
workflow_dispatch:
push:
branches: [ "dev", "main" ]
pull_request:
branches: [ "dev", "main" ]

jobs:
build:
runs-on: ubuntu-latest
env:
NODE_ENV: ${{ vars.NODE_ENV }}
LOCALHOST: ${{ vars.LOCALHOST }}
LOCALHOSTV6: ${{ vars.LOCALHOSTV6 }}
KEYA: ${{ secrets.KEYA }}
KEYB: ${{ secrets.KEYB }}
USER_TEST: ${{ secrets.USER_TEST }}

steps:
- uses: actions/checkout@v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
with:
node-version: '20'
cache: 'npm'
- run: echo "NODE_ENV = $NODE_ENV"
- run: npm ci
- run: npm run build --if-present
- name: Start server
run: |
sudo NODE_ENV=$NODE_ENV LOCALHOST=$LOCALHOST LOCALHOSTV6=$LOCALHOSTV6 KEYA=$KEYA KEYB=$KEYB USER_TEST=$USER_TEST npm start &
sleep 15 # Give server some time to start
- name: Check if server is running
run: |
curl --fail http://localhost:80 || exit 1
- name: Run app tests
run: npm run test:app
- name: Run login tests
run: npm run test:login
- name: Run unit tests
run: npm run test:unit
- name: Run integration tests
run: npm run test:integration



2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
data/

# Logs
logs
*.log
Expand Down
32 changes: 13 additions & 19 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -1,21 +1,15 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Debug server.ts",
"skipFiles": [
"<node_internals>/**"
],
"program": "${workspaceFolder}\\src\\app.ts",
"preLaunchTask": "tsc: build - tsconfig.json",
"outFiles": [
"${workspaceFolder}/dist/**/*.js"
]
}
]
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Debug TypeScript",
"skipFiles": [
"<node_internals>/**"
],
"program": "${workspaceFolder}\\src\\app.ts",
"preLaunchTask": "build"
}
]
}
18 changes: 18 additions & 0 deletions .vscode/tasks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "build",
"type": "shell",
"command": "npx tsc -p tsconfig.json",
"group": {
"kind": "build",
"isDefault": true
},
"presentation": {
"reveal": "silent"
},
"problemMatcher": "$tsc"
}
]
}
159 changes: 159 additions & 0 deletions httpdocs/color-table.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 1f7bb45

Please sign in to comment.