Authentication/Authorization microservice

[humphd]

Overview

Our current authentication/authorization system is designed for a monolithic architecture, where the whole app (back-end and front-end) are served from the same origin (i.e., https://domain:port is an origin, https://telescope.cdot.systems:443 is our prod origin).

It works like this:

• A user in our front-end wants to login to their Seneca SSO account
• We redirect them to a /login route
• This in turn creates a SAML formatted (special XML authentication format) POST message that gets sent to Seneca's server.
• The user is redirected to a login page on another machine not owned by us. They enter their username/password, and are redirected back to /login/callback in our server, and a session cookie is set with their user info (we get that back with this request).
• We redirect the user back to the web app they started in, and any time they do network requests to our server, if the route is protected, we check the session for their authentication cookie

Moving to Distributed Auth

We want to split the app up and run it on different origins. For example, microservices, Vercel, localhost, etc. That distributed architecture breaks our security model, since we can't share that cookie across our network of connected apps and services.

We need a way to let a user authenticate, then come back to another part of our system and prove that they are authenticated. A common way to do this type of thing is with OAuth2. OAuth2 is for authorization vs. authentication. It lets systems decide if a user is authorized to do something, and assumes authentication happened somewhere else.

I spent a long time looking into OAuth2 for our needs, but it's overly complicated for our (current) use cases. Because we aren't authorizing third-party apps (e.g., like GitHub or Google do with external apps that can connect to your account), and because we already have an in-house authentication system connected to our authorization needs, we can build a simpler flow.

A Plan

This is the beginnings of a new auth service. It does a few things:

• uses our SAML-based single-sign-on flow for user authentication. I've ported over our existing code and expanded it so that it allows Seneca users to login
• adds an oauth2 inspired authorization flow using JWT access tokens

Here's a rough sketch of what it will look like:

1. A user goes to our front-end app, perhaps https://telescope.cdot.systems
2. The front-end app wants to access a resource on one of our secure microservices, maybe user information.
3. The user needs to login, so clicks the Login link in the front-end app. A small bit of state (e.g., random string) is put into localStorage for later.
4. They are redirected to our auth service: api.telescope.cdot.systems/auth/login?redirect_uri=https://telescope.cdot.systems/&state=a3f1b3413. The URL contains two things: 1) redirect_uri containing a URL pointing back to the entry point of the front-end app; 2) some random state. The latter is used as a ride-along value on all the redirects that are about to take place, and lets the client know at the end that nothing was tampered with in between.
5. The auth service receives the request (/login?redirect_uri=https://telescope.cdot.systems/&state=a3f1b3413) and stores the redirect_uri and state in the session. It then prepares a SAML message for this user to authenticate, and redirects them to the SSO identify provider server.
6. The SSO identity provider receives the request to login, and shows the user a login page, where they enter their username and password. This either works or fails, and it both cases, they are sent back to the auth server with details about what happened
7. The auth server receives the result of the SSO login attempt at /login/callback and examines whether or not the user was authenticated. If they were, we create an access token (JWT) and the request is redirected back to the original app at the redirect_uri: https://telescope.cdot.systems?access_token=...jwt-token-here...&state=...original-state-here...
8. The frontend app examines the query string onload, and sees the access_token and state. It confirms the state is what it expects (e.g., compares to what's in localStorage). The token is then used with all subsequent API requests to our microservices.
9. A request is made to the secure microservice. The token is included in the HTTP Headers: Authorization: bearer <jwt token here>
10. The secure microservice gets the request, and pulls the bearer token out of the headers. It validates it, verifies it, and decides whether or not the user is allowed to get what they want. A 200 or 401 is returned.

Easy, right? I'm skipping some details, but that's the main thrust of what's happening here.

Running and Testing

You can run and test this (yes, I even wrote unit and end-to-end browser tests!). Both require you to run a fake SAML SSO Login server via docker.

Tests

For running the tests:

1. cd src/api
2. docker-compose -f docker-compose-tests.yml up
3. cd auth
4. npm run test:unit
5. npm run test:e2e

If you change line 10 of src/api/auth/test/e2e/e2e.test.js to browser = await chromium.launch({ headless: false, slowMo: 1000 }); you can see the browsers start and run the e2e tests visually. It's possible these will fail on your machine due to timing/resource limits. I am still getting the feel for how far I can push the test runner.

Manually

You can play with this manually too. I made a tiny static HTML web page that lets you use it. You need to run the auth server and login container together in docker:

1. cd src/api
2. docker-compose -f docker-compose-api.yml up --build auth login
3. cd auth
4. npm run test:manual

Now you can navigate to the various servers:

1. http://localhost:8888 is the web app (like running Telescope's front-end)
2. http://auth.docker.localhost/authorize (this is the auth server, not much to see, no UI)
3. http://login.docker.localhost/simplesamle (the SSO server, like Seneca's SSO)

To try logging in:

1. Go to http://localhost:8888 and click Login
2. You get redirected to the login server, something like http://login.docker.localhost/simplesaml/module.php/core/loginuserpass.php?AuthState=_a09df419f1488d4ca3bb879e96f4a42a2399d22095%3Ahttp%3A%2F%2Flogin.docker.localhost%2Fsimplesaml%2Fsaml2%2Fidp%2FSSOService.php%3Fspentityid%3Dhttp%253A%252F%252Fauth.docker.localhost%252Fsp%26cookieTime%3D1613785032
3. Enter your username and password: user1 and user1pass
4. You'll get redirected back to the web app, something like http://localhost:8888/?access_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwOi8vYXV0aC5kb2NrZXIubG9jYWxob3N0IiwiYXVkIjoiaHR0cDovL2xvY2FsaG9zdDo4ODg4LyIsInN1YiI6InVzZXIxQGV4YW1wbGUuY29tIiwiaWF0IjoxNjEzNzg1MDk1LCJleHAiOjE2MTM3ODg2OTV9.53s-dCQX5xPAryTvpEpV6Vo7NAT0jHSf4sUJkh0EXxU&state=R4j4vVKPceGESO368-plF
5. Notice the access_token and state on the URL. Copy the access_token (eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwOi8vYXV0aC5kb2NrZXIubG9jYWxob3N0IiwiYXVkIjoiaHR0cDovL2xvY2FsaG9zdDo4ODg4LyIsInN1YiI6InVzZXIxQGV4YW1wbGUuY29tIiwiaWF0IjoxNjEzNzg1MDk1LCJleHAiOjE2MTM3ODg2OTV9.53s-dCQX5xPAryTvpEpV6Vo7NAT0jHSf4sUJkh0EXxU)
6. Go to https://jwt.io/ and scroll down to the Encoded section. Paste it in to decode it:

That is the authorization info we'll pass around to different servers. We'll sign it cryptographiclly, so it can't be faked from some other source. We'll also add more claims to it (e.g., user vs. admin, name).

I need to add more production-level implementation to this, but it's probably ready to land and people can play with it. I also haven't connected the tests to CI properly yet.

I also think I want to integrate this with the work in #1642 that @chrispinkney is doing (i.e., it's tightly bound to the idea of authorization, so it might make sense to do it all in one service).

I wanted to get this up now because the other microservices are going to need it, and I didn't want to block progress for others during 1.8.

I'll do a proper demo of all this next week during our calls. Let me know if you have questions or thoughts.

[vercel]

@humphd is attempting to deploy a commit to a Personal Account owned by @Seneca-CDOT on Vercel.

@Seneca-CDOT first needs to authorize it.

[vercel]

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/humphd/telescope/hjpejts59
✅ Preview: https://telescope-git-fork-humphd-auth-service.humphd.now.sh

[humphd]

@chrispinkney I've added you to the reviewers list too, since you blogged that you're using an oauth2 flow in your systems project, and since you're going to connecting code to mine with the user service.

[manekenpix]

I get this when I try to run e2e tests as suggested:

# e2e tests, docker required
npm run services:start
npm run jest:e2e
npm run services:stop

Output:

Jest dev-server output:
npx: installed 23 in 1.858s
[Jest Process Manager] Starting up http-server, serving src/api/auth/test/e2e
Available on:
[Jest Process Manager]   http://127.0.0.1:8888
  http://192.168.2.17:8888
  http://172.21.0.1:8888
Hit CTRL-C to stop the server
Error: Host system is missing dependencies!
  Missing libraries are:
      libwebp.so.6
      libenchant.so.1
      libicui18n.so.66
      libicuuc.so.66

Tested on Linux distros: Manjaro and Fedora 32

[humphd]

@manekenpix playwright needs various libs to run the libraries. They have a docker container that lists them:

https://github.com/microsoft/playwright/blob/master/utils/docker/Dockerfile.bionic

I wonder how we should deal with this? Via update to the docs for installing the dev env?

It's interesting that it runs OK in CI on GitHub but not your distro.

[raygervais]

As long as we can run the Docker tests, I believe that is good enough. No point adding additional dependencies to the development environment which increase it's size. Thoughts?

[humphd]

@manekenpix is going to test more to see what those missing libs are all about. We do need to figure out at least a docs update to tell people that we use them for e2e tests.

In theory, running e2e tests only in CI is probably OK (e.g., not everyone needs to do it locally)

[birtony]

sounds secure

[humphd]

I'm going to try and land this tonight, and filed follow-ups on various things that need to happen after this goes in: #1868, #1869, #1870, #1871, #1872, #1873