Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(event_handler): add Middleware support for REST Event Handler #2917

Merged
merged 92 commits into from
Sep 7, 2023
Merged
Show file tree
Hide file tree
Changes from 88 commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
fe2341f
feat(rest-middleware): initial work for rest API middleware
walmsles Aug 2, 2023
4cfa87d
feat(rest-middlware): Add in Router based middleware and add Middlewa…
walmsles Aug 3, 2023
4e28b06
feat(api-middleware): remove compression middleware - not implemented…
walmsles Aug 3, 2023
edcc6f7
fix(typing): resolve mypy errors
walmsles Aug 3, 2023
2c67e10
Update aws_lambda_powertools/event_handler/api_gateway.py
walmsles Aug 3, 2023
822f16b
Update aws_lambda_powertools/event_handler/api_gateway.py
walmsles Aug 3, 2023
51c3da5
Update aws_lambda_powertools/event_handler/api_gateway.py
walmsles Aug 3, 2023
98da7dc
Update aws_lambda_powertools/event_handler/api_gateway.py
walmsles Aug 3, 2023
d775bba
Update aws_lambda_powertools/event_handler/api_gateway.py
walmsles Aug 3, 2023
d12fa20
Update aws_lambda_powertools/event_handler/api_gateway.py
walmsles Aug 3, 2023
e765ebc
Apply suggestions from code review
walmsles Aug 3, 2023
6e54256
fix(naming-intent): rename middleware to middlewares universally in a…
walmsles Aug 3, 2023
727effb
fix(args-debt): rename args to route_arguments in all places
walmsles Aug 3, 2023
0afe31c
fix(middleware-clarity): Rename and repackage middleware construction…
walmsles Aug 4, 2023
8501b9a
fix(jsdocs): added docs to MiddlewareStackWrapper and BaseMiddlewareH…
walmsles Aug 4, 2023
0d59e12
fix(types): Mkae MiddlewareStackWrapper __call__ return type more spe…
walmsles Aug 4, 2023
6b9b369
fix(middlewares): internalise registered_api_adapter and remove Cors/…
walmsles Aug 5, 2023
409da32
feat(middlewares): Add chema validation middleware and internalise re…
walmsles Aug 5, 2023
1a90b34
chore(security): relabel http to https for test JSON Schema (because …
walmsles Aug 5, 2023
a9d9cb2
Apply suggestions from code review
walmsles Aug 10, 2023
10080b5
chore(typing): Ficup typing and function/attribute scopes - keep priv…
walmsles Aug 10, 2023
de7247d
feat(middlewares): Add optional outbound schema dn formats to schema …
walmsles Aug 10, 2023
dbae938
fix(errors): add logger for config exceptions and return minimal details
walmsles Aug 10, 2023
13df122
chore(docs): fixup format of docstrings to numpy format
walmsles Aug 10, 2023
8067116
chore(docs): change docstrings to numpy format
walmsles Aug 10, 2023
6ce64f1
fix(tests): fix failing tests due to resposne change:
walmsles Aug 11, 2023
22b83e0
chore(tests): group middleware testing to one module
walmsles Aug 11, 2023
af719f4
chore(docs): api_agteway - improve docs for middleware and examples
walmsles Aug 13, 2023
0adfc70
chore(docs): document why mypy 'type: ignore' is used here
walmsles Aug 13, 2023
63db815
feat(api_middleware): add source diagram for middlewares docs
walmsles Aug 13, 2023
3131f3e
fix(tests): fix unreachable code
walmsles Aug 13, 2023
1a20c9e
fix(debug): fix processed middleware stack frame list, cleanup try/ca…
walmsles Aug 15, 2023
48a9462
fix(debug): revert small change to ensure debug noise reduced via han…
walmsles Aug 15, 2023
587348c
chore(comments): cleanup middleware execution
walmsles Aug 19, 2023
deabb26
feat(docs): Add docs for event_handler middleware
walmsles Aug 20, 2023
fbb36ab
feat(docs): Tidy up middleware docs and add to section to split routes
walmsles Aug 20, 2023
d4fc1ba
feat(route): expose matched route instance to Resolver in additional …
walmsles Aug 22, 2023
837613b
fix(resolver): Add 'powertols_route' to additional context
walmsles Aug 22, 2023
9aec250
Merge branch 'develop' into feat/api-middleware
heitorlessa Aug 28, 2023
49537a9
docs: make intro punchy plus intro diagram
heitorlessa Aug 28, 2023
d0be0e4
docs: use correlation id as first middleware
heitorlessa Aug 28, 2023
2b6ee0b
docs: add output for completeness
heitorlessa Aug 28, 2023
33cf3ad
docs: grammar
heitorlessa Aug 28, 2023
40b68a0
docs: note that middleware works for any resolver
heitorlessa Aug 28, 2023
7bd0dba
docs: add global middlewares section
heitorlessa Aug 28, 2023
947d0ce
docs: additional ctx for global middlewares order
heitorlessa Aug 28, 2023
efee2ae
docs: add early return section
heitorlessa Aug 28, 2023
741f2a5
feat(route): expose route instance in ephemeral context dict
walmsles Aug 30, 2023
388592f
feat(middlewwares): Expand tests to incdlue core gateway types
walmsles Aug 30, 2023
af15705
feat(middlewares): Additional Tests for route middleware, added debug…
walmsles Aug 30, 2023
bf439e4
feat(middlewares): Tidy up config, add additional context for _path f…
walmsles Aug 30, 2023
80aa67e
Merge branch 'develop' into feat/api-middleware
walmsles Sep 1, 2023
9fb4c6f
chore(docs): updated imges to be svg for middlewares from draw.io - l…
walmsles Sep 1, 2023
06b1d90
chore(example): fix custom middleware example to correct raise of cap…
walmsles Sep 2, 2023
61022be
chore: type middleware callback
heitorlessa Sep 2, 2023
21af597
chore: type middleware response
heitorlessa Sep 2, 2023
0ca6683
docs: improve early return; use kwargs over ctx
heitorlessa Sep 2, 2023
9704939
docs: add handling exceptions section
heitorlessa Sep 2, 2023
d166de3
docs: fix media typo
heitorlessa Sep 2, 2023
f476f7d
docs: add being a good citizen section
heitorlessa Sep 3, 2023
e8a071e
docs: add skeleton for class-based middleware
heitorlessa Sep 3, 2023
dd59a7f
Merge branch 'develop' into feat/api-middleware
heitorlessa Sep 4, 2023
1c0977a
chore: test class based middlewares
heitorlessa Sep 4, 2023
7d361b6
docs: rename class based section
heitorlessa Sep 4, 2023
85e65b0
feat(middlewares): remove kwargs from middleware signature for cleane…
walmsles Sep 4, 2023
3a20318
feat(middlewares): align signatures for middlewareprocessing
walmsles Sep 4, 2023
5edde6a
feat(middlewares): cleanup typing for next callbacks
walmsles Sep 4, 2023
a0e784b
feat(middlewares): align typing for next_middlewares for consistency
walmsles Sep 4, 2023
f20b778
Merge branch 'develop' into feat/api-middleware
heitorlessa Sep 4, 2023
d8cf386
chore: use generics to accept any event handler
heitorlessa Sep 4, 2023
f5a613a
refactor: tech debt overload on get_header_value for middleware examp…
heitorlessa Sep 4, 2023
2f2eb91
chore: enforce protocol type checking for app instance
heitorlessa Sep 4, 2023
03708bb
docs: complete extending middlewares section
heitorlessa Sep 4, 2023
5dc5719
fix: remove schema validation export due to optional dependency
heitorlessa Sep 4, 2023
f7800b2
refactor: improve docstrings for schema validation; typing
heitorlessa Sep 4, 2023
6f63b4f
chore: leftover from previous circular dep issue
heitorlessa Sep 4, 2023
03d15fa
docs: add native middleware section
heitorlessa Sep 4, 2023
b7bcb08
docs - refactor references of with kwargs to , remove **lwargs refer…
walmsles Sep 4, 2023
7ca4a3a
chore: add middleware order test
heitorlessa Sep 5, 2023
870545a
refactor: add debug log
heitorlessa Sep 5, 2023
44cb04b
docs: move staging area to router section
heitorlessa Sep 5, 2023
3832817
Merge branch 'develop' into feat/api-middleware
heitorlessa Sep 5, 2023
4e1a654
fix: remove leftover order from middleware order test
heitorlessa Sep 5, 2023
6400b28
Merge branch 'develop' into feat/api-middleware
heitorlessa Sep 5, 2023
19eaa73
middlewares: reverse internal function rename to ensure no breaking c…
walmsles Sep 6, 2023
592d9da
chore(tests): remove typing isues
walmsles Sep 6, 2023
d1508c7
docs: middleware in router
heitorlessa Sep 7, 2023
aa68e40
chore: last cleanups
heitorlessa Sep 7, 2023
98948ec
docs: leftover to highlight works for micro/mono fns
heitorlessa Sep 7, 2023
20c344a
docs: fix highlighting after refactoring
heitorlessa Sep 7, 2023
4ed0076
chore: remove backup drawio file
heitorlessa Sep 7, 2023
7fbb132
Merge branch 'develop' into feat/api-middleware
leandrodamascena Sep 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
410 changes: 381 additions & 29 deletions aws_lambda_powertools/event_handler/api_gateway.py

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions aws_lambda_powertools/event_handler/middlewares/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from aws_lambda_powertools.event_handler.middlewares.base import BaseMiddlewareHandler, NextMiddleware

__all__ = ["BaseMiddlewareHandler", "NextMiddleware"]
122 changes: 122 additions & 0 deletions aws_lambda_powertools/event_handler/middlewares/base.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
from abc import ABC, abstractmethod
from typing import Generic

from typing_extensions import Protocol

from aws_lambda_powertools.event_handler.api_gateway import Response
from aws_lambda_powertools.event_handler.types import EventHandlerInstance


class NextMiddleware(Protocol):
def __call__(self, app: EventHandlerInstance) -> Response:
"""Protocol for callback regardless of next_middleware(app), get_response(app) etc"""
...

def __name__(self) -> str: # noqa A003
"""Protocol for name of the Middleware"""
...


class BaseMiddlewareHandler(Generic[EventHandlerInstance], ABC):
"""Base implementation for Middlewares to run code before and after in a chain.


This is the middleware handler function where middleware logic is implemented.
The next middleware handler is represented by `next_middleware`, returning a Response object.

Examples
--------

**Correlation ID Middleware**

```python
import requests

from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler import APIGatewayRestResolver, Response
from aws_lambda_powertools.event_handler.middlewares import BaseMiddlewareHandler, NextMiddleware

app = APIGatewayRestResolver()
logger = Logger()


class CorrelationIdMiddleware(BaseMiddlewareHandler):
def __init__(self, header: str):
super().__init__()
self.header = header

def handler(self, app: APIGatewayRestResolver, next_middleware: NextMiddleware) -> Response:
# BEFORE logic
request_id = app.current_event.request_context.request_id
correlation_id = app.current_event.get_header_value(
name=self.header,
default_value=request_id,
)

# Call next middleware or route handler ('/todos')
response = next_middleware(app)

# AFTER logic
response.headers[self.header] = correlation_id

return response


@app.get("/todos", middlewares=[CorrelationIdMiddleware(header="x-correlation-id")])
def get_todos():
todos: requests.Response = requests.get("https://jsonplaceholder.typicode.com/todos")
todos.raise_for_status()

# for brevity, we'll limit to the first 10 only
return {"todos": todos.json()[:10]}


@logger.inject_lambda_context
def lambda_handler(event, context):
return app.resolve(event, context)

```

"""

@abstractmethod
def handler(self, app: EventHandlerInstance, next_middleware: NextMiddleware) -> Response:
"""
The Middleware Handler

Parameters
----------
app: EventHandlerInstance
An instance of an Event Handler that implements ApiGatewayResolver
next_middleware: NextMiddleware
The next middleware handler in the chain

Returns
-------
Response
The response from the next middleware handler in the chain

"""
raise NotImplementedError()

@property
def __name__(self) -> str: # noqa A003
return str(self.__class__.__name__)

def __call__(self, app: EventHandlerInstance, next_middleware: NextMiddleware) -> Response:
"""
The Middleware handler function.

Parameters
----------
app: ApiGatewayResolver
An instance of an Event Handler that implements ApiGatewayResolver
next_middleware: NextMiddleware
The next middleware handler in the chain

Returns
-------
Response
The response from the next middleware handler in the chain
"""
return self.handler(app, next_middleware)
124 changes: 124 additions & 0 deletions aws_lambda_powertools/event_handler/middlewares/schema_validation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
import logging
from typing import Dict, Optional

from aws_lambda_powertools.event_handler.api_gateway import Response
from aws_lambda_powertools.event_handler.exceptions import BadRequestError, InternalServerError
from aws_lambda_powertools.event_handler.middlewares import BaseMiddlewareHandler, NextMiddleware
from aws_lambda_powertools.event_handler.types import EventHandlerInstance
from aws_lambda_powertools.utilities.validation import validate
from aws_lambda_powertools.utilities.validation.exceptions import InvalidSchemaFormatError, SchemaValidationError

logger = logging.getLogger(__name__)


class SchemaValidationMiddleware(BaseMiddlewareHandler):
"""Middleware to validate API request and response against JSON Schema using the [Validation utility](https://docs.powertools.aws.dev/lambda/python/latest/utilities/validation/).

Examples
--------
**Validating incoming event**

```python
import requests

from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler import APIGatewayRestResolver, Response
from aws_lambda_powertools.event_handler.middlewares import BaseMiddlewareHandler, NextMiddleware
from aws_lambda_powertools.event_handler.middlewares.schema_validation import SchemaValidationMiddleware

app = APIGatewayRestResolver()
logger = Logger()
json_schema_validation = SchemaValidationMiddleware(inbound_schema=INCOMING_JSON_SCHEMA)


@app.get("/todos", middlewares=[json_schema_validation])
def get_todos():
todos: requests.Response = requests.get("https://jsonplaceholder.typicode.com/todos")
todos.raise_for_status()

# for brevity, we'll limit to the first 10 only
return {"todos": todos.json()[:10]}


@logger.inject_lambda_context
def lambda_handler(event, context):
return app.resolve(event, context)
```
"""

def __init__(
self,
inbound_schema: Dict,
inbound_formats: Optional[Dict] = None,
outbound_schema: Optional[Dict] = None,
outbound_formats: Optional[Dict] = None,
):
"""See [Validation utility](https://docs.powertools.aws.dev/lambda/python/latest/utilities/validation/) docs for examples on all parameters.

Parameters
----------
inbound_schema : Dict
JSON Schema to validate incoming event
inbound_formats : Optional[Dict], optional
Custom formats containing a key (e.g. int64) and a value expressed as regex or callback returning bool, by default None
JSON Schema to validate outbound event, by default None
outbound_formats : Optional[Dict], optional
Custom formats containing a key (e.g. int64) and a value expressed as regex or callback returning bool, by default None
""" # noqa: E501
super().__init__()
self.inbound_schema = inbound_schema
self.inbound_formats = inbound_formats
self.outbound_schema = outbound_schema
self.outbound_formats = outbound_formats

def bad_response(self, error: SchemaValidationError) -> Response:
message: str = f"Bad Response: {error.message}"
logger.debug(message)
raise BadRequestError(message)

def bad_request(self, error: SchemaValidationError) -> Response:
message: str = f"Bad Request: {error.message}"
logger.debug(message)
raise BadRequestError(message)

def bad_config(self, error: InvalidSchemaFormatError) -> Response:
logger.debug(f"Invalid Schema Format: {error}")
raise InternalServerError("Internal Server Error")

def handler(self, app: EventHandlerInstance, next_middleware: NextMiddleware) -> Response:
"""Validates incoming JSON payload (body) against JSON Schema provided.

Parameters
----------
app : EventHandlerInstance
An instance of an Event Handler
next_middleware : NextMiddleware
Callable to get response from the next middleware or route handler in the chain

Returns
-------
Response
It can return three types of response objects

- Original response: Propagates HTTP response returned from the next middleware if validation succeeds
- HTTP 400: Payload or response failed JSON Schema validation
- HTTP 500: JSON Schema provided has incorrect format
"""
try:
validate(event=app.current_event.json_body, schema=self.inbound_schema, formats=self.inbound_formats)
except SchemaValidationError as error:
return self.bad_request(error)
except InvalidSchemaFormatError as error:
return self.bad_config(error)

result = next_middleware(app)

if self.outbound_formats is not None:
try:
validate(event=result.body, schema=self.inbound_schema, formats=self.inbound_formats)
except SchemaValidationError as error:
return self.bad_response(error)
except InvalidSchemaFormatError as error:
return self.bad_config(error)

return result
5 changes: 5 additions & 0 deletions aws_lambda_powertools/event_handler/types.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from typing import TypeVar

from aws_lambda_powertools.event_handler import ApiGatewayResolver

EventHandlerInstance = TypeVar("EventHandlerInstance", bound=ApiGatewayResolver)
9 changes: 9 additions & 0 deletions aws_lambda_powertools/shared/types.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
import sys
from typing import Any, Callable, Dict, List, TypeVar, Union

AnyCallableT = TypeVar("AnyCallableT", bound=Callable[..., Any]) # noqa: VNE001
# JSON primitives only, mypy doesn't support recursive tho
JSONType = Union[str, int, float, bool, None, Dict[str, Any], List[Any]]


if sys.version_info >= (3, 8):
from typing import Protocol
else:
from typing_extensions import Protocol

__all__ = ["Protocol"]
21 changes: 19 additions & 2 deletions aws_lambda_powertools/utilities/data_classes/common.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import base64
import json
from collections.abc import Mapping
from typing import Any, Callable, Dict, Iterator, List, Optional
from typing import Any, Callable, Dict, Iterator, List, Optional, overload

from aws_lambda_powertools.shared.headers_serializer import BaseHeadersSerializer
from aws_lambda_powertools.utilities.data_classes.shared_functions import (
Expand Down Expand Up @@ -156,7 +156,24 @@ def get_query_string_value(self, name: str, default_value: Optional[str] = None)
default_value=default_value,
)

# Maintenance: missing @overload to ensure return type is a str when default_value is set
@overload
def get_header_value(
self,
name: str,
default_value: str,
case_sensitive: Optional[bool] = False,
) -> str:
...

@overload
def get_header_value(
self,
name: str,
default_value: Optional[str] = None,
case_sensitive: Optional[bool] = False,
) -> Optional[str]:
...

def get_header_value(
self,
name: str,
Expand Down
20 changes: 19 additions & 1 deletion aws_lambda_powertools/utilities/data_classes/vpc_lattice.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from typing import Any, Dict, Optional
from typing import Any, Dict, Optional, overload

from aws_lambda_powertools.shared.headers_serializer import (
BaseHeadersSerializer,
Expand Down Expand Up @@ -91,6 +91,24 @@ def get_query_string_value(self, name: str, default_value: Optional[str] = None)
default_value=default_value,
)

@overload
def get_header_value(
self,
name: str,
default_value: str,
case_sensitive: Optional[bool] = False,
) -> str:
...

@overload
def get_header_value(
self,
name: str,
default_value: Optional[str] = None,
case_sensitive: Optional[bool] = False,
) -> Optional[str]:
...

def get_header_value(
self,
name: str,
Expand Down
Loading