Skip to content

Releases: aws-powertools/powertools-lambda-python

v1.24.1

20 Jan 07:06
@release-drafter release-drafter
v1.24.1
fa07bc5
This commit was signed with the committer’s verified signature.
heitorlessa Heitor Lessa
GPG key ID: DA9C8E6177723B7D
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.1

Summary

This is an emergency release to fix a critical Batch bug spotted by @kimberlyamandalu, where multiple failed records weren't reported due to dictionary key uniqueness - If you are using BatchProcessor, please update to 1.24.1 as soon as possible.

New official tutorial

Thanks to the gigantic effort by @mploski and @am29d on reviewing it, we now finally have an official tutorial covering core Powertools features. The tutorial demonstrates how new customers can add one feature at a time. It also opens the door to create other tutorials covering other complex features in more depth.

Please do let us know your thoughts and what other tutorials we should focus on next ;-)

image

New tiny function to copy Logger config to external loggers

A common question we receive is: How can I enable powertools logging for imported libraries?

Thanks to @mploski, we now have a tiny standalone function you can use to copy your current Logger config to any or a select list of loggers you want.

Why would people want that? It's a fair question. Reason is sometimes you want external libraries to have the exact same structured logging that your application has. This function allows you to specify which explicit loggers you wanna copy config to (or all), and whether you want to set a different log level.

import logging

from aws_lambda_powertools import Logger
from aws_lambda_powertools.logging import utils

logger = Logger()

external_logger = logging.logger()

utils.copy_config_to_registered_loggers(source_logger=logger)
external_logger.info("test message")

Big thanks to new contributors @thehananbhat @j2clerck on helping us make documentation better (critical to us!).

Changes

🌟New features and non-breaking changes

  • feat(logger): clone powertools logger config to any Python logger (#927) by @mploski

🌟 Minor Changes

📜 Documentation updates

  • docs(tutorial): fix path to images (#963) by @mploski
  • docs(batch): snippet typo in custom batch processor (#961) by @thehananbhat
  • docs(batch): snippet typo on batch processed messages iteration (#951) by @j2clerck
  • docs(nav): make REST and GraphQL event handlers more explicit (#959) by @heitorlessa
  • docs(logger): fix code block syntax in FAQ (#952) by @mozz100
  • docs(tutorial): add new tutorial covering core features (#769) by @mploski
  • docs(homepage): link to typescript version (#950) by @michaelbrewer
  • fix(parameters): appconfig internal _get docstrings (#934) by @ran-isenberg
  • docs(batch): fix typo in context manager keyword (#938) by @heitorlessa
  • feat(logger): clone powertools logger config to any Python logger (#927) by @mploski

🐛 Bug and hot fixes

🔧 Maintenance

This release was made possible by the following contributors:

@dependabot, @dependabot[bot], @heitorlessa, @j2clerck, @michaelbrewer, @mozz100, @mploski, @ran-isenberg, @thehananbhat and Michal Ploski

Contributors

mploski, mozz100, and 8 other contributors
Loading

v1.24.0

31 Dec 17:51
@release-drafter release-drafter
v1.24.0
22f8719
This commit was signed with the committer’s verified signature.
heitorlessa Heitor Lessa
GPG key ID: DA9C8E6177723B7D
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.0

Summary

For the last release of the year (happy 2022!), we bring a major enhancements to Idempotency and Feature Flags.

We also addresses important papercuts like caching parsed JSON data in Event Sources, support for datetime format codes in Logger and the ability to ignore certain endpoints from being traced.

Did I say that 90% of this release was contributed by the community? thank you everyone!!!

HUGE shoutout to @DanyC97 on helping us make all of our documentation banners (warning, tip) consistent.

Big thanks to new contributors too (you rock!)

  • @trey-rosius for adding a medium-size GraphQL API example using a myriad of Lambda Powertools features
  • @nayaverdier for a future proof change on how we internally convert string to bool (distutils being deprecated in Python 3.12)

idempotent_function now supports dataclasses & Pydantic models

Docs with samples

When using idempotent_function to make any Python synchronous function idempotent, you might have data available as Dataclasses or Pydantic models, not just dictionaries.

This release gives you more flexibility on what data can be used as your idempotency token - it could be an entire Dataclass, Pydantic model, or fields within these models.

image

Going beyond boolean feature flags

Docs with sample

You can now use the new boolean_feature: false parameter in your schema to signal Feature Flags that you will return any JSON valid value.

Example scenario: you might have a list of features to unlock for premium customers, or a set of beta features for select customers

{
    "premium_features": {
        "boolean_type": false,
        "default": [],
        "rules": {
            "customer tier equals premium": {
                "when_match": ["no_ads", "no_limits", "chat"],
                "conditions": [
                    {
                        "action": "EQUALS",
                        "key": "tier",
                        "value": "premium"
                    }
                ]
            }
        }
    }
}

Translating to the following API:

from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore

app_config = AppConfigStore(
    environment="dev",
    application="product-catalogue",
    name="features"
)

feature_flags = FeatureFlags(store=app_config)

def lambda_handler(event, context):
    # Get customer's tier from incoming request
    ctx = { "tier": event.get("tier", "standard") }

    # Evaluate `has_premium_features` base don customer's tier
    premium_features: list[str] = feature_flags.evaluate(name="premium_features",
                                                        context=ctx, default=False)
    for feature in premium_features:
        # enable premium features
        ...

Ignoring HTTP endpoints from tracer

AWS X-Ray has a limit of 64K tracing data. This could be a problem if you're making hundreds of HTTP requests to the same endpoint.

Alternatively, there are sensitive endpoints you might want them to not be included in your tracing data.

You can now use ignore_endpoint for this purpose - globs (*) are allowed!

from aws_lambda_powertools import Tracer

tracer = Tracer()
# ignore all calls to `ec2.amazon.com`
tracer.ignore_endpoint(hostname="ec2.amazon.com")
# ignore calls to `*.sensitive.com/password` and  `*.sensitive.com/credit-card`
tracer.ignore_endpoint(hostname="*.sensitive.com", urls=["/password", "/credit-card"])


def ec2_api_calls():
    return "suppress_api_responses"

@tracer.capture_lambda_handler
def handler(event, context):
    for x in long_list:
        ec2_api_calls()

Changes

🌟New features and non-breaking changes

  • feat(logger): support use_datetime_directive for timestamps (#920) by @huonw
  • feat(feature_flags): support beyond boolean values (JSON values) (#804) by @ran-isenberg
  • feat(idempotency): support dataclasses & pydantic models payloads (#908) by @michaelbrewer
  • feat(tracer): ignore tracing for certain hostname(s) or url(s) (#910) by @michaelbrewer
  • feat(event-sources): cache parsed json in data class (#909) by @michaelbrewer

📜 Documentation updates

  • docs(tracer): new ignore_endpoint feature (#931) by @heitorlessa
  • feat(logger): support use_datetime_directive for timestamps (#920) by @huonw
  • feat(feature_flags): support beyond boolean values (JSON values) (#804) by @ran-isenberg
  • docs(general): consistency around admonitions and snippets (#919) by @DanyC97
  • docs(homepage): new GraphQL sample API in examples section (#930) by @trey-rosius
  • feat(idempotency): support dataclasses & pydantic models payloads (#908) by @michaelbrewer

🐛 Bug and hot fixes

🔧 Maintenance

This release was made possible by the following contributors:

@DanyC97, @dependabot, @dependabot[bot], @heitorlessa, @huonw, @michaelbrewer, @nayaverdier, @ran-isenberg and @trey-rosius

New Contributors

Full Changelog: v1.23.0...v1.23.1

Contributors

huonw, heitorlessa, and 6 other contributors
Loading
0 Join discussion

v1.23.0

20 Dec 11:18
@release-drafter release-drafter
v1.23.0
da5e1f0
This commit was signed with the committer’s verified signature.
heitorlessa Heitor Lessa
GPG key ID: DA9C8E6177723B7D
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.23.0

Summary

This Christmas release was heavily focused on papercuts and two must needed improvements - Batch now supports SQS, DynamoDB, and Kinesis using the new native partial response (decreased cost), and Event Handler API Gateway now supports exception handling and overriding HTTP 404 errors.

For the next release, we're going to further invest in Mypy compliance, support native serialization of Python Dataclasses and Pydantic Models in Idempotency utility, and more!

Join us on Slack #lambda-powertools to help us make the next release even better!

New Batch Processor

image

Docs: Infra required, SQS, Kinesis, and DynamoDB

A few weeks ago Lambda added native support for partial response. Since GA, Batch utility provided that capability to SQS by handling partial failures and deleting successful messages on your behalf.

With the new capability, we've added a new BatchProcessor that can process both SQS, Kinesis Data Streams, and DynamoDB Streams.

The new capability requires an infrastructure change in the Lambda Event Source Mapping hence why a new Batch Processor - We've made sure to add complete SAM examples for SQS, Kinesis, and DynamoDB Streams on everything you need to get going including minimal IAM permissions.

We also took this opportunity to a) provide tight integration with Event Source Data Classes for self-documented batch records, b) Parser (Pydantic) integration so you can bring your own models for data schema and validation, and c) mypy is fully compliant regardles of the event type or Pydantic model you bring.

Did I also say we refreshed the entire documentation with sample events, responses, how to unit test, and a Migration Guide?

Migration guide

If you were using SQSBatchProcessor before as a decorator or as a context manager, we've written a guide to help you transition to the new more performant and cost efficient way: https://awslabs.github.io/aws-lambda-powertools-python/develop/utilities/batch/#migration-guide

A warm start with tracing, logging, data transformation, and exception handling can run as fast as 8ms end-to-end now.

Event Handler API Gateway improvements

Overriding 404 errors

Event Handler API Gateway snippet demonstrating 404 override

You can now override how we handle unmatched routes (404) by using a decorator not_found. The experience is similar to how you'd define routes, except your function will receive an exception - NotFoundError in this case.

You can customize the response by simply returning Response like you would in a more advanced route -- This should give you more flexibility to generate metrics, log errors, or anything you might want for unmatched routes.

Exception handling

Event Handler API Gateway snippet demonstrating exception handling

Besides 404 errors, you might want to handle custom validation errors, or any exception you might raise as part of your route handling. You can now use exception_handler decorator and pass any valid Python Exception.

Tracer

X-Ray Trace Annotations screenshot for the handler subsegment

Tracer now automatically adds a Service annotation if either service parameter or POWERTOOLS_SERVICE_NAME is set - This makes it easier to slice and dice all of your traces by the exact service you have deployed.

Moreover, we now include ColdStart=false for warm start invocations so you can more easily sample and analyze performance differences between cold and warm starts.

CloudFormation Custom Resources

As our Java Powertools colleagues added a new utility for easily writing CloudFormation Custom Resources, we added a link in our docs pointing to the official Python library for crafting custom resources.... just in case it's not widely known: https://github.com/aws-cloudformation/custom-resource-helper

Changes

🌟New features and non-breaking changes

  • feat(logger): allow handler with custom kwargs signature (#913) by @heitorlessa
  • feat(batch): new BatchProcessor for SQS, DynamoDB, Kinesis (#886) by @heitorlessa
  • feat(apigateway): add exception_handler support (#898) by @michaelbrewer
  • feat(tracer): add service annotation when service is set (#861) by @heitorlessa
  • feat(apigateway): access parent api resolver from router (#842) by @cakepietoast

🌟 Minor Changes

  • fix(metrics): explicit type to single_metric ctx manager (#865) by @whardier

📜 Documentation updates

  • docs(apigateway): add new not_found feature (#915) by @heitorlessa
  • docs(nav): reference cloudformation custom resource helper (#914) by @heitorlessa
  • feat(batch): new BatchProcessor for SQS, DynamoDB, Kinesis (#886) by @heitorlessa
  • fix(parser): mypy support for payload type override as models (#883) by @heitorlessa
  • docs(apigateway): fix sample layout provided (#864) by @michaelbrewer
  • feat(tracer): add service annotation when service is set (#861) by @heitorlessa
  • fix(apigateway): allow list of HTTP methods in route method (#838) by @cakepietoast

🐛 Bug and hot fixes

  • fix(parser): overload parse when using envelope (#885) by @heitorlessa
  • fix(parser): kinesis sequence number is str, not int (#907) by @heitorlessa
  • fix(event-sources): Pass authorizer data to APIGatewayEventAuthorizer (#897) by @michaelbrewer
  • fix(parser): mypy support for payload type override as models (#883) by @heitorlessa
  • fix(parameters): appconfig transform and return types (#877) by @ran-isenberg
  • fix(event-sources): handle Cognito claimsOverrideDetails set to null (#878) by @michaelbrewer
  • fix(idempotency): include decorated fn name in hash (#869) by @michaelbrewer
  • fix(tracer): add warm start annotation (ColdStart=False) (#851) by @heitorlessa
  • fix(apigateway): allow list of HTTP methods in route method (#838) by @cakepietoast

🔧 Maintenance

This release was made possible by the following contributors:

@cakepietoast, @dependabot, @dependabot[bot], @heitorlessa, @michaelbrewer, @ran-isenberg and @whardier

Full Changelog: v1.22.0...v1.22.1

Contributors

whardier, heitorlessa, and 4 other contributors
Loading
0 Join discussion

v1.22.0

17 Nov 10:22
@release-drafter release-drafter
v1.22.0
075ac41
This commit was signed with the committer’s verified signature.
heitorlessa Heitor Lessa
GPG key ID: DA9C8E6177723B7D
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.22.0

Summary

This release adds two major changes: 1/ New Router feature in Event Handler utility including GraphQL Resolvers composition in AppSync, and 2/ Idiomatic tenet has been updated to Progressive.

Additionally, we now support ActiveMQ and RabbitMQ in the Event Source Data Classes, and primary composite key for Idempotency when using DynamoDB Storage. There's been lots of improvements to documentation around Lambda Layers install, and a bug fix for Parser (Pydantic) to address API Gateway v1/v2 supporting a null body.

This release note will primarily cover the new Router feature in Event Handler given how significant this is. Also, we created a new section named Considerations in the docs to share an opinionated set of trade-offs when going with a monolithic vs micro function approach, when using API Gateway, ALB, or AppSync.

Router feature in Event Handler

You can now use separate files to compose routes and GraphQL resolvers. Before this feature, you'd need all your routes or GraphQL resolvers in the same file where your Lambda handler is.

API Gateway and ALB

This is how it would look like before this feature in either API Gateway, ALB, and AppSync:

app.py

import itertools
from typing import Dict

from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver

logger = Logger(child=True)
app = ApiGatewayResolver()

USERS = {"user1": "details_here", "user2": "details_here", "user3": "details_here"}


@app.get("/users")
def get_users() -> Dict:
    # /users?limit=1
    pagination_limit = app.current_event.get_query_string_value(name="limit", default_value=10)

    logger.info(f"Fetching the first {pagination_limit} users...")
    ret = dict(itertools.islice(USERS.items(), int(pagination_limit)))
    return {"items": [ret]}

@app.get("/users/<username>")
def get_user(username: str) -> Dict:
    logger.info(f"Fetching username {username}")
    return {"details": USERS.get(username, {})}

With Router, you can now split the /users routes in a separate file and change ApiGatewayResolver with Router, for example:

users.py

import itertools
from typing import Dict

from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler.api_gateway import Router

logger = Logger(child=True)
router = Router()
USERS = {"user1": "details_here", "user2": "details_here", "user3": "details_here"}


@router.get("/users")
def get_users() -> Dict:
    # /users?limit=1
    pagination_limit = router.current_event.get_query_string_value(name="limit", default_value=10)

    logger.info(f"Fetching the first {pagination_limit} users...")
    ret = dict(itertools.islice(USERS.items(), int(pagination_limit)))
    return {"items": [ret]}

@router.get("/users/<username>")
def get_user(username: str) -> Dict:
    logger.info(f"Fetching username {username}")
    return {"details": USERS.get(username, {})}

Note that the user experience is exactly the same on accessing request details and defining routes, except we use Router instead of ApiGatewayResolver.

Next, within your Lambda entry point, you have to use the new include_router method to inject routes from /users at runtime:

app.py

from typing import Dict

from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler import ApiGatewayResolver
from aws_lambda_powertools.utilities.typing import LambdaContext

import users

logger = Logger()
app = ApiGatewayResolver()
app.include_router(users.router)

@logger.inject_lambda_context
def lambda_handler(event: Dict, context: LambdaContext):
    return app.resolve(event, context)

GraphQL Resolvers

Similarly to API Gateway and ALB, you can now use Router to split GraphQL resolvers allowing for further composition:

resolvers/location.py

from typing import Any, Dict, List

from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler.appsync import Router

logger = Logger(child=True)
router = Router()


@router.resolver(type_name="Query", field_name="listLocations")
def list_locations(merchant_id: str) -> List[Dict[str, Any]]:
    return [{"name": "Location name", "merchant_id": merchant_id}]


@router.resolver(type_name="Location", field_name="status")
def resolve_status(merchant_id: str) -> str:
    logger.debug(f"Resolve status for merchant_id: {merchant_id}")
    return "FOO"

app.py

from typing import Dict

from aws_lambda_powertools import Logger, Tracer
from aws_lambda_powertools.event_handler import AppSyncResolver
from aws_lambda_powertools.logging.correlation_paths import APPSYNC_RESOLVER
from aws_lambda_powertools.utilities.typing import LambdaContext

from resolvers import location

tracer = Tracer()
logger = Logger()
app = AppSyncResolver()
app.include_router(location.router)


@tracer.capture_lambda_handler
@logger.inject_lambda_context(correlation_id_path=APPSYNC_RESOLVER)
def lambda_handler(event: Dict, context: LambdaContext):
    app.resolve(event, context)

Tenet update

We've updated Idiomatic tenet to Progressive to reflect the new Router feature in Event Handler, and more importantly the new wave of customers coming from SRE, Data Analysis, and Data Science background.

  • BEFORE: Idiomatic. Utilities follow programming language idioms and language-specific best practices.
  • AFTER: Progressive. Utilities are designed to be incrementally adoptable for customers at any stage of their Serverless journey. They follow language idioms and their community’s common practices.

Changes

🌟New features and non-breaking changes

  • feat(idempotency): support composite primary key in DynamoDBPersistenceLayer (#740) by @Tankanow
  • feat(data-classes): ActiveMQ and RabbitMQ support (#770) by @michaelbrewer
  • feat(appsync): add Router to allow large resolver composition (#776) by @michaelbrewer
  • feat(apigateway): add Router to allow large routing composition (#645) by @BVMiko

🌟 Minor Changes

📜 Documentation updates

🐛 Bug and hot fixes

  • fix(parser): body/QS can be null or omitted in apigw v1/v2 (#820) by @heitorlessa

🔧 Maintenance

  • chore(deps): bump boto3 from 1.20.3 to 1.20.5 (#817) by @dependabot
  • chore(deps): bump boto3 from 1.19.6 to 1.20.3 (#809) by @dependabot
  • chore(deps-dev): bump mkdocs-material from 7.3.5 to 7.3.6 (#791) by @dependabot
  • fix: change supported python version from 3.6.1 to 3.6.2, bump black (#807) by @cakepietoast
  • chore(deps-dev): bump mkdocs-material from 7.3.3 to 7.3.5 (#781) by @dependabot
  • chore(deps-dev): bump flake8-isort from 4.0.0 to 4.1.1 (#785) by @dependabot
  • chore(deps): bump urllib3 from 1.26.4 to 1.26.5 (#787) by @dependabot
  • chore(deps-dev): bump flake8-eradicate from 1.1.0 to 1.2.0 (#784) by @dependabot
  • chore(deps): bump boto3 from 1.18.61 to 1.19.6 (#783) by @dependabot
  • chore(deps-dev): bump pytest-asyncio from 0.15.1 to 0.16.0 (#782) by @dependabot
  • chore(deps-dev): bump coverage from 6.0.1 to 6.0.2 (#764) by @dependabot
  • chore(deps): bump boto3 from 1.18.59 to 1.18.61 (#766) by @dependabot
  • chore(deps-dev): bump mkdocs-material from 7.3.2 to 7.3.3 (#758) by @dependabot
  • chore(deps-dev): bump flake8-comprehensions from 3.6.1 to 3.7.0 (#759) by @dependabot
  • chore(deps): bump boto3 from 1.18.58 to 1.18.59 (#760) by @dependabot
  • chore(deps-dev): bump coverage from 6.0 to 6.0.1 (#751) by @dependabot
  • chore(deps): bump boto3 from 1.18.56 to 1.18.58 (#755) by @dependabot

This release was made possible by the following contributors:

@AlessandroVol23, @BVMiko, @Tankanow, @arthurf1969, @cakepietoast, @dependabot, @dependabot[bot], @eldritchideen, @heitorlessa, @jonemo and @michaelbrewer

Contributors

jonemo, BVMiko, and 8 other contributors
Loading
0 Join discussion

v1.21.1

07 Oct 15:19
@release-drafter release-drafter
v1.21.1
eff5a17
This commit was signed with the committer’s verified signature.
heitorlessa Heitor Lessa
GPG key ID: DA9C8E6177723B7D
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.21.1

Summary

Patch release to address regression in Metrics with mypy not recognizing a Callable when using log_metrics().

New Public Lambda Layers ARNs

Oh! It's finally here!!!

This release adds our first batch of public Lambda Layers for every AWS region supported by AWS Lambda - huge thanks to @am29d.

This means you no longer need to deploy a SAR App in order to use Lambda Powertools as a Lambda Layer.

image

That being said, we will keep SAR App in order to give you the flexibility to choose which semantic version you want to use as a Lambda Layer, until it is officially supported by Lambda Layers.

Changes

📜 Documentation updates

  • docs: add new public layer ARNs (#746) by @am29d

🐛 Bug and hot fixes

  • revert(metrics): typing regression on log_metrics callable (#744) by @heitorlessa

🔧 Maintenance

This release was made possible by the following contributors:

@am29d, @dependabot, @dependabot[bot] and @heitorlessa

Contributors

am29d, heitorlessa, and dependabot
Loading
3 Join discussion

v1.21.0

05 Oct 14:36
@release-drafter release-drafter
v1.21.0
989a4f6
This commit was signed with the committer’s verified signature.
heitorlessa Heitor Lessa
GPG key ID: DA9C8E6177723B7D
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.21.0

Summary

After some vacation period, we're back with a new minor release with major features:

  • Bring your own boto3 sessions for cross-account operations & snapshot testing
  • New features on Feature Flags
  • Idempotency unit testing made easier
  • JSON Schema Validation utility contains new data elements to more easily construct a validation error
  • New utility: we're now exposing our internal custom JMESPath Functions so you can easily decode and deserialize JSON objects found in various formats within Lambda Event Sources.

New Contributors

I'd like to personally thank our new contributors to the project :)

Detailed changes

Boto3 sessions

You can now pass in your own boto3 session when using Parameters, Batch, and Idempotency.

This is helpful in two typical scenarios: 1/ You want to run an operation in another account like fetching secrets/parameters somewhere else, 2/ Use snapshot testing tools like Placebo that will replay session data that happened earlier when doing unit testing.

from aws_lambda_powertools.utilities import parameters
import boto3

boto3_session = boto3.session.Session()
ssm_provider = parameters.SSMProvider(boto3_session=boto3_session)

def handler(event, context):
    # Retrieve a single parameter
    value = ssm_provider.get("/my/parameter")
    ...

Feature flags

There's been three main improvements in Feature flags utility as part of this release: New rule conditions, Bring your own Logger for debugging, and Getting a copy of fetched configuration from the store

New rule conditions

You can now use the following new rule conditions to evaluate your feature flags for inequality, comparison, and more explicit contains logic, where a is the key and b is the value passed as a context input for evaluation:

Action Equivalent expression
KEY_GREATER_THAN_VALUE lambda a, b: a > b
KEY_GREATER_THAN_OR_EQUAL_VALUE lambda a, b: a >= b
KEY_LESS_THAN_VALUE lambda a, b: a < b
KEY_LESS_THAN_OR_EQUAL_VALUE lambda a, b: a <= b
KEY_IN_VALUE lambda a, b: a in b
KEY_NOT_IN_VALUE lambda a, b: a not in b
VALUE_IN_KEY lambda a, b: b in a
VALUE_NOT_IN_KEY lambda a, b: b not in a

Example

Feature flag schema

{
    "premium_features": {
        "default": false,
        "rules": {
            "customer tier equals premium": {
                "when_match": true,
                "conditions": [
                    {
                        "action": "VALUE_IN_KEY",
                        "key": "groups",
                        "value": "PAID_PREMIUM",
                    }
                ]
            }
        }
    },
    "ten_percent_off_campaign": {
        "default": false
    }
}

App

from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore

app_config = AppConfigStore(
    environment="dev",
    application="product-catalogue",
    name="features"
)

feature_flags = FeatureFlags(store=app_config)

def lambda_handler(event, context):
    # groups: ["FREE_TIER", "PAID_BASIC", "PAID_PREMIUM"]
    ctx={"tenant_id": "6", "username": "a", "groups": event.get("groups", [])}

    # Evaluate whether customer's tier has access to premium features
    # based on `has_premium_features` rules
    has_premium_features: bool = feature_flags.evaluate(name="premium_features",
                                                        context=ctx, default=False)
    if has_premium_features:
        # enable premium features
        ...

Accessing raw configuration fetched

Previously, if you were using a single application configuration and a feature schema in a single AppConfig key, we would only use the feature flags schema and discard the rest.

You can now access the raw configuration with a new property get_raw_configuration within AppConfig Store:

from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore

app_config = AppConfigStore(
    environment="dev",
    application="product-catalogue",
    name="configuration",
    envelope = "feature_flags"
)

feature_flags = FeatureFlags(store=app_config)

config = app_config.get_raw_configuration

Unit testing idempotency

We have improved how you can unit test your code when using @idempotent and @idempotent_function decorators.

You can now disable all interactions with the idempotence store using POWERTOOLS_IDEMPOTENCY_DISABLED environment variable, and monkeypatch the DynamoDB resource client Idempotency utility uses if you wish to either use DynamoDB Local or mock all I/O operations.

import boto3

import app

def test_idempotent_lambda():
    # Create our own Table resource using the endpoint for our DynamoDB Local instance
    resource = boto3.resource("dynamodb", endpoint_url='http://localhost:8000')
    table = resource.Table(app.persistence_layer.table_name)
    app.persistence_layer.table = table

    result = app.handler({'testkey': 'testvalue'}, {})
    assert result['payment_id'] == 12345

New data elements for JSON Schema validation errors

When validating input/output with the Validator, you can now access new properties in SchemaValidationError to more easily construct your custom errors based on what went wrong.

Property Type Description
message str Powertools formatted error message
validation_message str, optional Containing human-readable information what is wrong, e.g. data.property[index] must be smaller than or equal to 42
name str, optional name of a path in the data structure, e.g. data.property[index]
path List, optional path as an array in the data structure, e.g. ['data', 'property', 'index']
value Any, optional The invalid value, e.g. {"message": "hello world"}
definition Any, optional JSON Schema definition
rule str, optional rule which the data is breaking (e.g. maximum, required)
rule_definition Any, optional The specific rule definition (e.g. 42, ['message', 'username'])

Sample

image

from aws_lambda_powertools.utilities.validation import validate
from aws_lambda_powertools.utilities.validation.exceptions import SchemaValidationError

import schemas

def handler(event, context):
    try:
        validate(event=event, schema=schemas.INPUT)
    except SchemaValidationError as e:
        message = "data must contain ['message', 'username'] properties"

        assert str(e.value) == e.value.message
        assert e.value.validation_message == message
        assert e.value.name == "data"
        assert e.value.path is not None
        assert e.value.value == data
        assert e.value.definition == schema
        assert e.value.rule == "required"
        assert e.value.rule_definition == schema.get("required")

        raise

    return event

New JMESPath Powertools functions

Last but not least, as part of a documentation revamp in Idempotency by @walmsles, we're now exposing an internal feature used by many Lambda Powertools utilities which is the ability to extract and decode JSON objects.

You can now use JMESPath (JSON Query language) Lambda Powertools functions to easily decode and deserialize JSON often found as compressed (Kinesis, CloudWatch Logs, etc), as strings (SNS, SQS, EventBridge, API Gateway, etc), or as base64 (Kinesis).

We're exposing three custom JMESPath functions you can use such as powertools_json, powertools_base64, powertools_base64_gzip, and a new standalone function that will use JMESPath to search and extract the data you want called extract_data_from_envelope.

Sample

from aws_lambda_powertools.utilities.jmespath_utils import extract_data_from_envelope
from aws_lambda_powertools.utilities.typing import LambdaContext


def handler(event: dict, context: LambdaContext...
Read more

Contributors

gwlester, Tankanow, and 7 other contributors
Loading
0 Join discussion

v1.20.2

02 Sep 18:06
@release-drafter release-drafter
v1.20.2
e54263e
Compare
Choose a tag to compare
v1.20.2

Summary

Bugfix release to address a bug with event handler which caused issues for api gateway events when using strip_prefix together with a root level handler. Thanks @BVMiko for the fix!

Changes

🐛 Bug and hot fixes

🔧 Maintenance

  • chore(deps-dev): bump mkdocs-material from 7.2.4 to 7.2.6 (#665) by @dependabot
  • chore(deps): bump boto3 from 1.18.26 to 1.18.32 (#663) by @dependabot
  • chore(deps-dev): bump pytest from 6.2.4 to 6.2.5 (#662) by @dependabot
  • chore(license): Add THIRD-PARTY-LICENSES to pyproject.toml (#641) by @BastianZim

This release was made possible by the following contributors:

@BVMiko, @BastianZim, @cakepietoast

Contributors

BVMiko, to-mc, and 2 other contributors
Loading

v1.20.1

22 Aug 06:52
@release-drafter release-drafter
v1.20.1
cdef9c4
This commit was signed with the committer’s verified signature.
heitorlessa Heitor Lessa
GPG key ID: DA9C8E6177723B7D
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.20.1

Summary

Hotfix release to address a bug in idempotency hashing logic to ensure data is sorted as part of the process - Huge thanks to @walmsles!

We also now support CodeSpaces to ease contribution, also thanks to @michaelbrewer

Changes

📜 Documentation updates

🐛 Bug and hot fixes

🔧 Maintenance

This release was made possible by the following contributors:

@BastianZim, @heitorlessa and @michaelbrewer

Contributors

walmsles, heitorlessa, and 2 other contributors
Loading
0 Join discussion

v1.20.0

21 Aug 14:17
@release-drafter release-drafter
v1.20.0
27e3930
This commit was signed with the committer’s verified signature.
heitorlessa Heitor Lessa
GPG key ID: DA9C8E6177723B7D
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.20.0

Summary

This release highlights 1/ support for Python 3.9, 2/ support for [API Gateway][apigateway-http-authorizer] and [AppSync Lambda Authorizers][appsync-authorizer], 3/ support for API Gateway Custom Domain Mappings, 4/ support to make [any Python synchronous function idempotent][idempotency_function], and a number of documentation improvements & bugfixes.

Lambda Authorizer support

AppSync

This release adds Data Class support for AppSyncAuthorizerEvent, AppSyncAuthorizerResponse, and correlation ID in Logger.

You can use AppSyncAuthorizerEvent to easily access all self-documented properties, and AppSyncAuthorizerResponse to serialize the response in the expected format.

You can read more in the announcement blog post for more details

from typing import Dict

from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.logging.logger import Logger
from aws_lambda_powertools.utilities.data_classes.appsync_authorizer_event import (
    AppSyncAuthorizerEvent,
    AppSyncAuthorizerResponse,
)
from aws_lambda_powertools.utilities.data_classes.event_source import event_source

logger = Logger()


def get_user_by_token(token: str):
    """Look a user by token"""


@logger.inject_lambda_context(correlation_id_path=correlation_paths.APPSYNC_AUTHORIZER)
@event_source(data_class=AppSyncAuthorizerEvent)
def lambda_handler(event: AppSyncAuthorizerEvent, context) -> Dict:
    user = get_user_by_token(event.authorization_token)

    if not user:
        # No user found, return not authorized
        return AppSyncAuthorizerResponse().to_dict()

    return AppSyncAuthorizerResponse(
        authorize=True,
        resolver_context={"id": user.id},
        # Only allow admins to delete events
        deny_fields=None if user.is_admin else ["Mutation.deleteEvent"],
    ).asdict()

API Gateway

This release adds support for both Lambda Authorizer for payload v1 - APIGatewayAuthorizerRequestEvent, APIGatewayAuthorizerResponse - and v2 formats APIGatewayAuthorizerEventV2, APIGatewayAuthorizerResponseV2.

Similar to AppSync, you can use APIGatewayAuthorizerRequestEvent and APIGatewayAuthorizerEventV2 to easily access all self-documented properties available, and its corresponding APIGatewayAuthorizerResponse and APIGatewayAuthorizerResponseV2 to serialize the response in the expected format.

You can read more in the announcement blog post for more details

v2 format

from aws_lambda_powertools.utilities.data_classes import event_source
from aws_lambda_powertools.utilities.data_classes.api_gateway_authorizer_event import (
    APIGatewayAuthorizerEventV2,
    APIGatewayAuthorizerResponseV2,
)
from secrets import compare_digest


def get_user_by_token(token):
    if compare_digest(token, "Foo"):
        return {"name": "Foo"}
    return None


@event_source(data_class=APIGatewayAuthorizerEventV2)
def handler(event: APIGatewayAuthorizerEventV2, context):
    user = get_user_by_token(event.get_header_value("x-token"))

    if user is None:
        # No user was found, so we return not authorized
        return APIGatewayAuthorizerResponseV2().asdict()

    # Found the user and setting the details in the context
    return APIGatewayAuthorizerResponseV2(authorize=True, context=user).asdict()

v1 format

from aws_lambda_powertools.utilities.data_classes import event_source
from aws_lambda_powertools.utilities.data_classes.api_gateway_authorizer_event import (
    APIGatewayAuthorizerRequestEvent,
    APIGatewayAuthorizerResponse,
    HttpVerb,
)
from secrets import compare_digest


def get_user_by_token(token):
    if compare_digest(token, "admin-foo"):
        return {"isAdmin": True, "name": "Admin"}
    elif compare_digest(token, "regular-foo"):
        return {"name": "Joe"}
    else:
        return None


@event_source(data_class=APIGatewayAuthorizerRequestEvent)
def handler(event: APIGatewayAuthorizerRequestEvent, context):
    user = get_user_by_token(event.get_header_value("Authorization"))

    # parse the `methodArn` as an `APIGatewayRouteArn`
    arn = event.parsed_arn
    # Create the response builder from parts of the `methodArn`
    policy = APIGatewayAuthorizerResponse(
        principal_id="user",
        region=arn.region,
        aws_account_id=arn.aws_account_id,
        api_id=arn.api_id,
        stage=arn.stage
    )

    if user is None:
        # No user was found, so we return not authorized
        policy.deny_all_routes()
        return policy.asdict()

    # Found the user and setting the details in the context
    policy.context = user

    # Conditional IAM Policy
    if user.get("isAdmin", False):
        policy.allow_all_routes()
    else:
        policy.allow_route(HttpVerb.GET, "/user-profile")

    return policy.asdict()

Custom Domain API Mappings

When using Custom Domain API Mappings feature, you must use the new strip_prefixes param in the ApiGatewayResolver constructor.

Scenario: You have a custom domain api.mydomain.dev and set an API Mapping payment to forward requests to your Payments API, the path argument will be /payment/<your_actual_path>.

This will lead to a HTTP 404 despite having your Lambda configured correctly. See the example below on how to account for this change.

from aws_lambda_powertools import Logger, Tracer
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver

tracer = Tracer()
logger = Logger()
app = ApiGatewayResolver(strip_prefixes=["/payment"])

@app.get("/subscriptions/<subscription>")
@tracer.capture_method
def get_subscription(subscription):
    return {"subscription_id": subscription}

@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
@tracer.capture_lambda_handler
def lambda_handler(event, context):
    return app.resolve(event, context)

Make any Python function idempotent

Previously, you could only make the entire Lambda function handler idempotent. You can now make any Python function idempotent with the new [idempotent_function][idempotency_function].

This also enables easy integration with any other utility in Powertools. Take example the Batch utility, where you wouldn't want to make the entire Lambda handler idempotent as the batch will vary, instead you'd want to make sure you can process a given message only once.

As a trade-off to allow any Python function with an arbitrary number of parameters, you must call your function with a keyword argument, and you tell us upfront which one that might be using data_keyword_argument, so we can apply all operations like hashing the idempotency token, payload extraction, parameter validation, etc.

import uuid

from aws_lambda_powertools.utilities.batch import sqs_batch_processor
from aws_lambda_powertools.utilities.idempotency import idempotent_function, DynamoDBPersistenceLayer, IdempotencyConfig


dynamodb = DynamoDBPersistenceLayer(table_name="idem")
config =  IdempotencyConfig(
    event_key_jmespath="messageId",  # see "Choosing a payload subset for idempotency" docs section
    use_local_cache=True,
)

@idempotent_function(data_keyword_argument="data", config=config, persistence_store=dynamodb)
def dummy(arg_one, arg_two, data: dict, **kwargs):
    return {"data": data}


@idempotent_function(data_keyword_argument="record", config=config, persistence_store=dynamodb)
def record_handler(record):
    return {"message": record["body"]}


@sqs_batch_processor(record_handler=record_handler)
def lambda_handler(event, context):
    # `data` parameter must be called as a keyword argument to work
    dummy("hello", "universe", data="test")
    return {"statusCode": 200}

Changes

🌟New features and non-breaking changes

📜 Documentation updates

🐛 Bug and hot fixes

🔧 Maintenance

Read more

Contributors

heitorlessa, michaelbrewer, and 2 other contributors
Loading
0 Join discussion

v1.19.0

11 Aug 06:28
@release-drafter release-drafter
v1.19.0
f627b02
This commit was signed with the committer’s verified signature.
heitorlessa Heitor Lessa
GPG key ID: DA9C8E6177723B7D
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.19.0

Summary

This release highlights 1/ a brand new Feature Flags utility, 2/ auto-disable Tracer in non-Lambda environments to ease unit testing, 3/ API Gateway event handler now supports a custom JSON serializer, and a number of documentation improvements & bugfixes.

We hope you enjoy this new utility as much as we did working on it!!

New Feature Flags utility in Beta

Special thanks to: @risenberg-cyberark, @michaelbrewer, @pcolazurdo and @am29d

You might have heard of feature flags when:

  • Looking to conditionally enable a feature in your application for your customers
  • A/B testing a new feature for a subset of your customers
  • Working with trunk-based development where a feature might not be available right now
  • Working on short-lived features that will only be enabled for select customers

This new utility makes this entire process so much easier by fetching feature flags configuration from AWS AppConfig, and evaluating contextual values against rules you defined to decide whether a feature should be enabled.

Let's dive into the code to better understand what this all means.

Evaluating whether a customer should have access to premium features

from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore

app_config = AppConfigStore(
    environment="dev",
    application="product-catalogue",
    name="features"
)

feature_flags = FeatureFlags(store=app_config)

def lambda_handler(event, context):
    # Get customer's tier from incoming request
    ctx = { "tier": event.get("tier", "standard") }

    has_premium_features: bool = feature_flags.evaluate(name="premium_features",
                                                        context=ctx, default=False)
    if has_premium_features:
        # enable premium features
        ...

Sample feature flag configuration in AWS AppConfig

{
  "premium_features": {
    "default": false,
    "rules": {
      "customer tier equals premium": {
        "when_match": true,
        "conditions": [
          {
            "action": "EQUALS",
            "key": "tier",
            "value": "premium"
          }
        ]
      }
    }
  },
  "ten_percent_off_campaign": {
    "default": false
  }
}

Notice we have premium_features flag that will conditionally be available for premium customers, and a static feature flag named ten_percent_off_campaign that is disabled by default.

Sample invocation event for this function

{
    "username": "lessa",
    "tier": "premium",
    "basked_id": "random_id"
}

There's a LOT going on here. Allow me to break it down:

  1. We're defining a feature flag configuration that is stored in AWS AppConfig
  2. We initialize an AppConfigStore using AWS AppConfig values created via Infrastructure as code (available on docs)
  3. We initialize FeatureFlags and use our previously instantiated AppConfigStore
  4. We call evaluate method and pass the name of the premium feature, along with our contextual information our rules should run against, and a sentinel value to be used in case service errors happen
  5. Feature flags go through the rules defined in premium_features and evaluate whether tier key has the value premium
  6. FeatureFlags returns True which is then stored as has_premium_features variable

But what if you have multiple feature flags and only want all enabled features?

We've got you covered! You can use get_enabled_features to make a single call to AWS AppConfig and return a list of all enabled features at once.

from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver
from aws_lambda_powertools.utilities.feature_flags import FeatureFlags, AppConfigStore

app = ApiGatewayResolver()

app_config = AppConfigStore(
    environment="dev",
    application="product-catalogue",
    name="features"
)

feature_flags = FeatureFlags(store=app_config)


@app.get("/products")
def list_products():
    ctx = {
        **app.current_event.headers,
        **app.current_event.json_body
    }

    # all_features is evaluated to ["geo_customer_campaign", "ten_percent_off_campaign"]
    all_features: list[str] = feature_flags.get_enabled_features(context=ctx)

    if "geo_customer_campaign" in all_features:
        # apply discounts based on geo
        ...

    if "ten_percent_off_campaign" in all_features:
        # apply additional 10% for all customers
        ...

def lambda_handler(event, context):
    return app.resolve(event, context)

But hang on, why Beta?

We want to hear from you on the UX and evaluate how we can make it easier for you to bring your own feature flag store such as Redis, HashiCorp Consul, etc.

When would you use feature flags vs env variables vs Parameters utility?

Environment variables. For when you need simple configuration that will rarely if ever change, because changing it requires a Lambda function deployment.

Parameters utility. For when you need access to secrets, or fetch parameters in different formats from AWS System Manager Parameter Store or Amazon DynamoDB.

Feature flags utility. For when you need static or feature flags that will be enable conditionally based on the input and on a set of rules per feature whether that applies for all customers or on a per customer basis.

In both Parameters and Feature Flags utility you can change their config without having to change your application code.

Changes

Changes

🌟New features and non-breaking changes

  • feat(tracer): auto-disable tracer when for non-Lambda envs (#598) by @michaelbrewer
  • feat(feature-flags): Add not_in action and rename contains to in (#589) by @risenberg-cyberark
  • refactor(feature-flags): add debug for all features evaluation" (#590) by @heitorlessa
  • refactor(feature-flags): optimize UX and maintenance (#563) by @heitorlessa
  • feat(api-gateway): add support for custom serializer (#568) by @michaelbrewer
  • feat(params): expose high level max_age, raise_on_transform_error (#567) by @michaelbrewer
  • feat(data-classes): decode json_body if based64 encoded (#560) by @michaelbrewer

📜 Documentation updates

🐛 Bug and hot fixes

  • fix(feature-flags): bug handling multiple conditions (#599) by @risenberg-cyberark
  • fix(parser): apigw wss validation check_message_id; housekeeping (#553) by @michaelbrewer

🔧 Maintenance

This release was made possible by the following contributors:

@am29d, @dependabot, @dependabot[bot], @dreamorosi, @heitorlessa, @michaelbrewer, @risenberg-cyberark and @pcolazurdo

Contributors

pcolazurdo, am29d, and 5 other contributors
Loading
0 Join discussion