Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Cloud Mappings for Nitric Resources #682

Merged
merged 9 commits into from
Dec 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions dictionary.txt
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,7 @@ preflight
nav
MacOS
quantized
VPC
[0-9]+px
^.+[-:_]\w+$
[a-z]+([A-Z0-9]|[A-Z0-9]\w+)
Expand Down
8 changes: 8 additions & 0 deletions docs/apis.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1308,3 +1308,11 @@ accountsApi.get("/orgs/:id", (ctx) async {
```

</CodeSwitcher>

## Cloud Service Mapping

Each cloud provider comes with a set of default services used when deploying resources. You can find the default services for each cloud provider below.

- [AWS](/providers/mappings/aws/apis)
- [Azure](/providers/mappings/azure/apis)
- [Google Cloud](/providers/mappings/gcp/apis)
10 changes: 10 additions & 0 deletions docs/batch.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -239,3 +239,13 @@ void main() {
```

</CodeSwitcher>

## Cloud Service Mapping

Each cloud provider comes with a set of default services used when deploying resources. You can find the default services for each cloud provider below.

- [AWS](/providers/mappings/aws/batch)
- Azure - Coming soon
- [Google Cloud](/providers/mappings/gcp/batch)

If you need support for additional clouds, let us know by [opening an issue](https://github.com/nitrictech/nitric/issues) or joining the conversation on [Discord](https://nitric.io/chat).
8 changes: 8 additions & 0 deletions docs/keyvalue.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -437,3 +437,11 @@ keys.forEach((String key) {
```

</CodeSwitcher>

## Cloud Service Mapping

Each cloud provider comes with a set of default services used when deploying resources. You can find the default services for each cloud provider below.

- [AWS](/providers/mappings/aws/keyvalue)
- [Azure](/providers/mappings/azure/keyvalue)
- [Google Cloud](/providers/mappings/gcp/keyvalue)
16 changes: 16 additions & 0 deletions docs/messaging.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -702,3 +702,19 @@ await Future.wait(messages.map((message) async {
It's common to ask when to use a queue or a topic. From a publisher's point of view, both queues and topics are almost identical. The difference is primarily on the receiver/subscriber side. Topics push new messages to their subscribers, immediately spinning up workers to process them, while queues rely on the receiver to ask for new messages to process.

For these reasons, we usually default to Topics. Queues are more suitable for batch workloads or situations where there are occasional surges of requests that can be processed at a later time.

## Cloud Service Mapping

Each cloud provider comes with a set of default services used when deploying resources. You can find the default services for each cloud provider below.

### Topics

- [AWS](/providers/mappings/aws/topics)
- [Azure](/providers/mappings/azure/topics)
- [Google Cloud](/providers/mappings/gcp/topics)

### Queues

- [AWS](/providers/mappings/aws/queues)
- [Azure](/providers/mappings/azure/queues)
- [Google Cloud](/providers/mappings/gcp/queues)
2 changes: 1 addition & 1 deletion docs/providers/custom/extend.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ You'll notice that we also override the `Bucket` value to use the pulumi spaces

### Config

Now we can create an extension configuration to allow adding digital ocean configuration to our stack file. You can find the base AWS configuration [here](../pulumi/aws/configuration).
Now we can create an extension configuration to allow adding digital ocean configuration to our stack file. You can find the base AWS configuration [here](/providers/pulumi/aws#stack-configuration).

Start by defining the type of configuration we want. To deploy to digital ocean we require setting a Digital Ocean token as well as a spaces key, secret, and region.

Expand Down
27 changes: 15 additions & 12 deletions docs/providers/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,21 @@ The following is a description of the underlying cloud services that each of the

Currently, both the Pulumi and Terraform providers use the same underlying cloud services to deploy your application. The following table shows the cloud services used by default on each cloud:

| **Resource** | **AWS** | **Azure** | **Google Cloud** | **Local** |
| -------------------------------------- | ----------------------------------------------------------- | -------------------------------------------------- | --------------------------------------------------- | ---------------------------------------------------------------------------------- |
| [APIs](/apis) | [API Gateway](./providers/pulumi/aws/apis) | [API Management](./providers/pulumi/azure/apis) | [API Gateway](./providers/pulumi/gcp/apis) | [Custom](https://github.com/nitrictech/cli/blob/main/pkg/cloud/gateway/gateway.go) |
| [Key Value Stores ](/keyvalue) | [DynamoDB](./providers/pulumi/aws/keyvalue) | [Table Storage](./providers/pulumi/azure/keyvalue) | [FireStore](./providers/pulumi/gcp/keyvalue) | BoltDB |
| [Messaging: Topics](/messaging#topics) | [SNS](./providers/pulumi/aws/topics) | [Event Grid](./providers/pulumi/azure/topics) | [PubSub](./providers/pulumi/gcp/topics) | Custom |
| [Messaging: Queues](/messaging#queues) | [SQS](./providers/pulumi/aws/queues) | [Storage Queues](./providers/pulumi/azure/queues) | [PubSub](./providers/pulumi/gcp/queues) | Custom |
| [Schedules](/schedules) | [CloudWatch Event Bridge](./providers/pulumi/aws/schedules) | [Dapr Binding](./providers/pulumi/azure/schedules) | [Cloud Scheduler](./providers/pulumi/gcp/schedules) | Custom |
| [Secrets](/secrets) | [Secrets Manager](./providers/pulumi/aws/secrets) | [Key Vault](./providers/pulumi/azure/secrets) | [Secret Manager](./providers/pulumi/gcp/secrets) | Custom |
| [Storage](/storage#buckets) | [S3](./providers/pulumi/aws/storage) | [Blob Storage](./providers/pulumi/azure/storage) | [Cloud Storage](./providers/pulumi/gcp/storage) | SeaweedFS |
| Services | Lambda | Container Apps | CloudRun | Docker |

The code is open-source on [GitHub](https://github.com/nitrictech/nitric), so you can see exactly how resources are deployed and handled at runtime, then make any changes you see fit.
| **Resource** | **AWS** | **Azure** | **Google Cloud** | **Local** |
| -------------------------------------- | ------------------------------------------------------------ | --------------------------------------------------- | ---------------------------------------------------- | ---------------------------------------------------------------------------------- |
| [APIs](/apis) | [API Gateway](/providers/mappings/aws/apis) | [API Management](/providers/mappings/azure/apis) | [API Gateway](/providers/mappings/gcp/apis) | [Custom](https://github.com/nitrictech/cli/blob/main/pkg/cloud/gateway/gateway.go) |
| [Batch](/batch) | [AWS Batch](/providers/mappings/aws/batch) | Coming soon | [GCP Batch](/providers/mappings/gcp/batch) | Custom |
| [Schedules](/schedules) | [CloudWatch Event Bridge](/providers/mappings/aws/schedules) | [Dapr Binding](/providers/mappings/azure/schedules) | [Cloud Scheduler](/providers/mappings/gcp/schedules) | Custom |
| [Websockets](/websockets) | [API Gateway](/providers/mappings/aws/websockets) | Not implemented | Not implemented | Custom |
| [Storage](/storage#buckets) | [S3](/providers/mappings/aws/storage) | [Blob Storage](/providers/mappings/azure/storage) | [Cloud Storage](/providers/mappings/gcp/storage) | SeaweedFS |
| [Key Value Stores](/keyvalue) | [DynamoDB](/providers/mappings/aws/keyvalue) | [Table Storage](/providers/mappings/azure/keyvalue) | [FireStore](/providers/mappings/gcp/keyvalue) | BoltDB |
| [SQL Databases](/sql) | [RDS](/providers/mappings/aws/sql) | [Azure Database](/providers/mappings/azure/sql) | [Cloud SQL](/providers/mappings/gcp/sql) | Docker |
| [Messaging: Topics](/messaging#topics) | [SNS](/providers/mappings/aws/topics) | [Event Grid](/providers/mappings/azure/topics) | [PubSub](/providers/mappings/gcp/topics) | Custom |
| [Messaging: Queues](/messaging#queues) | [SQS](/providers/mappings/aws/queues) | [Storage Queues](/providers/mappings/azure/queues) | [PubSub](/providers/mappings/gcp/queues) | Custom |
| [Secrets](/secrets) | [Secrets Manager](/providers/mappings/aws/secrets) | [Key Vault](/providers/mappings/azure/secrets) | [Secret Manager](/providers/mappings/gcp/secrets) | Custom |
| Services | Lambda | Container Apps | CloudRun | Docker |

The code is open-source on [GitHub](https://github.com/nitrictech/nitric), so you can see exactly how resources are deployed and handled at runtime, then make any changes you see fit. Some features may be unimplemented, but any interest or requests are appreciated. Let us know on [Discord](https://nitric.io/chat) or by creating an issue on our [GitHub repository](https://github.com/nitrictech/nitric/issues).

## Custom Providers

Expand Down
29 changes: 29 additions & 0 deletions docs/providers/mappings/aws/batch.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
description: 'How Nitric deploys Batch Jobs to AWS'
---

# AWS Resources - Batch

Nitric Batch Jobs are deployed to AWS using [AWS Batch](https://aws.amazon.com/batch/).

## AWS Resources

The following resources are created when deploying Batch Jobs to AWS:

- AWS Batch Compute Environments
- AWS Batch Job Queues
- AWS Batch Job Definitions
- EC2 Instances
- EC2 Security Groups
- ECR Images
- IAM Policies

## Deployment

During deployment the Nitric CLI deploys your batch jobs:

- Built container images are pushed to the [Amazon Elastic Container Registry](https://aws.amazon.com/ecr/) as private images
- Files referenced by the `batch-services` key in the `nitric.yaml` file are built into Job Definitions
- IAM roles and policies are created
- AWS Batch Compute Environments are created
- AWS Batch Job Queues are created
35 changes: 35 additions & 0 deletions docs/providers/mappings/aws/sql.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
description: 'How Nitric deploys SQL Databases to AWS'
---

# AWS Resources - SQL Database

Nitric SQL Databases are deployed to AWS using [Amazon RDS](https://aws.amazon.com/rds).

## AWS Resources

The following resources are created when deploying SQL Databases to AWS:

- Virtual Private Cloud (VPC)
- Security Group
- Aurora PostgreSQL Cluster
- Cluster Instance
- Subnet Group
- CodeBuild Project for migrations

## Deployment

During deployment the Nitric CLI deploys your SQL Databases:

- Virtual Private Cloud (VPC) and Subnets are created
- Declared databases are deployed as Aurora PostgreSQL Clusters
- Creates new Subnet Groups for the Aurora PostgreSQL Cluster
- Appropriate IAM policies are setup enabling VPC access to the database
- Migration images are built and pushed to the [Amazon Elastic Container Registry](https://aws.amazon.com/ecr/) as private images
- CodeBuild projects are created to run migrations on the database

## Other Engine Support

Currently, Nitric supports only PostgreSQL for AWS RDS databases. Support for other engines like MySQL is not available yet, but you can create a [custom provider](/providers/custom) to add support for your preferred database engine.

If you'd like to see support for additional engines, please share your [feedback](https://github.com/nitrictech/nitric/issues/684) with the Nitric team!
28 changes: 28 additions & 0 deletions docs/providers/mappings/aws/websockets.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
description: 'How Nitric deploys Websockets to AWS'
---

# AWS Resources - Websockets

Nitric Websockets are deployed to AWS using [Amazon API Gateway](https://aws.amazon.com/api-gateway/).

## AWS Resources

The following resources are created when deploying Websockets to AWS:

- API Gateway v2 Websocket APIs
- API Gateway v2 Stages
- API Gateway Integration
- Lambda Functions
- ECR Images
- IAM Policies

## Deployment

During deployment the Nitric CLI builds your Websockets API's routes, methods and handlers:

- Files referenced by the `services` key in the `nitric.yaml` file are built into container images
- Built container images are pushed to the [Amazon Elastic Container Registry](https://aws.amazon.com/ecr/) as private images
- Functions in [AWS Lambda](https://aws.amazon.com/lambda/) are created to run the container images
- IAM policies are created enabling Websocket API Gateway to execute the Lambdas
- `$default`, `$connect`, and `$disconnect` routes are added to the Websocket API Gateway
31 changes: 31 additions & 0 deletions docs/providers/mappings/azure/sql.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
description: 'How Nitric deploys SQL Databases to Azure'
---

# Azure Resources - SQL Database

Nitric SQL Databases are deployed to Azure using [Azure Database for PostgreSQL](https://azure.microsoft.com/en-us/products/postgresql).

## Azure Resources

The following resources are created when deploying SQL Databases to Azure:

- Virtual Network
- Subnet
- PostgreSQL Databases
- Container Instance Group for migrations

## Deployment

During deployment the Nitric CLI deploys your SQL Databases:

- Virtual Network and Subnet are created
- Declared databases are deployed as PostgreSQL databases
- Container Group Instances are created to run migrations
- Container Apps are setup to run the containers

## Other Engine Support

Currently, Nitric supports only PostgreSQL for Azure SQL databases. Support for other engines like MySQL is not available yet, but you can create a [custom provider](/providers/custom) to add support for your preferred database engine.

If you'd like to see support for additional engines, please share your [feedback](https://github.com/nitrictech/nitric/issues/684) with the Nitric team!
27 changes: 27 additions & 0 deletions docs/providers/mappings/gcp/batch.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
description: 'How Nitric deploys Batch Jobs to Google Cloud'
---

# Google Cloud Resources - Batch

Nitric Batch Jobs are deployed to Google Cloud using [Batch](https://cloud.google.com/batch).

## Google Cloud Resources

The following resources are created when deploying Batch Jobs to Google Cloud:

- Google Cloud Batch Compute Environments
- Google Cloud Batch Job Queues
- Google Cloud Batch Job Definitions
- EC2 Instances
- EC2 Security Groups
- ECR Images
- IAM Policies

## Deployment

During deployment the Nitric CLI deploys your batch jobs:

- Built container images are pushed to the [Amazon Elastic Container Registry](https://aws.amazon.com/ecr/) as private images
- Files referenced by the `batch-services` key in the `nitric.yaml` file are built into Job Definitions
- IAM policies are created
32 changes: 32 additions & 0 deletions docs/providers/mappings/gcp/sql.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
description: 'How Nitric deploys SQL Databases to Google Cloud'
---

# Google Cloud Resources - SQL Database

Nitric SQL Databases are deployed to Google Cloud using [Cloud SQL](https://cloud.google.com/sql).

## Google Cloud Resources

The following resources are created when deploying SQL Databases to Google Cloud:

- Private Network
- Subnet
- PostgreSQL Databases
- Cloud Run Jobs for migrations

## Deployment

During deployment the Nitric CLI deploys your SQL Databases:

- Private Network is created for the Google project
- Subnet is created for the network
- Declared databases are deployed as PostgreSQL databases
- Migration images are built and deployed to the artifact registry
- Migrations are run against the deployed databases using cloud run jobs

## Other Engine Support

Currently, Nitric supports only PostgreSQL for Google Cloud SQL databases. Support for other engines like MySQL is not available yet, but you can create a [custom provider](/providers/custom) to add support for your preferred database engine.

If you'd like to see support for additional engines, please share your [feedback](https://github.com/nitrictech/nitric/issues/684) with the Nitric team!
Loading
Loading