Skip to content

Commit

Permalink
Merge pull request #4 from estigma88/security-no-lambda
Browse files Browse the repository at this point in the history 
Security no lambda
  • Loading branch information
@estigma88
estigma88 authored Sep 27, 2020
2 parents f664a36 + ddc2485 commit c665e64
Show file tree
Hide file tree
Showing 44 changed files with 1,120 additions and 337 deletions.
162 changes: 162 additions & 0 deletions README.MD
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# Social Media Publisher

Publisher of social media posts using AWS Free Tier with Lambdas, DynamoDB and scheduled CloudWatch events.

## The Problem

Authors of blogs, books or ideas usually like to share their content on social media like Twitter, Linkedin, Facebook and so on, to create engagement and followers.

However, this process is tedious, as you need to go to each social media account and publish your content, so, imagine you want to publish everyday, three times through the day, in your 4 social media accounts, that means, you will
need to connect 4 x 3 = 12 times at day to the social media accounts. This is time you loose you can invest in creating your content.

Applications in the market exists to automate this process, where you schedule posts to different social media, to be published in a future time like Crowdfire. These applications have limitations, like amount of social media accounts, amount of scheduled posts and few analytics regarding your content.

If you want more features, you will need to pay for it.

I decided to build my own. For more details about why and how, you can check my post [How I Built a Social Media Publisher on AWS for Free](https://coderstower.com/2020/09/28/how-i-built-a-social-media-publisher-on-aws-for-free/).

## The Solution

A lambda function who reads posts saved somewhere and publish them in a defined time and rate.

Technologies:
* Spring Boot
* Spring Security
* Spring MVC
* Spring Repositories
* AWS Lambda
* AWS Dynamodb
* AWS API Gateway
* AWS Cloudwatch

The AWS technologies stay in the forever free services, which means, you won't be charged for deploying this application to production.

## Installation
You need to create a AWS infrastructure to deploy the app as follows.

### Social Media Support

The following are the social media supports:

* Linkedin: You need to create a client app (OAuth2) [here](https://www.linkedin.com/developers/).
* Twitter : You need to create a client app (OAuth1.1) [here](https://developer.twitter.com/en/apps).

### AWS Dynamodb

AWS Dynamodb is a documental database. You should create the following tables:

#### OAuth1Credentials

Saves the OAuth1 credentials. This is used by Twitter API.

You need to create a record with id "twitter" and the below information can be obteined from the Twitter app you created. Th OAuth1 credentials are static, they won't expire. The following is an example:

```json
{
"accessToken": "xxx",
"consumerKey": "xxx",
"consumerSecret": "xxx",
"id": "twitter",
"tokenSecret": "xxx"
}
```

#### OAuth2Credentials

Saves the OAuth2 credentials. This is used by Linkedin API.

You need to create a record with id "linkedin" and the below information will be updated using the Solcial Media Publisher app after you deploy it to production. You will see how in the following sections.

The OAuth2 credentials expire, so, you will need to update them before it expires. The Social Media Publisher app will help you. The following is an example:

```json
{
"accessToken": "xxx",
"expirationDate": "2020-05-28T00:00:00",
"id": "linkedin"
}
```

#### Posts

Saves the Posts you want to publish. The following is an example:

```json
{
"description": "Hi everyone, do you really understand what Dependency Inversion is? you should, and you should use it. In this post, I talk about it.",
"id": "1",
"name": "Dependency Inversion: why you should NOT avoid it",
"publishedDate": "2020-01-18T00:00:00",
"tags": [
"coderstower", "java", "solid", "oop", "dependencyinversion" ], "url": "https://coderstower.com/2019/03/26/dependency-inversion-why-you-shouldnt-avoid-it/"
}
```
### AWS Lambda
You should grab the latest release of the Social Media Publisher app and deployed to a Java lambda function.
You will need to add the following variables as environment variables:

- `spring.profiles.active`: You can activate `twitter` or `linkedin`, or both.

You will need to build a JSON with those environment variables and set the AWS Lambda variable to SPRING_APPLICATION_JSON. A JSON example is shown below:

```json
{
"spring": {
"profiles": {
"active": "linkedin,twitter"
}
}
}
```

Also, the handler method for the Lambda should be `com.coderstower.socialmediapubisher.springpublisher.main.aws.StreamLambdaHandler::handleRequest`

Besides, as destination, you should connect the Lambda with AWS SNS, so, if the Lambda execution suceed or fail, you will get an email.

### AWS Cloudwatch
You will use Cloudwatch to schedule when you want to publish the posts to the registered social networks. So, you need to create a event, with a cron expresion configuration, for instance `0 18 ? * SUN,WED,FRI *` which means the posts will be publish each Sunday, Wednesday and Friday, at 18 hours UTC.

As target, you will add the AWS Lambda you created before, sending a fake gateway event to simulate calling the endping `/posts/next`. You can find a fake gateway event in the AWS Lambda console, when you create tests requests, you can create a Gateway one. The JSON generated could be used from AWS Cloudwatch.

## Updating OAuth2 Credentials
OAuth2 credentials expire, so, we need to refresh them frequently.

This process is handle by the Social Media Publisher application, but, not deployed in AWS, instead, running locally in your machine. The following are the steps:

1. Login to AWS using the `aws-cli configure` operation.
2. Setup the necessary properties in the application-secure.yml file

- `social-media-publisher.principal-names-allowed.linkedin`: This is your unique Linkedin id.
- `spring.security.oauth2.client.registration.linkedin.client-id`: This is the Linkedin cliend id. You can find the value in the Linkedin app you created before.
- `spring.security.oauth2.client.registration.linkedin.client-secret`: This is the Linkedin cliend secret. You can find the value in the Linkedin app you created before.

3. Start the application using the AWSSpringPublisherApplication class and the secure spring profile active:
- `spring.profiles.active`: You must activate `secure` profile.

4. Access the url `http://localhost:8080/oauth2/linkedin/credentials`. The application will redirect to login on LinkedIn, and after you set your credentials, the OAuth2 credentials for linkedin will be updated in DynamoDB

## Local Testing
You can run the application locally and connect it to a local DynamoDB as follows:

1. Run AWS Dynamodb locally: `docker run -p 8000:8000 amazon/dynamodb-local -jar DynamoDBLocal.jar -inMemory -sharedDb`
2. Create the tables locally:

- `aws dynamodb create-table --table-name OAuth1Credentials --attribute-definitions AttributeName=id,AttributeType=S --key-schema AttributeName=id,KeyType=HASH --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 --endpoint-url http://localhost:8000`
- `aws dynamodb create-table --table-name OAuth2Credentials --attribute-definitions AttributeName=id,AttributeType=S --key-schema AttributeName=id,KeyType=HASH --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 --endpoint-url http://localhost:8000`
- `aws dynamodb create-table --table-name Posts --attribute-definitions AttributeName=id,AttributeType=S --key-schema AttributeName=id,KeyType=HASH --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 --endpoint-url http://localhost:8000`

3. Check the tables were created: `aws dynamodb list-tables --endpoint-url http://localhost:8000`
4. Start the application using the AWSSpringPublisherApplication class and the secure spring profile active:
- `spring.profiles.active`: You must activate `local` profile. If you want to do a full test with your social media accounts, add `local,linkedin,twitter`

## Releases

You can find the current releases [here](https://github.com/estigma88/social-media-publisher/releases)

## Contributing
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

## License
[MIT](https://choosealicense.com/licenses/mit/)
108 changes: 0 additions & 108 deletions aws-publisher/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,108 +0,0 @@
# aws-publisher serverless API
The aws-publisher project, created with [`aws-serverless-java-container`](https://github.com/awslabs/aws-serverless-java-container).

The starter project defines a simple `/ping` resource that can accept `GET` requests with its tests.

The project folder also includes a `sam.yaml` file. You can use this [SAM](https://github.com/awslabs/serverless-application-model) file to deploy the project to AWS Lambda and Amazon API Gateway or test in local with [SAM Local](https://github.com/awslabs/aws-sam-local).

Using [Maven](https://maven.apache.org/), you can create an AWS Lambda-compatible zip file simply by running the maven package command from the project folder.
```bash
$ mvn archetype:generate -DartifactId=aws-publisher -DarchetypeGroupId=com.amazonaws.serverless.archetypes -DarchetypeArtifactId=aws-serverless-springboot-archetype -DarchetypeVersion=1.4 -DgroupId=my.service -Dversion=1.0-SNAPSHOT -Dinteractive=false
$ cd aws-publisher
$ mvn clean package

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.546 s
[INFO] Finished at: 2018-02-15T08:39:33-08:00
[INFO] Final Memory: XXM/XXXM
[INFO] ------------------------------------------------------------------------
```

You can use [AWS SAM Local](https://github.com/awslabs/aws-sam-local) to start your project.

First, install SAM local:

```bash
$ npm install -g aws-sam-local
```

Next, from the project root folder - where the `sam.yaml` file is located - start the API with the SAM Local CLI.

```bash
$ sam local start-api --template sam.yaml

...
Mounting com.amazonaws.serverless.archetypes.StreamLambdaHandler::handleRequest (java8) at http://127.0.0.1:3000/{proxy+} [OPTIONS GET HEAD POST PUT DELETE PATCH]
...
```

Using a new shell, you can send a test ping request to your API:

```bash
$ curl -s http://127.0.0.1:3000/ping | python -m json.tool

{
"pong": "Hello, World!"
}
```

You can use the [AWS CLI](https://aws.amazon.com/cli/) to quickly deploy your application to AWS Lambda and Amazon API Gateway with your SAM template.

You will need an S3 bucket to store the artifacts for deployment. Once you have created the S3 bucket, run the following command from the project's root folder - where the `sam.yaml` file is located:

```
$ aws cloudformation package --template-file sam.yaml --output-template-file output-sam.yaml --s3-bucket <YOUR S3 BUCKET NAME>
Uploading to xxxxxxxxxxxxxxxxxxxxxxxxxx 6464692 / 6464692.0 (100.00%)
Successfully packaged artifacts and wrote output template to file output-sam.yaml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file /your/path/output-sam.yaml --stack-name <YOUR STACK NAME>
```

As the command output suggests, you can now use the cli to deploy the application. Choose a stack name and run the `aws cloudformation deploy` command from the output of the package command.

```
$ aws cloudformation deploy --template-file output-sam.yaml --stack-name ServerlessSpringApi --capabilities CAPABILITY_IAM
```

Once the application is deployed, you can describe the stack to show the API endpoint that was created. The endpoint should be the `ServerlessSpringApi` key of the `Outputs` property:

```
$ aws cloudformation describe-stacks --stack-name ServerlessSpringApi
{
"Stacks": [
{
"StackId": "arn:aws:cloudformation:us-west-2:xxxxxxxx:stack/ServerlessSpringApi/xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx",
"Description": "AWS Serverless Spring API - com.amazonaws.serverless.archetypes::aws-serverless-springboot-archetype",
"Tags": [],
"Outputs": [
{
"Description": "URL for application",
"ExportName": "AwsPublisherApi",
"OutputKey": "AwsPublisherApi",
"OutputValue": "https://xxxxxxx.execute-api.us-west-2.amazonaws.com/Prod/ping"
}
],
"CreationTime": "2016-12-13T22:59:31.552Z",
"Capabilities": [
"CAPABILITY_IAM"
],
"StackName": "ServerlessSpringApi",
"NotificationARNs": [],
"StackStatus": "UPDATE_COMPLETE"
}
]
}
```

Copy the `OutputValue` into a browser or use curl to test your first request:

```bash
$ curl -s https://xxxxxxx.execute-api.us-west-2.amazonaws.com/Prod/ping | python -m json.tool

{
"pong": "Hello, World!"
}
```
10 changes: 8 additions & 2 deletions aws-publisher/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<modelVersion>4.0.0</modelVersion>

<artifactId>aws-publisher</artifactId>
<version>1.0-SNAPSHOT</version>
<version>0.3.0-SNAPSHOT</version>
<packaging>jar</packaging>

<name>aws-publisher</name>
Expand All @@ -28,7 +28,7 @@
<dependency>
<groupId>com.amazonaws.serverless</groupId>
<artifactId>aws-serverless-java-container-springboot2</artifactId>
<version>1.4</version>
<version>1.5</version>
</dependency>

<dependency>
Expand Down Expand Up @@ -65,6 +65,12 @@
<version>0.5.1</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

<build>
Expand Down
16 changes: 7 additions & 9 deletions ...derstower/socialmediapubisher/springpublisher/main/aws/AWSSpringPublisherApplication.java
View file
Original file line number Diff line number Diff line change
@@ -1,23 +1,21 @@
package com.coderstower.socialmediapubisher.springpublisher.main.aws;

import com.coderstower.socialmediapubisher.springpublisher.main.controller.ErrorHandler;
import com.coderstower.socialmediapubisher.springpublisher.main.controller.OAuth2CredentialsController;
import com.coderstower.socialmediapubisher.springpublisher.main.controller.PostsController;
import com.coderstower.socialmediapubisher.springpublisher.main.factory.SpringPublisherFactory;
import com.coderstower.socialmediapubisher.springpublisher.main.factory.SecurityFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.servlet.support.SpringBootServletInitializer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Import;
import org.springframework.web.servlet.HandlerExceptionResolver;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;


@SpringBootApplication
// We use direct @Import instead of @ComponentScan to speed up cold starts
// @ComponentScan(basePackages = "my.service.controller")
@Import({PostsController.class, SpringPublisherFactory.class, SpringPublisherDynamoDBRepositoryFactory.class})
@Import({PostsController.class, OAuth2CredentialsController.class, SecurityFactory.class,
SpringPublisherFactory.class, SpringPublisherDynamoDBRepositoryFactory.class, ErrorHandler.class})
public class AWSSpringPublisherApplication extends SpringBootServletInitializer {

/*
Expand Down Expand Up @@ -45,7 +43,7 @@ public HandlerAdapter handlerAdapter() {
*
* To enable custom @ControllerAdvice classes remove this bean.
*/
@Bean
/*@Bean
public HandlerExceptionResolver handlerExceptionResolver() {
return new HandlerExceptionResolver() {
Expand All @@ -54,7 +52,7 @@ public ModelAndView resolveException(HttpServletRequest request, HttpServletResp
return null;
}
};
}
}*/

public static void main(String[] args) {
SpringApplication.run(AWSSpringPublisherApplication.class, args);
Expand Down
Loading

0 comments on commit c665e64

Please sign in to comment.