New Resource: Timestream Scheduled Query

[nmagrogan]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

A new feature for aws timestream was anounced at re:Invent 2021 called Scheduled Queries.

The current provider does not yet support creation of scheduled queries. Terraform Docs

AWS example on how to create a scheduled query using the SKD. Here

Forgive me for any mistakes in this request, this is my first time reporting an issue. 🙂

New or Affected Resource(s)

• aws_timestreamquery or aws_timestreamwrite_scheduled_query (I'm not sure which name should be used. First is consistent with AWS SDK, second is consistent with current Terraform Timestream resources)

Potential Terraform Configuration

resource "aws_timestreamquery" "scheduled_query" {
  query_name      = <name of your query>
  query_statement = <SQL string of your query (not sure on the best format for this)>

  desination_table_schema = <JSON mapping of query's resulting columns to: data type, attribute type, and column name (again not sure on the best format for this)>

  run_schedule = <cron string or fixed rate>

   # Database and table separately created resources.
  target_database_name = aws_timestreamwrite_database.database.database_name
  target_table_name    = aws_timestreamwrite_table.table.table_name

  # Other resources that need to be created sepratly for a scheduled query
  kms_key_id             = aws_kms_key.example.arn
  iam_role               = aws_iam_role.example.arn
  sns_notification_topic = aws_sns_topic.example.arn
  error_logs_bucket      = aws_s3_bucket.example.bucket

  tags = {
    tag = "example"
  }
}

References

• https://docs.aws.amazon.com/timestream/latest/developerguide/scheduledqueries.html
• https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/timestreamwrite_table
• https://github.com/awslabs/amazon-timestream-tools/blob/mainline/sample_apps_reinvent2021/goV2/scheduled-query-sample.go
• https://docs.aws.amazon.com/timestream/latest/developerguide/API_query_CreateScheduledQuery.html

[prullanferragut]

How hard would this task be for a first-timer?

[sysadmin1139]

Could look like...

resource "aws_timestreamquery_create" {
  scheduled_query_name = "example-rollup"
  query_string         = "SELECT [...] GROUP BY bin(time, 1m), service"
  schedule_expression  = "cron(0/5 * * * ? *)"
  execution_role_arn   = aws_iam_role.example-rollup.arn

  dimension_mapping = [
    {
      name       = "service"
      value_type = "varchar"
    }
  ]

  multi_measure_mappings = {
    target_measure_name = "1m"
    attribute_mappings = [
      {
        source_column = "max"
        value_type    = "DOUBLE"
      },
        source_column = "min"
        value_type    = "DOUBLE
     }
    ]
  }

  notification_configuration {
    sns_configuration {
      sns_topic = aws_sns_topic.example-rollup-results.arn
    }
  }

  error_configuration {
    s3_configuration {
      bucket_name = aws_s3_bucket.example-rollup.id
      object_key_prefix = "1m/"
    }
  }
} 

The API itself has support for expanding the error-report and notification-configuration blocks for additional types.

[jelazos7]

Our company has just started using timestream, including scheduled queries, and we are a big fan of infrastructure as code. I've noticed this hasn't been picked up yet, and I'm happy to make a contribution!

I'm brand new at contributing to the terraform aws provider, brand new at Go, and largely brand new at open source contributing, but will do my best 😅 . But please stop me if you know of someone else doing this work.

I'm following the contribution guidelines, and created a fork here: https://github.com/jelazos7/terraform-provider-aws/tree/f-timestream-sched-query

As for the resource name, i saw that was a question -- I'm thinking "aws_timestreamquery_scheduled_query" looks to be the most fitting, based on the way the sdk is structured (here's the javascript one for example: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/TimestreamQuery.html#createScheduledQuery-property)

I've made a request to join slack, so I won't be so cagey about development progress.

[StuAtGit]

FYI for anyone else that gets mislead by this bug... this already exists:

https://registry.terraform.io/providers/hashicorp/awscc/latest/docs/resources/timestream_scheduled_query

FYI @jelazos7

[jelazos7]

Darn, someone beat me to it. Awesome though, thanks!!

…

On Wed, Jan 17, 2024, 4:36 PM Stu S ***@***.***> wrote: FYI for anyone else that gets mislead by this bug... this already exists: https://registry.terraform.io/providers/hashicorp/awscc/latest/docs/resources/timestream_scheduled_query FYI @jelazos7 <https://github.com/jelazos7> — Reply to this email directly, view it on GitHub <#22507 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJ5EPDNYUQE56BQXOFLYBTYPA76FAVCNFSM5LUKEGR2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBZGY4DMNJVG42A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kevdonde]

d@jelazos7 , @StuAtGit

https://registry.terraform.io/providers/hashicorp/awscc/latest/docs/resources/timestream_scheduled_query. The awscc provider which this is part of is in technical preview and not recommended for production workloads, according to https://developer.hashicorp.com/terraform/tutorials/aws/aws-cloud-control?in=terraform%2Faws .

Ideally, the resource for timestream scheduled queries would be added to both providers, at least while the awscc provider is in technical preview.

[github-actions]

Warning

This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

[github-actions]

This functionality has been released in v5.85.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!