Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Task resource v1 readiness part 2 #3170

Merged
merged 23 commits into from
Nov 14, 2024
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion docs/resources/task.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ resource "snowflake_task" "test_task" {
- `quoted_identifiers_ignore_case` (Boolean) Specifies whether letters in double-quoted object identifiers are stored and resolved as uppercase letters. By default, Snowflake preserves the case of alphabetic characters when storing and resolving double-quoted identifiers (see [Identifier resolution](https://docs.snowflake.com/en/sql-reference/identifiers-syntax.html#label-identifier-casing)). You can use this parameter in situations in which [third-party applications always use double quotes around identifiers](https://docs.snowflake.com/en/sql-reference/identifiers-syntax.html#label-identifier-casing-parameter). For more information, check [QUOTED_IDENTIFIERS_IGNORE_CASE docs](https://docs.snowflake.com/en/sql-reference/parameters#quoted-identifiers-ignore-case).
- `rows_per_resultset` (Number) Specifies the maximum number of rows returned in a result set. A value of 0 specifies no maximum. For more information, check [ROWS_PER_RESULTSET docs](https://docs.snowflake.com/en/sql-reference/parameters#rows-per-resultset).
- `s3_stage_vpce_dns_name` (String) Specifies the DNS name of an Amazon S3 interface endpoint. Requests sent to the internal stage of an account via [AWS PrivateLink for Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/privatelink-interface-endpoints.html) use this endpoint to connect. For more information, see [Accessing Internal stages with dedicated interface endpoints](https://docs.snowflake.com/en/user-guide/private-internal-stages-aws.html#label-aws-privatelink-internal-stage-network-isolation). For more information, check [S3_STAGE_VPCE_DNS_NAME docs](https://docs.snowflake.com/en/sql-reference/parameters#s3-stage-vpce-dns-name).
- `schedule` (String) The schedule for periodically running the task. This can be a cron or interval in minutes. (Conflicts with finalize and after)
- `schedule` (Block List, Max: 1) The schedule for periodically running the task. This can be a cron or interval in minutes. (Conflicts with finalize and after) (see [below for nested schema](#nestedblock--schedule))
sfc-gh-asawicki marked this conversation as resolved.
Show resolved Hide resolved
- `search_path` (String) Specifies the path to search to resolve unqualified object names in queries. For more information, see [Name resolution in queries](https://docs.snowflake.com/en/sql-reference/name-resolution.html#label-object-name-resolution-search-path). Comma-separated list of identifiers. An identifier can be a fully or partially qualified schema name. For more information, check [SEARCH_PATH docs](https://docs.snowflake.com/en/sql-reference/parameters#search-path).
- `statement_queued_timeout_in_seconds` (Number) Amount of time, in seconds, a SQL statement (query, DDL, DML, etc.) remains queued for a warehouse before it is canceled by the system. This parameter can be used in conjunction with the [MAX_CONCURRENCY_LEVEL](https://docs.snowflake.com/en/sql-reference/parameters#label-max-concurrency-level) parameter to ensure a warehouse is never backlogged. For more information, check [STATEMENT_QUEUED_TIMEOUT_IN_SECONDS docs](https://docs.snowflake.com/en/sql-reference/parameters#statement-queued-timeout-in-seconds).
- `statement_timeout_in_seconds` (Number) Amount of time, in seconds, after which a running SQL statement (query, DDL, DML, etc.) is canceled by the system. For more information, check [STATEMENT_TIMEOUT_IN_SECONDS docs](https://docs.snowflake.com/en/sql-reference/parameters#statement-timeout-in-seconds).
Expand Down Expand Up @@ -159,6 +159,15 @@ resource "snowflake_task" "test_task" {
- `parameters` (List of Object) Outputs the result of `SHOW PARAMETERS IN TASK` for the given task. (see [below for nested schema](#nestedatt--parameters))
- `show_output` (List of Object) Outputs the result of `SHOW TASKS` for the given task. (see [below for nested schema](#nestedatt--show_output))

<a id="nestedblock--schedule"></a>
### Nested Schema for `schedule`

Optional:

- `minutes` (Number) Specifies an interval (in minutes) of wait time inserted between runs of the task. Accepts positive integers only.
- `using_cron` (String) Specifies a cron expression and time zone for periodically running the task. Supports a subset of standard cron utility syntax.


<a id="nestedatt--parameters"></a>
### Nested Schema for `parameters`

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@ import (
"errors"
"fmt"
"reflect"
"slices"
"testing"

"github.com/stretchr/testify/assert"

"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk"
)

Expand All @@ -32,42 +33,31 @@ func (t *TaskAssert) HasNotEmptyId() *TaskAssert {
return t
}

func (t *TaskAssert) HasPredecessors(ids ...sdk.SchemaObjectIdentifier) *TaskAssert {
func (t *TaskAssert) HasPredecessorsInAnyOrder(ids ...sdk.SchemaObjectIdentifier) *TaskAssert {
t.AddAssertion(func(t *testing.T, o *sdk.Task) error {
t.Helper()
if len(o.Predecessors) != len(ids) {
return fmt.Errorf("expected %d (%v) predecessors, got %d (%v)", len(ids), ids, len(o.Predecessors), o.Predecessors)
}
var errs []error
for _, id := range ids {
if !slices.ContainsFunc(o.Predecessors, func(predecessorId sdk.SchemaObjectIdentifier) bool {
return predecessorId.FullyQualifiedName() == id.FullyQualifiedName()
}) {
errs = append(errs, fmt.Errorf("expected id: %s, to be in the list of predecessors: %v", id.FullyQualifiedName(), o.Predecessors))
}
if !assert.ElementsMatch(t, ids, o.Predecessors) {
return fmt.Errorf("expected %v predecessors in task relations, got %v", ids, o.TaskRelations.Predecessors)
}
return errors.Join(errs...)
return nil
})
return t
}

func (t *TaskAssert) HasTaskRelations(expected sdk.TaskRelations) *TaskAssert {
t.AddAssertion(func(t *testing.T, o *sdk.Task) error {
t.Helper()
if len(o.TaskRelations.Predecessors) != len(expected.Predecessors) {
return fmt.Errorf("expected %d (%v) predecessors in task relations, got %d (%v)", len(expected.Predecessors), expected.Predecessors, len(o.TaskRelations.Predecessors), o.TaskRelations.Predecessors)
}
var errs []error
for _, id := range expected.Predecessors {
if !slices.ContainsFunc(o.TaskRelations.Predecessors, func(predecessorId sdk.SchemaObjectIdentifier) bool {
return predecessorId.FullyQualifiedName() == id.FullyQualifiedName()
}) {
errs = append(errs, fmt.Errorf("expected id: %s, to be in the list of predecessors in task relations: %v", id.FullyQualifiedName(), o.TaskRelations.Predecessors))
}
errs := make([]error, 0)
if !assert.ElementsMatch(t, o.TaskRelations.Predecessors, expected.Predecessors) {
errs = append(errs, fmt.Errorf("expected %v predecessors in task relations, got %v", expected.Predecessors, o.TaskRelations.Predecessors))
}
if !reflect.DeepEqual(expected.FinalizerTask, o.TaskRelations.FinalizerTask) {
errs = append(errs, fmt.Errorf("expected finalizer task: %v; got: %v", expected.FinalizerTask, o.TaskRelations.FinalizerTask))
}
if expected.FinalizedRootTask != nil {
// This is not supported because we would have to traverse the task graph to find the root task.
errs = append(errs, fmt.Errorf("asserting FinalizedRootTask is not supported"))
}
return errors.Join(errs...)
})
return t
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,27 @@ import (
"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/acceptance/bettertestspoc/assert"
)

func (t *TaskResourceAssert) HasAfterIds(ids ...sdk.SchemaObjectIdentifier) *TaskResourceAssert {
func (t *TaskResourceAssert) HasAfterIdsInOrder(ids ...sdk.SchemaObjectIdentifier) *TaskResourceAssert {
t.AddAssertion(assert.ValueSet("after.#", strconv.FormatInt(int64(len(ids)), 10)))
for i, id := range ids {
t.AddAssertion(assert.ValueSet(fmt.Sprintf("after.%d", i), id.FullyQualifiedName()))
}
return t
}

func (t *TaskResourceAssert) HasScheduleMinutes(minutes int) *TaskResourceAssert {
t.AddAssertion(assert.ValueSet("schedule.#", "1"))
t.AddAssertion(assert.ValueSet("schedule.0.minutes", strconv.Itoa(minutes)))
return t
}

func (t *TaskResourceAssert) HasScheduleCron(cron string) *TaskResourceAssert {
t.AddAssertion(assert.ValueSet("schedule.#", "1"))
t.AddAssertion(assert.ValueSet("schedule.0.using_cron", cron))
return t
}

func (t *TaskResourceAssert) HasNoScheduleSet() *TaskResourceAssert {
t.AddAssertion(assert.ValueSet("schedule.#", "0"))
return t
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,18 @@ func (t *TaskShowOutputAssert) HasTaskRelations(expected sdk.TaskRelations) *Tas
}
return t
}

func (t *TaskShowOutputAssert) HasNoSchedule() *TaskShowOutputAssert {
t.AddAssertion(assert.ResourceShowOutputValueSet("schedule", ""))
return t
}

func (t *TaskShowOutputAssert) HasScheduleMinutes(minutes int) *TaskShowOutputAssert {
t.AddAssertion(assert.ResourceShowOutputValueSet("schedule", fmt.Sprintf("%d MINUTE", minutes)))
return t
}

func (t *TaskShowOutputAssert) HasScheduleCron(cron string) *TaskShowOutputAssert {
t.AddAssertion(assert.ResourceShowOutputValueSet("schedule", fmt.Sprintf("USING CRON %s", cron)))
return t
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

25 changes: 25 additions & 0 deletions pkg/acceptance/bettertestspoc/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,31 @@ func ConfigVariablesFromModel(t *testing.T, model ResourceModel) tfconfig.Variab
return variables
}

// ConfigVariablesFromModels can be used to create a list of objects that are referring to the same resource model.
// It's useful when there's a need to create associations between objects of the same type in Snowflake.
func ConfigVariablesFromModels(t *testing.T, variableName string, models ...ResourceModel) tfconfig.Variables {
sfc-gh-asawicki marked this conversation as resolved.
Show resolved Hide resolved
t.Helper()
allVariables := make([]tfconfig.Variable, 0)
for _, model := range models {
rType := reflect.TypeOf(model).Elem()
rValue := reflect.ValueOf(model).Elem()
variables := make(tfconfig.Variables)
for i := 0; i < rType.NumField(); i++ {
field := rType.Field(i)
if jsonTag, ok := field.Tag.Lookup("json"); ok {
name := strings.Split(jsonTag, ",")[0]
if fieldValue, ok := rValue.Field(i).Interface().(tfconfig.Variable); ok {
variables[name] = fieldValue
}
}
}
allVariables = append(allVariables, tfconfig.ObjectVariable(variables))
}
return tfconfig.Variables{
variableName: tfconfig.ListVariable(allVariables...),
}
}

type nullVariable struct{}

// MarshalJSON returns the JSON encoding of nullVariable.
Expand Down
14 changes: 14 additions & 0 deletions pkg/acceptance/bettertestspoc/config/model/task_model_ext.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,3 +71,17 @@ func (t *TaskModel) WithUserTaskManagedInitialWarehouseSizeEnum(warehouseSize sd
t.UserTaskManagedInitialWarehouseSize = tfconfig.StringVariable(string(warehouseSize))
return t
}

func (t *TaskModel) WithScheduleMinutes(minutes int) *TaskModel {
t.Schedule = tfconfig.MapVariable(map[string]tfconfig.Variable{
"minutes": tfconfig.IntegerVariable(minutes),
})
return t
}

func (t *TaskModel) WithScheduleCron(cron string) *TaskModel {
t.Schedule = tfconfig.MapVariable(map[string]tfconfig.Variable{
"cron": tfconfig.StringVariable(cron),
})
return t
}
5 changes: 1 addition & 4 deletions pkg/acceptance/bettertestspoc/config/model/task_model_gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 11 additions & 0 deletions pkg/resources/resource_helpers_create.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,17 @@ func attributeDirectValueCreate[T any](d *schema.ResourceData, key string, creat
return nil
}

func attributeMappedValueCreate[T any](d *schema.ResourceData, key string, createField **T, mapper func(value any) (*T, error)) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: we can call this in other functions above.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed them, but it seemed to complicate the code (it's easier to understand the above functions the way they are right now). I would leave them as they are right now.

if v, ok := d.GetOk(key); ok {
value, err := mapper(v)
if err != nil {
return err
}
*createField = value
}
return nil
}

func copyGrantsAttributeCreate(d *schema.ResourceData, isOrReplace bool, orReplaceField, copyGrantsField **bool) error {
if isOrReplace {
*orReplaceField = sdk.Bool(true)
Expand Down
17 changes: 17 additions & 0 deletions pkg/resources/resource_helpers_read.go
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,20 @@ func setBooleanStringFromBoolProperty(d *schema.ResourceData, key string, proper
}
return nil
}

func attributeMappedValueReadIfNotEmptyElse[T, R any](d *schema.ResourceData, key string, value *T, mapper func(*T) (R, error), defaultValue any) error {
sfc-gh-asawicki marked this conversation as resolved.
Show resolved Hide resolved
if value != nil {
mappedValue, err := mapper(value)
if err != nil {
return err
}
if err := d.Set(key, mappedValue); err != nil {
return err
}
} else {
if err := d.Set(key, defaultValue); err != nil {
return err
}
}
sfc-gh-jmichalak marked this conversation as resolved.
Show resolved Hide resolved
return nil
}
Loading
Loading