Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RESOURCE-134 F/pipeline runs #514

Merged
merged 12 commits into from
Feb 17, 2022
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,8 @@ The following is a list of static resources.
- [azure_data_factory](docs/resources/azure_data_factory.md)
- [azure_data_factory_linked_service](docs/resources/azure_data_factory_linked_service.md)
- [azure_data_factory_linked_services](docs/resources/azure_data_factory_linked_services.md)
- [azure_data_factory_pipeline_run_resource](docs/resources/azure_data_factory_pipeline_run_resource.md)
- [azure_data_factory_pipeline_run_resources](docs/resources/azure_data_factory_pipeline_run_resources.md)
- [azure_db_migration_service](docs/resources/azure_db_migration_service.md)
- [azure_db_migration_services](docs/resources/azure_db_migration_services.md)
- [azure_ddos_protection_resource](docs/resources/azure_ddos_protection_resource.md)
Expand Down
91 changes: 91 additions & 0 deletions docs/resources/azure_data_factory_pipeline_run_resource.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
title: About the azure_data_factory_pipeline_run_resource Resource
platform: azure
---

# azure_data_factory_pipeline_run_resource

Use the `azure_data_factory_pipeline_run_resource` InSpec audit resource to test the properties of an Azure Data Factory pipeline run.

For additional information, see the [`Azure Data Factory pipeline runs API documentation`](https://docs.microsoft.com/en-us/rest/api/datafactory/pipeline-runs/query-by-factory).

## Azure REST API Version, Endpoint, and HTTP Client Parameters

This resource interacts with API versions supported by the resource provider. The `api_version` is defined as a resource parameter.
If not provided, the latest version is used. For more information, refer to [`azure_generic_resource`](azure_generic_resource.md).

Unless defined, `azure_cloud` global endpoint and default values for the HTTP client is used. For more information, refer to the resource pack [README](../../README.md).

## Availability

### Installation

This resource is available in the [InSpec Azure resource pack](https://github.com/inspec/inspec-azure). For an example, `inspec.yml` file and how to set up your Azure credentials, refer to resource pack [README](../../README.md#Service-Principal).

## Syntax

```ruby
describe azure_data_factory_pipeline_run_resource(resource_group: `RESOURCE_GROUP`, factory_name: `FACTORY_NAME`, run_id: `RUN_ID`) do
#...
end
```

## Parameters

`resource_group` _(required)_

Azure resource group that the targeted resource resides in.

`factory_name` _(required)_

The factory name.

`run_id` _(required)_

The name of the pipeline runs.

## Properties

| Name | Description |
|--------------------------------|----------------------------------------------------------------------------------|
| invokedBy.name | The unique resource names. |
| pipelineName | The pipeline name. |
| status | The status of a pipeline run. |
| runId | Identifiers of a run. |
| runStart | Start time of a pipeline run in ISO8601 format. |
| runEnd | End time of a pipeline run in ISO8601 format. |
| runStart | The properties of the resource. |

## Examples

### Test properties of a pipeline runs

```ruby
describe azure_data_factory_pipeline_run_resource(resource_group: `RESOURCE_GROUP`, name: 'FACTORY_NAME', run_id: `RUN_ID`) do
its('invokedBy.name') { should include 'INVOKED_BY_NAME' }
its('pipelineNames') { should include 'PIPELINE_NAME' }
its('status') { should include 'PIPELINE_STATUS' }
end
```

## Matchers

### Test that a pipeline runs exists

```ruby
describe azure_data_factory_pipeline_run_resource(resource_group: `RESOURCE_GROUP`, factory_name: `FACTORY_NAME`, run_id: `RUN_ID`) do
it { should exist }
end
```

### Test that a pipeline runs does not exist

```ruby
describe azure_data_factory_pipeline_run_resource(resource_group: `RESOURCE_GROUP`, factory_name: `FACTORY_NAME`, run_id: 'RUN_ID') do
it { should_not exist }
end
```

## Azure Permissions

Your [Service Principal](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal) must be set up with a `contributor` role on the subscription you wish to test.
91 changes: 91 additions & 0 deletions docs/resources/azure_data_factory_pipeline_run_resources.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
title: About the azure_data_factory_pipeline_run_resources Resource
platform: azure
---

# azure_data_factory_pipeline_run_resources

Use the `azure_data_factory_pipeline_run_resources` InSpec audit resource to test the properties multiple Azure Data Factory pipeline runs for a resource group or the entire subscription.

For additional information, see the [`Azure Data Factory pipeline runs API documentation`](https://docs.microsoft.com/en-us/rest/api/datafactory/pipeline-runs/query-by-factory).

## Azure REST API Version, Endpoint, and HTTP Client Parameters

This resource interacts with API versions supported by the resource provider. The `api_version` is defined as a resource parameter. If not provided, the latest version is used. For more information, refer to [`azure_generic_resource`](azure_generic_resource.md).

Unless defined, `azure_cloud` global endpoint and default values for the HTTP client is used. For more information, refer to the resource pack [README](../../README.md).

## Availability

### Installation

This resource is available in the [InSpec Azure resource pack](https://github.com/inspec/inspec-azure). For an example, `inspec.yml` file and how to set up your Azure credentials, refer to resource pack [README](../../README.md#Service-Principal).

## Syntax

An `azure_data_factory_pipeline_run_resources` resource block returns all Azure Data Factory pipeline runs.

```ruby
describe azure_data_factory_pipeline_run_resources(resource_group: `RESOURCE_GROUP`, factory_name: 'FACTORY_NAME') do
#...
end
```

## Parameters

`resource_group` _(required)_

Azure resource group that the targeted resource resides in.

`factory_name` _(required)_

Azure factory name for which pipeline runs are retrieved.


## Properties

| Property | Description | Filter Criteria<superscript>*</superscript> |
|-----------------|---------------------------------------------------------|-----------------------|
| invokedBy_names | A list of the unique resource names. | `invokedBy_name` |
| pipelineNames | A list of the pipeline names. | `pipelineName` |
| statuses | The statuses of the pipeline runs. | `status` |
| runIds | The list of identifiers of runs. | `runId` |
| runStart | The list of start times of pipeline runs in ISO8601 format. | `runStart` |
| runEnd | The list of end times of pipeline runs in ISO8601 format. | `runEnd` |

<superscript>*</superscript> For information on how to use filter criteria on plural resources, refer to [FilterTable usage](https://github.com/inspec/inspec/blob/master/dev-docs/filtertable-usage.md).

## Examples

### Filter pipeline runs in a resource group by properties

```ruby
describe azure_data_factory_pipeline_run_resources(resource_group: `RESOURCE_GROUP`, factory_name: 'FACTORY_NAME') do
its('invokedBy_names') { should include 'INVOKED_BY_NAME' }
its('pipelineNames') { should include 'PIPELINE_NAME' }
its('statuses') { should include 'PIPELINE_STATUS' }
end
```

## Matchers

### Test if any pipeline runs exist in the resource group

```ruby
describe azure_data_factory_pipeline_run_resources(resource_group: `RESOURCE_GROUP`, factory_name: 'FACTORY_NAME') do
it { should exist }
end
```

### Test that there aren't any pipeline runs in a resource group

```ruby
# Should not exist if no pipeline runs are in the resource group
describe azure_data_factory_pipeline_run_resources(resource_group: `RESOURCE_GROUP`, factory_name: 'FACTORY_NAME') do
it { should_not exist }
end
```

## Azure Permissions

Your [Service Principal](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal) must be set up with a `contributor` role on the subscription you wish to test.
27 changes: 27 additions & 0 deletions libraries/azure_data_factory_pipeline_run_resource.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
require 'azure_generic_resource'

class AzureDataFactoryPipelineRunResource < AzureGenericResource
name 'azure_data_factory_pipeline_run_resource'
desc 'Verifies settings for an Azure DataSet'
example <<-EXAMPLE
describe azure_data_factory_pipeline_run_resource(resource_group: 'example',factory_name: 'factory_name', run_id: 'run_id') do
it { should exists }
end
EXAMPLE

def initialize(opts = {})
# Options should be Hash type. Otherwise Ruby will raise an error when we try to access the keys.
raise ArgumentError, 'Parameters must be provided in an Hash object.' unless opts.is_a?(Hash)

opts[:resource_provider] = specific_resource_constraint('Microsoft.DataFactory/factories', opts)
opts[:required_parameters] = %i(factory_name)
opts[:resource_path] = [opts[:factory_name], 'pipelineruns'].join('/')
opts[:resource_identifiers] = %i(run_id)
# static_resource parameter must be true for setting the resource_provider in the backend.
super(opts, true)
end

def to_s
super(AzureDataFactoryPipelineRunResource)
end
end
61 changes: 61 additions & 0 deletions libraries/azure_data_factory_pipeline_run_resources.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
require 'azure_generic_resources'

class AzureDataFactoryPipelineRunResources < AzureGenericResources
name 'azure_data_factory_pipeline_run_resources'
desc 'Lists DataFactoryDataSets'
example <<-EXAMPLE
azure_data_factory_pipeline_run_resources(resource_group: 'example', factory_name: 'factory_name') do
it{ should exist }
end
EXAMPLE

attr_reader :table

def initialize(opts = {})
# Options should be Hash type. Otherwise Ruby will raise an error when we try to access the keys.
raise ArgumentError, 'Parameters must be provided in an Hash object.' unless opts.is_a?(Hash)

opts[:resource_provider] = specific_resource_constraint('Microsoft.DataFactory/factories', opts)
opts[:required_parameters] = %i(factory_name)
opts[:resource_path] = [opts[:factory_name], 'queryPipelineRuns'].join('/')
opts[:method] = 'post'
# static_resource parameter must be true for setting the resource_provider in the backend.
super(opts, true)

# Check if the resource is failed.
# It is recommended to check that after every usage of superclass methods or API calls.
return if failed_resource?

# Define the column and field names for FilterTable.
# - column: It is defined as an instance method, callable on the resource, and present `field` values in a list.
# - field: It has to be identical with the `key` names in @table items that will be presented in the FilterTable.
# @see https://github.com/inspec/inspec/blob/master/docs/dev/filtertable-usage.md

# FilterTable is populated at the very end due to being an expensive operation.
populate_filter_table_from_response
end

def to_s
super(AzureDataFactoryPipelineRunResources)
end

private

def flatten_hash(hash)
hash.each_with_object({}) do |(k, v), h|
if v.is_a? Hash
flatten_hash(v).map do |h_k, h_v|
h["#{k}_#{h_k}".to_sym] = h_v
end
else
h[k] = v
end
end
end

def populate_table
@resources.each do |resource|
@table << flatten_hash(resource)
end
end
end

This file was deleted.

17 changes: 17 additions & 0 deletions test/unit/resources/azure_data_factory_pipeline_run_resource.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
require_relative 'helper'
require 'azure_data_factory_pipeline_run_resource'

class AzureDataFactoryPipelineRunResourceTestConstructorTest < Minitest::Test
def test_empty_param_not_ok
assert_raises(ArgumentError) { AzureDataFactoryPipelineRunResource.new }
end

# resource_provider should not be allowed.
def test_resource_provider_not_ok
assert_raises(ArgumentError) { AzureDataFactoryPipelineRunResource.new(resource_provider: 'some_type') }
end

def test_resource_group
assert_raises(ArgumentError) { AzureDataFactoryPipelineRunResource.new(name: 'my-name') }
end
end
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
require_relative 'helper'
require 'azure_data_factory_pipeline_run_resources'

class AzureDataFactoryPipelineRunResourcesTestConstructorTest < Minitest::Test
# resource_type should not be allowed.
def test_resource_type_not_ok
assert_raises(ArgumentError) { AzureDataFactoryPipelineRunResources.new(resource_provider: 'some_type') }
end

def tag_value_not_ok
assert_raises(ArgumentError) { AzureDataFactoryPipelineRunResources.new(tag_value: 'some_tag_value') }
end

def tag_name_not_ok
assert_raises(ArgumentError) { AzureDataFactoryPipelineRunResources.new(tag_name: 'some_tag_name') }
end

def test_resource_id_not_ok
assert_raises(ArgumentError) { AzureDataFactoryPipelineRunResources.new(resource_id: 'some_id') }
end

def test_name_not_ok
assert_raises(ArgumentError) { AzureDataFactoryPipelineRunResources.new(name: 'some_name') }
end
end