[Azure Monitor] Update Data Collection Rules to 2021-09-01-preview

[herreraj-ms]

Resource Provider

• RP: Azure Monitor Control Service
• Resources:
  • Data Collection Rules
  • Data Collection Rule Associations
  • Data Collection Endpoints

Description of Feature or Work Requested

• Update AMCS resources (DCR, DCRA, and DCE) to the latest API specification 2021-09-01-preview
• Documentation Links:
  • DCR overview: https://docs.microsoft.com/en-us/azure/azure-monitor/essentials/data-collection-rule-overview
  • Rest Api: https://docs.microsoft.com/en-us/rest/api/monitor/data-collection-rules
  • Python SDK operations are still pending:
    • Request: https://github.com/Azure/sdk-release-request/issues/2521
    • PR: [AutoRelease] t2-monitor-2022-03-10-10406 azure-sdk-for-python#23465

Minimum API Version Required

• 2021-09-01-preview

Swagger Link

• DCR: https://github.com/Azure/azure-rest-api-specs/blob/main/specification/monitor/resource-manager/Microsoft.Insights/preview/2021-09-01-preview/dataCollectionRules_API.json
• DCRA: https://github.com/Azure/azure-rest-api-specs/blob/main/specification/monitor/resource-manager/Microsoft.Insights/preview/2021-09-01-preview/dataCollectionRuleAssociations_API.json
• DCE: https://github.com/Azure/azure-rest-api-specs/blob/main/specification/monitor/resource-manager/Microsoft.Insights/preview/2021-09-01-preview/dataCollectionEndpoints_API.json

Target Date

• April 2022

Support for load payload from file

• Currently the customer must express a complete rule in a flatten schema, example.

az monitor data-collection rule create --resource-group "________" --location "westus2" --name "myCollectionRule" --data-flows destinations="ertyuifjftdghf" streams="Microsoft-Perf" streams="Microsoft-Syslog" streams="Microsoft-WindowsEvent" --log-analytics name="ertyuifjftdghf" resource-id="/subscriptions/_____________________________________/resourceGroups/________________________/providers/Microsoft.OperationalInsights/workspaces/ertyuifjftdghf" --performance-counters name="cloudTeamCoreCounters" counter-specifiers=\\Processor(_Total)\\% Processor Time counter-specifiers=\\Memory\\Committed Bytes counter-specifiers=\\LogicalDisk(_Total)\\Free Megabytes counter-specifiers=\\PhysicalDisk(_Total)\\Avg. Disk Queue Length sampling-frequency=15 transfer-period="PT1M" streams="Microsoft-Perf" --performance-counters name="appTeamExtraCounters" counter-specifiers=[\\Process(_Total)\\Thread Count](file://process%28_total%29/Thread%20Count) sampling-frequency=30 transfer-period="PT5M" streams="Microsoft-Perf" --syslog name="cronSyslog" facility-names="cron" log-levels="Debug" log-levels="Critical" log-levels="Emergency" streams="Microsoft-Syslog" --syslog name="syslogBase" facility-names="syslog" log-levels="Alert" log-levels="Critical" log-levels="Emergency" streams="Microsoft-Syslog" --windows-event-logs name="cloudSecurityTeamEvents" transfer-period="PT1M" streams="Microsoft-WindowsEvent" x-path-queries="Security!" --windows-event-logs name="appTeam1AppEvents" transfer-period="PT5M" streams="Microsoft-WindowsEvent" x-path-queries="System![System[(Level = 1 or Level = 2 or Level = 3)]]" x-path-queries="Application!*[System[(Level = 1 or Level = 2 or Level = 3)]]"

• This is not scalable with rules growing in complexity.
• What we want to do for this version is to take the approach from PowerShell where the cmdlet takes the entire payload (rule) as a parameter, something like:
  • Link: https://docs.microsoft.com/en-us/powershell/module/az.monitor/new-azdatacollectionrule?view=azps-7.3.0

az monitor data-collection rule create  --resource-group "_" --location "_" --name "_" --rule-file "C:\samples\dcrEx1.json"

[yonzhan]

monitor

[kairu-ms]

Hi @herreraj-ms, we still use flatten schema this time. In our next generation of CodeGen tool, we can support entire payload as a parameter.