Filtering within specific exports

[briankronberg]

Description

I have used Microsoft365DSC so far on two projects to dump the configuration prior to importing it into a new tenant. Both of these projects were divestitures, so I was only interested in a sub-set of the source tenant. I would like a way to filter inside some of the configuration items. For example:

• AADConditionalAccessPolicy: Filter: {DisplayName -like "SubCompany*"}
• O365Group: Filter: {PrimarySMTPAddress -match "domainname.com"}

Proposed properties

Special considerations or limitations

I understand this would take a lot of work but it would be essential for getting to a no-code way of exporting and importing it without having to filter through the files manually.

[andikrueger]

You could use DSCParser to generate an object representation of your configuration. This object can be filtered and you should be able to extract the resources. Generating a new configuration would still be tricky though.

https://nikcharlebois.com/dscparser-analyze-your-powershell-dsc-scripts/

[briankronberg]

This is a great start! Thanks for sharing. This will at least allow me to quickly filter out what I need. What it doesn't solve though is when I need only 100 groups out of 100,000. Run time will still be horrific but I can at least filter afterwards.

My main goal with the filter was to reduce run time of the source query against really large tenants.

[andikrueger]

Just a few thoughts on this topic on how we could add this functionality: Basically, we would need a further parameter for Export-M365DSCConfiguration. This parameter could be named Filter and would allow a hashtable of FilterScripts:

@{
   AADConditionalAccessPolicy = 'DisplayName -like "SubCompany*""'
   O365Group = 'PrimarySMTPAddress -match "domainname.com"'
}

We would need to pass this parameter to Start-M365DSCConfigurationExtract. In this function we could elaborate, if a resource allows filters for exporting the data. This could be very similar to the already available checks:

Microsoft365DSC/Modules/Microsoft365DSC/Modules/M365DSCReverse.psm1

Lines 417 to 426 in 0c00b6f

	Import-Module $resource.FullName | Out-Null
	$MaxProcessesExists = (Get-Command 'Export-TargetResource').Parameters.Keys.Contains("MaxProcesses")
	$AppSecretExists = (Get-Command 'Export-TargetResource').Parameters.Keys.Contains("ApplicationSecret")
	$CertThumbprintExists = (Get-Command 'Export-TargetResource').Parameters.Keys.Contains("CertificateThumbprint")
	$TenantIdExists = (Get-Command 'Export-TargetResource').Parameters.Keys.Contains("TenantId")
	$AppIdExists = (Get-Command 'Export-TargetResource').Parameters.Keys.Contains("ApplicationId")
	$AppSecretExists = (Get-Command 'Export-TargetResource').Parameters.Keys.Contains("ApplicationSecret")
	$CredentialExists = (Get-Command 'Export-TargetResource').Parameters.Keys.Contains("Credential")
	$CertPathExists = (Get-Command 'Export-TargetResource').Parameters.Keys.Contains("CertificatePath")
	$CertPasswordExists = (Get-Command 'Export-TargetResource').Parameters.Keys.Contains("CertificatePassword")

If the resource allows filtered exports, we could pick the matching object out of the hash table and then generate a limited export of this resource.

This would not be a breaking change. @NikCharlebois Do you see any obstacles with this idea?

[NikCharlebois]

I am way late to the game. Apologies folks. This is actually a great suggestion @andikrueger and I've been spending the past hour working on this with very promising results. This is exciting to finally have a sustainable way to specify resource level filters. My plan is to have this available (with a small subset of resources supporting it) for next week's release.