Check your Power Platform solutions for secrets with Pester and PowerShell

/

Lately I’ve been doing some work on deployments to the Power Platform. I’ve chosen to use Azure DevOps pipelines as the deployment solution. The reason behind this I leave out of the scope of this blogpost.

When using Azure DevOps you can use GitHub Advanced Security for Azure DevOps but this is quite costly and might be to much for your applications. In our case we just wanted to have a simple scanning for secrets when exporting solution from the DEV environment and deploying them. Just to make sure a developer didn’t accidently use an url with a token in it or have some hard coded username and password in the solution.

To achieve this I used the PowerShell PSSecretScanner module by Björn Sundling. This module uses several different regex queries to determine if there is a secret in your repository. To get a good overview of the results I used Pester the PowerShell testing framework to create tests around this secret scanner.

To use Pester you can create a test file in your repository. Let’s assume we have a repository structure like this:

Project
|  README.md
|  pipeline.yml
└───Settings
|   └───Solution1
|   |   └───TST
|   |   |   | Solution1.json
|   |   └───PROD
|   |       | Solution1.json
|   └───Solution2
|   |   └───TST
|   |   |   | Solution2.json
|   |   └───PROD
|   |       | Solution2.json
└───Solutions
|   └───Solution1
|   |   └───Managed
|   |   |   └───environmentvariabledefinitions
|   |   |   |   └───Var1
|   |   |   |   | Var1.xml
|   |   |   |   └───Var2
|   |   |   |   | Var2.xml
|   |   |   └───Other
|   |   |   |   | Customizations.xml
|   |   |   |   | Solution.xml
|   |   |   └───Workflows
|   |   |   |   | Flow1.json
|   |   |   |   | Flow1.json.data.xml
|   └───Solution2
|   |   └───Managed
|   |   |   └───environmentvariabledefinitions
|   |   |   |   └───Var1
|   |   |   |   | Var1.xml
|   |   |   |   └───Var2
|   |   |   |   | Var2.xml
|   |   |   └───Other
|   |   |   |   | Customizations.xml
|   |   |   |   | Solution.xml
|   |   |   └───Workflows
|   |   |   |   | Flow1.json
|   |   |   |   | Flow1.json.data.xml
└───Tests
|   | Secrets.tests.ps1

It’s possible to also have a unmanaged version in the structure or to have more stages to deploy, but for simplicity sake I’ve limited it to this. The test I will provide will offer the option to expend it easily to more types.

The Secret.tests.ps1 file looks like this:

param (
    [string] $solution
)

BeforeDiscovery {
    Install-Module PSSecretScanner -Force
    $ScanTypes = @(
        @{
            type      = "Settings"
            subtypes  = @(@{ subtype = "TST"},@{ subtype = "PRD"})
        }, 
        @{
            type      = "Solutions"
            subtypes  = @(@{ subtype = "Managed"})
        }
    )
}

Describe "Secrets in <type>" -ForEach $ScanTypes {
    BeforeAll {
        Import-Module PSSecretScanner
    }
    It " $solution Should not contain secrets in <SubType>" -ForEach $subtypes {
        # Check for secrets
        $path = Join-Path -Path $PSScriptRoot -ChildPath "..\$type\$solution\$subType\"
        $secrets = Find-Secret -Path "$path"
        $secrets.results | Foreach-Object {
            "Problem in $($_.Filename) at line $($_.LineNumber) with pattern: $($_.PatternName)"
        } | Should -BeNullOrEmpty
    }
}

This test uses a parameter to provide the name of the solution, this way in the pipeline it’s possible to make a pipeline that will only deploy certain solutions and have the tests run for only those solutions.

In the beforediscovery part the settings and solutions are defined and their underlying folders. So if you have more folders you just add them here to the array.

The rest of the code is the test which will get all files in the folders and scans them for secrets. If a secret is found it will be added to an array with an easily readable text. So this way if a problem is found you can easily check the logs of the tests to see where the problem is.

Here is the pipeline.yml file

trigger:
- none

pool:
  vmImage: ubuntu-latest

parameters:
  - name: solutions
    type: stringList
    displayName: Solutions to Deploy
    values:
      - Solution1
      - Solution2
    default:
      - Solution1
      - Solution2

jobs:
- job: Run_Tests
  displayName: Run Tests over code
  steps:
    - checkout: self
    - ${{ each solution in parameters.solutions }}:
      - task: PowerShell@2
        displayName: Run Tests for ${ }
        inputs:
          targetType: 'inline'
          script: |
            # Make sure Pester is installed
              Install-Module Pester -Force

            # Set up Pester Configuration
              $configuration = New-PesterConfiguration
              $configuration.Run.Path = "$(System.DefaultWorkingDirectory)/Tests/*.tests.ps1"
              $configuration.TestResult.Enabled = $true
              $configuration.TestResult.OutputPath = "$(System.DefaultWorkingDirectory)/${ }_testresults.xml"
              $configuration.Run.PassThru = $true
              $configuration.Output.Verbosity = 'Detailed'

            # Set up the test container
              $container = New-PesterContainer -Path "*.tests.ps1" -Data @{ solution = "${ }"}
              $configuration.Run.Container = @($container)

            # Run Pester
              $PesterOutput = Invoke-Pester -Configuration $configuration
          pwsh: true
        continueOnError: true
        condition: in(variables['Agent.JobStatus'], 'Succeeded', 'SucceededWithIssues', 'Failed')
    - task: PublishTestResults@2
      displayName: Publish TestResults
      inputs:
        testResultsFormat: "NUnit"
        testResultsFiles: "*testresults.xml"
        failTaskOnFailedTests: false
        testRunTitle: 'Testresults'
      condition: in(variables['Agent.JobStatus'], 'Succeeded', 'SucceededWithIssues', 'Failed')

This will create a pipeline with a dropdown box where you can select which solution to test. It’s using the fairly new StringList parameter type. The tasks are set up to continue even when failed so that if your test fail the results will still be published.

If you run the pipeline in Azure DevOps you will get a testreport like this

If you click the failed tests you will see an error message like:

Expected $null or empty, but got 'Problem in Solution1.json at line 91 with pattern: Password equal no quotes'.

This should help you track down the secrets in your environments.