Multiple ways to automate code testing

/

During PSConfEU I gave a talk about testing your code quality with Pester and PSScriptAnalyzer (also available on youtube). At the end of this talk there was a question about how to do this automatically still on your own device. During the event I gave a small community session about one option but I now want to use this opportunity to cover a few ways you can automatically run your (PowerShell) tests. The following ways will be covered:

  • Using Azure DevOps Pipelines

  • Using VSCode extensions

  • Using Git Hooks

Using Azure DevOps Pipelines

This option I’ve written about before, but I only glanced over the option to automatically trigger the tests in Azure DevOps. When working with Azure DevOps pipelines there are actually multiple ways to use these pipelines to automatically run the tests.
If you want a pipeline to run once a pull request is created or updates you can use branch policies to set this up. This can be done by going to the main branch in your repository and selecting branch policies next to it.

When you click on the build validation in the next screen you can select a pipeline to run. Inside this pipeline you can set up your tests.

Now YAML pipelines have more ways of running automatically. If you look at the first part of a YAML pipeline it will often have a trigger attribute. You can use this trigger to for example have a pipeline run once code has been pushed to it. It’s possible to have a pipeline which is selected in the build validation and have this pipeline also trigger on code being pushed to your main branch. This way you can have it run checks in the pull request and do one last check once the code is actually in your main branch.

Something you could do is have some stages be depended on if the pipeline runs in a pull request or not. For example you could use the Build.Reason variable in Azure DevOps to determine if the run is from a pull request or not. By using conditions in your stages you can then have the deployment stages only run when the pipeline is triggered outside a pull request.

Using VSCode extensions

The tool VSCode has a build in API to enable testing. There are many different plugins which will use this API. In this blogpost I will look at a extension for PowerShell. There is a similar extension available for most popular programming languages.

For PowerShell we can use the Pester Extension in the VSCode marketplace. Once you install this extension it will detect any .tests.ps1 files in your repository and show them in a overview.

At the top you can press the “run” button to execute the tests. One you’ve done this it will run all tests using the PowerShell integrated terminal and show the results. You can expand the tests to see more information about the different tests that have been executed.

By default the tests will run every time you make alterations to a test file. There are several settings you can use to alter the behavior of this extension. For example you can have the tests run in a separate PowerShell instance or define a folder where it should look for tests instead of the root of the workplace and all folders below.

Using GitHooks

Githooks are a tool build into git to enable some automation. If you open the .git folder in a repository and browse to the “hooks” folder you can find several examples. By changing the name of one of them and removing the “.sample” behind it you can activate it. Now this script will run every time a certain action is triggered. When talking about testing the two most usefull githooks are:

Pre-Commit - This hook will run once you try to commit some code
Pre-Push - This hook will run once you try to push the code to a remove repository

You can alter a file to have code like this:

#!/bin/sh
pwsh.exe -c "Invoke-Pester .\get-info.tests.ps1"

This piece of code will invoke the test called “get-info.tests.ps1”. It does require Pester to be installed and available in a default PowerShell session. This will run the tests and show if any problems have occurred. If you are not running the git command from a terminal you wont see these results and the commit or push will continue without a problem. If you want the action to be blocking you can alter it to this:

#!/bin/sh
pwsh.exe -c "if((Invoke-Pester .\get-info.tests.ps1 -PassThru).Result -eq 'Failed') {throw 'tests no successfull'}"

This will throw an error if the tests are not successful. If an error is thrown in a githook it will stop the action. So this way you can prevent a user from pushing code to a remote repository unless all the tests are done.

A note needs to be made here that githooks are set up on the device locally. It is possible to share githooks in a repository by creating a separate folder in the repository containing the githooks. But you will need to run a command like this to actually have git recognize the githooks.

git config --local core.hooksPath <PATH_TO_GITHOOKS>

If your repository has some kind of setup script to setup up other things or you are using dev containers you can make sure in there this setting is applied. If not you will need to trust your developers to set it up properly.

Conclusion

There are many different ways to set up automated testing, all have their advantages and disadvantages and are useful in different situations. If you are aware of the options you can pick the right way in every case. This is why I wrote this blogpost and I hope these options will help you to set up your testing strategy better in the future!