Jenkins NUnit Integration

This integration contains:

• A CI job that runs NUnit tests and creates HTML reports after building a .NET project
• Installing HTML publish plugin
• Generating NUnit report with ReportUnit
• Generating code coverage reports with OpenCover
• Generating code coverage HTML reports with ReportUnit

But doesn't contain:

• How to install and Jenkins
• How to create a build job from scratch
• How to integrate the source control managers
• How to write unit tests
• etc.

Setup

This is relatively an easy task to integrate Jenkins with NUnit and the reports as you'll see in the batch script. But the tricky part is installing the right tools and running them on the console separately with the right parameters. If you can accomplish that, Jenkins integration is easy.

Step 1: Installing Nuget packages

Install these packages to your test project:

• NUnit
• NUnit.ConsoleRunner
• ReportUnit
• OpenCover
• ReportGenerator

If you check your packages directory, you'll see these packages installed for example NUnit Console Runner is in:

packages\NUnit.ConsoleRunner.X.X.X\tools\nunit3-console.exe

Write at least a few unit tests, and test the project using this exe from command line.

Watch out the version numbers! You will need to arrange the batch script with the installed versions. If you update the packages and the version numbers change, you would need to update the script again.

Step 2: Create a bat executable

Create a jenkins_unittests.bat file in your project folder like this:

 1REM Path variables
 2SET FolderPath=%1
 3SET ResultsPath=C:\UnitTestResults
 4SET CoverageHistoryPath=C:\CoverageHistory
 5
 6SET NunitPath=%FolderPath%\packages\NUnit.ConsoleRunner.3.5.0\tools
 7SET ReportUnitPath=%FolderPath%\packages\ReportUnit.1.5.0-beta1\tools
 8SET OpenCoverPath=%FolderPath%\packages\OpenCover.4.6.519\tools
 9SET ReportGeneratorPath=%FolderPath%\packages\ReportGenerator.2.4.5.0\tools
10
11SET UnitTestProj=%FolderPath%\MyProject.UnitTests\MyProject.UnitTests.csproj
12
13REM Recreate Results Folder
14rd /S /Q %ResultsPath%
15md %ResultsPath%
16
17REM Create coverage history folder if not exists
18if not exist "%CoverageHistoryPath%" mkdir %CoverageHistoryPath%
19
20REM Run Nunit3 for tests, it produces TestResult.xml report
21%NunitPath%\nunit3-console.exe %UnitTestProj% --result=%ResultsPath%\TestResult.xml
22
23REM Get nunit command errorlevel
24SET NunitError=%ERRORLEVEL%
25
26REM Run ReportUnit to create HTML Report from Nunit XML report
27%ReportUnitPath%\ReportUnit.exe %ResultsPath%\TestResult.xml
28
29REM Run OpenCover to create coverage XML report
30%OpenCoverPath%\OpenCover.Console.exe -register:user -register:user -target:%NunitPath%\nunit3-console.exe -targetargs:%UnitTestProj% -output:%ResultsPath%\opencovertests.xml
31
32REM Run ReportGenerator to create coverage HTML report from coverage XML
33%ReportGeneratorPath%\ReportGenerator.exe -reports:%ResultsPath%\opencovertests.xml -targetDir:%ResultsPath% -historydir:%CoverageHistoryPath%
34
35REM Fail if Nunit has found an error on tests
36exit /b %NunitError%

You need to add this file to the source control as well. In this way jenkins could access the file after pulling (TFS: checkout) your source repository.

To update or maintain the script easily, I used some parameters and variables across the script. %1 is the first (and only) parameter that we will use. We will pass the directory of our project that jenkins will use from the jenkins job.

Please update the script as the right paths and versions of your packages. Also test project name must be changed according to yours.

Step 3: Creating paths

I used these 2 folders for reports generated.

• C:\UnitTestResults
• C:\CoverageHistory

Create these folders on the computer jenkins runs.

Step 4: Understand and test

The basic flow is:

You can also try to run each script separately to see how it works, or to troubleshoot. In each step an output file will be created. When NUnit tests run, an XML file (we gave TestResult.xml as the output file name) will be created in the specified path. Then ReportUnit will create a single HTML file (TestResult.html) in the same folder. This will be the first report and will give us unit test results.

Second report will be the coverage report. OpenCover will create report XML (opencovertests.xml) using NUnit, then ReportGenerator will create an HTML site with that file in the given path (C:\UnitTestResults) and addition to that, the old reports will be saved at (C:\CoverageHistory), so each time you run coverage, you can also track coverage changes.

Step 5: Configuring Jenkins

You need these plugins or equivalents in order to make this integration

• A source control plugin (In my case it is Team Foundation Server Plugin)
• A build plugin (In my case it is MSBuild Plugin)
• HTML publisher plugin

Search and install these plugins from Manage Jenkins -> Manage Plugins

Step 6: Create/Update your jenkins job

After you setup your jenkins job and see that your build is successful, we can go on and integrate the tests.

After the MSBuild step, you need to add a Execute Windows batch command step. Command will be:

1[PATH_TO_YOUR_PROJECT_ROOT]\jenkins_unittest.bat [PATH_TO_YOUR_PROJECT_ROOT]

This way we will pass the project root as a parameter to the batch script.

Then we need to add two HTML reports from Add post-build action button, and selecting Publish HTML reports.

We need to add two reports by clicking the add button twice. First one will be the unit test report, and second will be the Coverage report.

Fill the forms with the appropriate values

1. First report
   • HTML directory to archive: C:\UnitTestResults
   • Index page[s]: TestResult.html
   • Report title: Unit Test Report
2. Second report
   • HTML directory to archive: C:\CoverageHistory
   • Index page[s]: index.htm
   • Report title: Code Coverage Report

Select Keep past HTML reports and Always link to last build from Publishing options.

Notes

HTML publisher plugin uses external resources (js, css) and that is not allowed on jenkins' default configuration. We need to change it as this wiki page explains.

In my case jenkins is a windows service and I can change the configuration via jenkins.xml and restart the service.

1<arguments>-Xrs -Xmx256m -Dhudson.model.DirectoryBrowserSupport.CSP= -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\jenkins.war" --httpPort=8080 --webroot="%BASE%\war"</arguments>

As suggested, check the setting by running System.getProperty("hudson.model.DirectoryBrowserSupport.CSP") command in Jenkins Script Console