Code Coverage

This is the seventh blog of a series of posts about the topics Continuous Integration and Continuous Delivery. It explains how important continuous integration is to deliver software in short iterations to the customer with a high quality standard.

Code coverage is a measure which indicates how much percentage of the code has been tested. There are different techniques but it usually describes how many lines of the code have been executed during the Unit Test and how many have not been. It does not say anything about the quality of the tests itself. Even a high percentage of Code Coverage does not help if the unit tests do not cover the use cases how the component / class is used. But it is a good indicator to find out which parts are tested at all and which have a lack of testing.

Enable Code Coverage
It can be easily activated in the Visual Studio Menu "Test", "Edit Test Settings" and select the Test Settings file.

After that the Code Coverage can be enabled on the "Data and Diagnostics" tab.

The assemblies which should be instrumented have to be selected by clicking the "Configure" button. It has to be checked that just productive assemblies are selected and no unit test projects.

Code Coverage Check during CI Build

A code coverage check can be implemented in order to ensure that a certain amount of unit tests are written and stay in a healthy state. It can check the code coverage percentage and fail the build if the value is under a defined amount. This prevents the tests from getting removed from the build because it would drop the code coverage value. Of course, this says nothing about the quality of the tests but it makes a least sure that the tests are executed and increase with the code basis.

The following example coverage output file is written during the build process if the code coverage has been enabled. The value can be checked reading the BlocksCovered and BlocksNotCovered nodes and compared to a defined value which is the criteria to fail the build or not.

<CoverageDSPriv>
  <xs:schema id="CoverageDSPriv">...</xs:schema>  
  <Module>
    <ModuleName>TSTune.CodeExamples.dll</ModuleName>
    <ImageSize>57344</ImageSize>
    <ImageLinkTime>0</ImageLinkTime>
    <LinesCovered>7</LinesCovered>
    <LinesPartiallyCovered>0</LinesPartiallyCovered>
    <LinesNotCovered>7</LinesNotCovered>
    <BlocksCovered>7</BlocksCovered>
    <BlocksNotCovered>6</BlocksNotCovered>
  </Module>
  <SourceFileNames>
    <SourceFileID>1</SourceFileID>
    <SourceFileName>OrderServiceProxy.cs</SourceFileName>
  </SourceFileNames>
  <SourceFileNames>
    <SourceFileID>2</SourceFileID>
    <SourceFileName>OrderManagement.cs</SourceFileName>
  </SourceFileNames>
</CoverageDSPriv>

In this simple example 7 of 13 blocks have been covered during the test, which is a code coverage of: 7 / (6 + 7) = 0.5385 = 53.85 %.