Unit Tests with Dependencies in Visual Studio and Team Foundation Server

When running unit tests as part of the build process on TFS (such as with continuous integration) the tests are executed on a folder dedicated to the test run, which contains the needed assemblies and will hold the test results. This folder is not the build output folder; instead, the assemblies are copied to that folder after the build. If you have some additional items that your tests depend on (e.g. a sample text file) will need to instruct the build to copy those items to the test run folder.

Suppose that we have a super useful class that has a method to check if a file is a txt file. In our test project we need some sample files, as illustrated bellow.

One way of doing this is through the test settings file (look for it under Solution Items and add one if not present). One can specify the additional files that should be deploy with the tests:

This file can then be selected as the settings file on the build definition for TFS:

When using this option, the files are deployed to the root of the folder, i.e. they will not keep the folder structure, even if you add the parent directory. This means that you can’t have files with the same name.  Also, the tests have to be written assuming that their dependencies are on the “current” folder:

        [TestMethod]
        public void TestIsTextFileWithTextFile()
        {
            bool res = FileUtils.IsTxtFile("TextFile.txt");
            Assert.IsTrue(res);
        }

        [TestMethod]
        public void TestIsTextFileWithXmlFile()
        {
            bool res = FileUtils.IsTxtFile("XMLFile.xml");
            Assert.IsFalse(res);
        }

Note that you’ll also need a local test settings file that includes the needed items so that local test runs are ok. If you don’t want to use test settings files (personally, I think that the test dependencies are not so clear that way) you can use the DeploymentItem attribute. This attribute enables the copy of an item at a given path to the test deployment folder (or another child folder). It can be used as illustrated bellow (note that the paths are relative to the output directory; to use them as is, enable the “Copy to output directory” option on each of them).

        [TestMethod]
        [DeploymentItem(@"TestFiles\TextFile.txt")]
        public void TestIsTextFileWithTextFile()
        {
            bool res = FileUtils.IsTxtFile("TextFile.txt");
            Assert.IsTrue(res);
        }

        [TestMethod]
        [DeploymentItem(@"TestFiles\XMLFile.xml", "Samples")]
        public void TestIsTextFileWithXmlFile()
        {
            bool res = FileUtils.IsTxtFile(@"Samples\XMLFile.xml");
            Assert.IsFalse(res);
        }

Note that to use the attribute you still have to check the “Enable Deployment” option on test settings.

Additional information:

DeploymentItemAttribute Class

Create Test Settings to Run Automated Tests from Visual Studio

How to: Configure Test Deployment

Advertisement

Everest 3G

From Learn Something Every Day

Mushroom houses, anyone?

Someone is making bricks out of mushrooms! Solid, clean, light and biodegradable.

One Day Will People Be Living In Shroom Houses? @ TechCrunch

Azure Service Packages with Unreferenced Assemblies

Applications often rely on assemblies that are not statically linked (i.e. referenced) from their projects. A common scenario is to have exchangeable components that implement a well-know interface but are only resolved at runtime (e.g. accordingly to some entry on web.config). This can happen when using custom log4net appenders, ASP.NET membership providers, trace listeners and, of course, any other application-specific types. Somewhere along the build process you need to copy the needed assemblies to the “bin” folder so that it can be located at runtime.

Let me introduce an example. Consider the following ultra-useful interface and the Visual Studio projects illustrated bellow.

public interface ISomeInterface
{
	int GenerateSomeNumber();
}

The projects are organized as follows:

• ISomeInterface  is defined on the Common assembly.
• The Web application references Common and has a page that shows some numbers obtained through an ISomeInterface.  The name of the type of the actual ISomeInterface realization is read from web.config and the type is instantiated using reflection.
• The Impl assembly defines the type that implements ISomeInterface.
• The CloudServicePackage is a cloud project that references the web application as a web role.

The final objective is to include the Impl assembly on the could project’s web role so that the ISomeInterface realization can be used.

I don’t want any dependencies from Web to Impl other than configuration. Thus, my first approach was to copy the Impl assembly to the Web\bin folder on a post-build event of Impl build process. And this actually worked when running the cloud project on the development fabric. However, when I published the project, the Impl assembly wasn’t included in the package.. What?!

After digging a bit into the build output and the build targets definition (on {Program Files x86}\MSBuild\Microsoft\Cloud Service\1.0\Visual Studio 10.0\Microsoft.CloudService.targets) it seems that the build behaves differently between a “build-and-run” and a publish operation (there is a PackageWebRole variable which is set to false in the first and to true in the last). In the first case, the process is simpler: there are validations of service definition and configuration, the role references are resolved and the web application binaries and content seem to be copied to the cloud project’s output folder. However, the package itself is not created. On the other hand, when publishing the service, after the previous actions, additional things are happening, namely with respect to resolving the needed files and assemblies. The build process seems to gather the final list of files accordingly to the project’s items and dependencies! This means that despite being on the output folder, the Impl assembly is not included in the package.

I don’t fully understand the reason, but it is restrictive. So we need an alternative. Since the build ensures, at some point, that the “unneeded” files are not present, the next option was to copy the Impl assembly directly into the cloud project’s output folder after that point. After further digging into the build output and Microsoft.CloudService.targets, I extracted some of the important steps on the build process (many targets/tasks were removed for readability):

Build-and-run:

Target "FindServiceDefinition"
Target "FindServiceConfiguration"
Target "ValidateRoleProjects"
Target "ValidateServiceFiles"
Target "ValidateComputeProject"
Target "ResolveWebRoleReferences"
  Target "ResolveSingleWebRoleReference"
    Task "Message"
      PackageWebRole=False
    Task "Message"
      WebRoleProject=..\Azure.UnreferencedDependencies.Web\Azure.UnreferencedDependencies.Web.csproj
    Task "Message"
      IntermediateWebOutputPathItem=obj\Debug\Azure.UnreferencedDependencies.Web\
    Task "MakeDir"
    Task "Copy"
      (list of files in the Web bin directory here!)
Target "CopyServiceDefinitionAndConfiguration"
Target "CorePackageComputeService"
  Task "Message"
    OutputDirectory is (...)bin\Debug\Azure.UnreferencedDependencies.CloudServicePackage.csx\
  Task "Message"
    PackRoles  is (...)Azure.UnreferencedDependencies.CloudServicePackage\obj\Debug\Azure.UnreferencedDependencies.Web\
  Task "CSPack"

Publish:

Target "BeforePublish"
Target "PrepareForPackaging"
Target "ValidateRoleProjects"
Target "ValidateServiceFiles"
Target "ValidateComputeProject"
Target "ResolveWebRoleReferences"
  Target "ResolveSingleWebRoleReference"
    Task "Message"
      IntermediateWebOutputPathItem=obj\Debug\Azure.UnreferencedDependencies.Web\
    Task "MakeDir"
    Task "Copy"
(full list of Web project files here!)
Target "CopyServiceDefinitionAndConfiguration"
Target "CorePackageComputeService"
  Task "Message"
    OutputDirectory is (...)bin\Debug\Azure.UnreferencedDependencies.CloudServicePackage.csx\
  Task "Message"
    PackRoles is (...)Azure.UnreferencedDependencies.CloudServicePackage\obj\Debug\Azure.UnreferencedDependencies.Web\
  Task "CSPack"
Target "CheckRoleInstanceCount"
Target "CorePublish"
  Task "Message"
    CorePublish: PackageWebRole = True
  Task "Message"
    Publishing to '(...)bin\Debug\Publish'
  Task "Message"
    Roles is (...)Azure.UnreferencedDependencies.CloudServicePackage\obj\Debug\Azure.UnreferencedDependencies.Web\
  Task "CSPack"

The process is similar: the roles are resolved, compiled and copied to obj the folder and then “packaged” into the bin folder. The main difference is in the list of files that are included (probably controlled by the PackageWebRole flag). In the first case seems that only the Web role’s bin folder is copied (in a sort of relax mode) while in the second there’s a strict list of the files to be included. Anyway, the obj folder seems to be the base for the final package creation. And it has to be ready (i.e. with all the needed files) right before the CorePackageComputeService target. So we just need to ensure that it is!

A possible solution is to define a target on the cloud project that copies the Impl assembly to the appropriate place inside the output folder. In this case, I want it to be copied to the bin folder of the Web app inside the package output folder (obj). Unload the cloud project and add the following code:

<ItemGroup>
  <LateBoundAssemblies Include="$(SolutionDir)Azure.UnreferencedDependencies.Impl\bin\$(Configuration)\Azure.UnreferencedDependencies.Impl.dll">
    <Destination>Azure.UnreferencedDependencies.Web\bin\Azure.UnreferencedDependencies.Impl.dll</Destination>
  </LateBoundAssemblies>
</ItemGroup>

<Target Name="CopyLateBoundAssemblies" BeforeTargets="CorePackageComputeService">
  <Copy SourceFiles="@(LateBoundAssemblies->'%(Identity)')"
	  DestinationFiles="@(LateBoundAssemblies->'$(ProjectDir)obj\$(Configuration)\%(Destination)')" />
  <!-- Using full IIS support; also copy to the bin on the web project so that it works on dev fabric -->
  <Copy SourceFiles="@(LateBoundAssemblies->'%(Identity)')"
          DestinationFiles="@(LateBoundAssemblies->'$(SolutionDir)%(Destination)')" />
</Target>

With this actions, when the CorePackageComputeService target runs, the Impl assembly will be in the right folder and will be packaged along. This technique also keeps these kind of actions contained on the cloud project and enables the inclusion of assemblies that are not part of the solution.