What Is the Best Practice for "Copy Local" and with Project References

What is the best practice for Copy Local and with project references?

In a previous project I worked with one big solution with project references and bumped into a performance problem as well. The solution was three fold:

1. Always set the Copy Local property to false and enforce this via a custom msbuild step

2. Set the output directory for each project to the same directory (preferably relative to $(SolutionDir)

3. The default cs targets that get shipped with the framework calculate the set of references to be copied to the output directory of the project currently being built. Since this requires calculating a transitive closure under the 'References' relation this can become VERY costly. My workaround for this was to redefine the GetCopyToOutputDirectoryItems target in a common targets file (eg. Common.targets ) that's imported in every project after the import of the Microsoft.CSharp.targets. Resulting in every project file to look like the following:

   <Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
     <PropertyGroup>
       ... snip ...
     </ItemGroup>
     <Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
     <Import Project="[relative path to Common.targets]" />
     <!-- To modify your build process, add your task inside one of the targets below and uncomment it. 
          Other similar extension points exist, see Microsoft.Common.targets.
     <Target Name="BeforeBuild">
     </Target>
     <Target Name="AfterBuild">
     </Target>
     -->
   </Project>
   

This reduced our build time at a given time from a couple of hours (mostly due to memory constraints), to a couple of minutes.

The redefined GetCopyToOutputDirectoryItems can be created by copying the lines 2,438–2,450 and 2,474–2,524 from C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Microsoft.Common.targets into Common.targets.

For completeness the resulting target definition then becomes:

<!-- This is a modified version of the Microsoft.Common.targets
     version of this target it does not include transitively
     referenced projects. Since this leads to enormous memory
     consumption and is not needed since we use the single
     output directory strategy.
============================================================
                    GetCopyToOutputDirectoryItems

Get all project items that may need to be transferred to the
output directory.
============================================================ -->
<Target
    Name="GetCopyToOutputDirectoryItems"
    Outputs="@(AllItemsFullPathWithTargetPath)"
    DependsOnTargets="AssignTargetPaths;_SplitProjectReferencesByFileExistence">

    <!-- Get items from this project last so that they will be copied last. -->
    <CreateItem
        Include="@(ContentWithTargetPath->'%(FullPath)')"
        Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(ContentWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(_EmbeddedResourceWithTargetPath->'%(FullPath)')"
        Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(Compile->'%(FullPath)')"
        Condition="'%(Compile.CopyToOutputDirectory)'=='Always' or '%(Compile.CopyToOutputDirectory)'=='PreserveNewest'">
        <Output TaskParameter="Include" ItemName="_CompileItemsToCopy"/>
    </CreateItem>
    <AssignTargetPath Files="@(_CompileItemsToCopy)" RootFolder="$(MSBuildProjectDirectory)">
        <Output TaskParameter="AssignedFiles" ItemName="_CompileItemsToCopyWithTargetPath" />
    </AssignTargetPath>
    <CreateItem Include="@(_CompileItemsToCopyWithTargetPath)">
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_CompileItemsToCopyWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_CompileItemsToCopyWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(_NoneWithTargetPath->'%(FullPath)')"
        Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(_NoneWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>
</Target>

With this workaround in place I found it workable to have as much as > 120 projects in one solution, this has the main benefit that the build order of the projects can still be determined by VS instead of doing that by hand by splitting up your solution.

Is Copy Local transitive for project references?

it would also appear that it is important to differentiate file dependencies, where the dependency references a dll assembly file and project dependencies (i.e. what I'm asking about), where the dependency references a project and implicitly the output file of that project.

Not really, no.

MSBuild doesn't really care if the reference points to another project in the solution or to a DLL.

If ProjectA depends on ProjectB to build ProjectA ProjectB must be already built (and up-to-date), MSBuild will then pull its DLL (not its C# code) and link it to ProjectA.

Adding a project reference instead of a DLL is "syntactic sugar" for your convenience: this way MSBuild knows it must pick the output of the referenced project, whatever the output is.

Otherwise, you'll have to manually pre-build the dependency, find its DLL and link it to the project, repeating the process whenever you switch build configuration, move or rename things. Not really practical.

Will the other two dlls also be copied to the output directory?

If any kind of element from a dependency is used directly from the project where the assembly is referenced, that reference will be copied.

An example could be this solution layout:

• MySolution
• MySolution.ConsoleApplication
• MySolution.FirstDependency
• MySolution.SecondDependency
• MySolution.ThirdDependency
• MySolution.FourthDependency

With this dependency chain:

• MySolution.ConsoleApplication
• MySolution.FirstDependency
  
  • MySolution.SecondDependency
    
    • MySolution.ThirdDependency
    • MySolution.FourthDependency

If you build this solution you'll notice that in MySolution.ConsoleApplication output directory there will be the DLLs for MySolution.FirstDependency, MySolution.SecondDependency and MySolution.ThirdDependency but no DLL for MySolution.FourthDependency.

Why is it so? When MSBuild builds MySolution.SecondDependency it notices that there's a dependency declared to MySolution.FourthDependency, but since it can't find any usage of any kind of element from MySolution.FourthDependency in MySolution.SecondDependency code it decides to perform some "optimization" and omits MySolution.FourthDependency assembly from the output.

This same issue bit me in the past when I added through NuGet AutoMapper to a "deep dependency": adding AutoMapper adds two assembly references, AutoMapper and AutoMapper.Net4, where the second assembly is loaded by the first through reflection when it needs to perform certain kind of action on the new collection objects introduced by the .NET Framework 4. Since the second assembly is loaded through reflection MSBuild thinks it's unused and doesn't bother to copy it around.

So, yes, they will be copied as long as you're using them directly and not through reflection.

Is this documented somewhere?

This behavior seems to be a "feature" of MSBuild, I managed to find a blog post by some folks from Microsoft back when I experienced this issue, but I can't find it again at the moment.

Best practice for referencing a project or assembly in a solution

It's normal to have code you want to share between multiple solutions.

For this, we use projects like 'Infrastructure' or 'Logging' with their own CI builds. When done, we create a release build which uploads the dll's to a private nuget server.

These projects are than included as dll's in the other projects through nuget and updated when needed. You also don't break other solutions when you change something in your logging, you have to update the logging version first.

Reference another project with Copy Local on it's reference

Not possible with Visual Studio or msbuild it seems.

Copy Local for website references

Are you adding a reference to the DLL in your project? Doing that SHOULD add it to the bin directory in your project and include it in your deployments.

Does Copy Local matter if all projects are pointing to the same output folder?

I think what you are asking is "does it take any time to copy a file onto itself?" As you might expect, it doesn't. You can see this by using Reflector or ILSpy on the Microsoft.Build.Tasks assembly, the Copy class performs the copy. Its DoCopyIfNecessary() method contains this line of code:

    if (string.Compare(sourceFileState.Name, destinationFileState.Name, StringComparison.OrdinalIgnoreCase) != 0)
    {
        flag = this.DoCopyWithRetries(sourceFileState, destinationFileState, copyFile);
    }

Or in other words, it skips the copy if the source and destination file are the same. Necessarily so, File.Copy() won't be happy.

How to make multiple .net projects copy references only one time in the build

One common solution is to have all projects output to the same directory, and have all references set to CopyLocal=False, and then have one extra project (or an existing one, like the main application) targetting the same output directory and which contains all references and has CopyLocal=True. Then that one project effectively takes care of getting all your references where you want them. To get all projects use the same output directory I'd suggest to modify the .csproj files to all import the same file which sets the OutputPath property instead of manually changing it in each project.

Alternatively, since you mention the main problem with a single output directory and CopyLocal set to True is that some files get copied multiple times, you could modify how the copying occurs. Normally, by default, the Copy tasks used will skip files with matching timestamps. This is controlled by the SkipCopyUnchangedFiles flag so you could force it to true and see if that changes anything:

msbuild my.sln /p:SkipCopyUnchangedFiles=True

Or you could even ask it to use hardlinks instead of copying, this should effectively mean no copies are made at all:

msbuild my.sln /p:CreateHardLinksForCopyLocalIfPossible=True

or symbolic links:

msbuild my.sln /p:CreateSymbolicLinksForCopyLocalIfPossible=True

If all else fails and files are still copied more than once it could mean e.g. that one project references A.dll and another one also references A.dll but from a different directory or a different version etc, so it will get copied twice no matter what unless you fix that.

reference dll and when to have Copy Local set to true

You could add something to the Post-Build event (Project->Properties->BuildEvents) of your common dll. For example you could call a batch file that you keep updated with all the output folders of the projects dependent on the dll

if /I "$(ConfigurationName)" == "Release" 
         Call $(SolutionDir)\distribute_library.cmd $(TargetPath) 

(Just one line, splitted here for readability)

Where distribute_library.cmd is the batch file containing the commands to copy the $(TargetPath) to various destinations.

Something like this:

D:
COPY %1 \MyProject1\bin\release
....other targets ....

Not to forget: Set the Run the post-build event combo to On Successful build

---

Related Topics