The title is a bit specific for this blog post, but that's the gist of it: we ended up with a bunch of references to an in-between version of .NET (4.6.1) that was falsely advertising itself as a more optimal candidate for satisfying 4.6.2 dependencies. This is a known issue; there are several links to MS GitHub issues below.
In this blog, I will discuss direct vs. transient dependencies as well as internal vs. runtime dependencies.
If you've run into problems with an application targeted to .NET Framework 4.6.2 that does not compile on certain machines, it's possible that the binding redirects Visual Studio has generated for you use versions of assemblies that aren't installed anywhere but on a machine with Visual Studio installed.
How I solved this issue:
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>to your project)
The product should now run locally and on other machines.
For more details, background and the story of how I ran into and solved this problem, read on.
What do we mean when we say that we "build" an application?
Building is the process of taking a set of inputs and producing an artifact targeted at a certain runtime. Some of these inputs are included directly while others are linked externally.
The machine does exactly what you tell it to, so it's up to you to make sure that your instructions are as precise as possible. However, you also want your application to be flexible so that it can run on as wide an array of environments as possible.
Your source code consists of declarations. We've generally got the direct inputs under control. The code compiles and produces artifacts as expected. It's the external-input declarations where things go awry.
What kind of external inputs does our application have?
How is this stitched together to produce the application that is executed?
The NuGet dependencies are resolved at build time. All resources are pulled and added to the release on the build machine. There are no run-time decisions to make about which versions of which assemblies to use.
Dependencies come in two flavors:
It is with the transient references that we run into issues. The following situations can occur:
An application generally includes an
app.config (desktop applications or services) or
web.config XML file that includes a section where binding redirects are listed. A binding redirect indicates the range of versions that can be mapped (or redirected) to a certain fixed version (which is generally also included as a direct dependency).
A redirect looks like this (a more-complete form is further below):
<bindingRedirect oldVersion="0.0.0.0-220.127.116.11" newVersion="18.104.22.168"/>
When the direct dependency is updated, the binding redirect must be updated as well (generally by updating the maximum version number in the range and the version number of the target of the redirect). NuGet does this for you when you're using
package.config. If you're using Package References, you must update these manually. This situation is currently not so good, as it increases the likelihood that your binding redirects remain too restrictive.
NuGet packages are resolved at build time. These dependencies are delivered as part of the deployment. If they could be resolved on the build machine, then they are unlikely to cause issues on the deployment machine.
Where the trouble comes in is with dependencies that are resolved at execution time rather than build time. The .NET Framework assemblies are resolved in this manner. That is, an application that targets .NET Framework expects certain versions of certain assemblies to be available on the deployment machine.
We mentioned above that the algorithm sometimes chooses the desired version or higher. This is not the case for dependencies that are in the assembly-binding redirects. Adding an explicit redirect locks the version that can be used.
This is generally a good idea as it increases the likelihood that the application will only run in a deployment environment that is extremely close or identical to the development, building or testing environment.
How can we avoid these pesky run-time dependencies? There are several ways that people have come up with, in increasing order of flexibility:
To sum up:
Our application targets .NET Framework (for now). We're looking into .NET Core, but aren't ready to take that step yet.
To sum up the information from above, problems arise when the build machine contains components that are not available on the deployment machine.
How can this happen? Won't the deployment machine just use the best match for the directives included in the build?
Ordinarily, it would. However, if you remember our discussion of assembly-binding redirects above, those are set in stone. What if you included binding redirects that required versions of system dependencies that are only available on your build machine ... or even your developer machine?
We actually discovered an issue in our deployment because the API server was running, but the Authentication server was not. The Authentication server was crashing because it couldn't find the runtime it needed in order to compile its Razor views (it has ASP.Net MVC components). We only discovered this issue on the deployment server because the views were only ever compiled on-the-fly.
To catch these errors earlier in the deployment process, you can enable pre-compiling views in release mode so that the build server will fail to compile instead of a producing a build that will sometimes fail to run.
<MvcBuildViews>true</MvcBuildViews> to any MVC projects in the
PropertyGroup for the release build, as shown in the example below:
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' "> <DebugType>pdbonly</DebugType> <Optimize>true</Optimize> <OutputPath>bin</OutputPath> <DefineConstants>TRACE</DefineConstants> <ErrorReport>prompt</ErrorReport> <WarningLevel>4</WarningLevel> <LangVersion>6</LangVersion> <MvcBuildViews>true</MvcBuildViews> </PropertyGroup>
We mentioned above that NuGet is capable of updating these redirects when the target version changes. An example is shown below. As you can see, they're not very easy to write:
<?xml version="1.0" encoding="utf-8"?> <configuration> <runtime> <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1"> <dependentAssembly> <assemblyIdentity name="System.Reflection.Extensions" publicKeyToken="B03F5F7F11D50A3A" culture="neutral"/> <bindingRedirect oldVersion="0.0.0.0-22.214.171.124" newVersion="126.96.36.199"/> </dependentAssembly> <!-- Other bindings... --> </assemblyBinding> </runtime> </configuration>
Most bindings are created automatically when MSBuild emits a warning that one would be required in order to avoid potential runtime errors. If you compile with MSBuild in Visual Studio, the warning indicates that you can double-click the warning to automatically generate a binding.
If the warning doesn't indicate this, then it will tell you that you should add the following to your project file:
After that, you can rebuild to show the new warning, double-click it and generate your assembly-binding redirect.
When MSBuild generates a redirect, it uses the highest version of the dependency that it found on the build machine. In most cases, this will be the developer machine. A developer machine tends to have more versions of the runtime targets installed than either the build or the deployment machine.
A Visual Studio installation, in particular, includes myriad runtime targets, including many that you're not using or targeting. These are available to MSBuild but are ordinarily ignored in favor of more appropriate ones.
That is, unless there's a bit of a bug in one or more of the assemblies included with one of the SDKs...as there is with the net461 distribution in Visual Studio 2017.
Even if you are targeting .NET Framework 4.6.2, MSBuild will still sometimes reference assemblies from the 461 distribution because the assemblies are incorrectly marked as having a higher version than those in 4.6.2 and are taken first.
I found the following resources somewhat useful in explaining the problem (though none really offer a solution):
How can you fix the problem if you're affected?
You'll generally have a crash on the deployment server that indicates a certain assembly could not be loaded (e.g.
System.Runtime). If you show the properties for that reference in your web application, do you see the path
C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\net461 somewhere in there? If so, then your build machine is linking in references to this incorrect version. If you let MSBuild generate binding redirects with those referenced paths, they will refer to versions of runtime components that do not generally exist on a deployment machine.
Tips for cleaning up:
C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\net461in the output?
A sample warning message:
[ResolvePackageFileConflicts] Encountered conflict between 'Platform:System.Collections.dll' and 'CopyLocal:C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\net461\lib\System.Collections.dll'. Choosing 'CopyLocal:C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\net461\lib\System.Collections.dll' because AssemblyVersion '188.8.131.52' is greater than '184.108.40.206'.
As mentioned above, but reiterated here, this what I did to finally stabilize my applications:
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>to your project)
When you install any update of Visual Studio, it will silently repair these missing files for you. So be aware and check the folder after any installations or upgrades to make sure that the problem doesn't creep up on you again.
Sign up for our Newsletter