Referencing system assemblies in Roslyn compilations

With the recent stable release of the Roslyn packages (.NET Compiler Platform) I finally decided to give it a try. I’ve found it pretty simple to compile some code in runtime, load the resulting assembly and execute it. Overall it’s pretty impressive, given the amount of work that’s being done.

There are loads of resources on Roslyn online, namely Josh Varty’s Learn Roslyn Now series and Rolsyn’s documentation on GitHub. On this post, however, I’ll focus one aspect that wasn’t obvious to me in the first place: referencing shared framework assemblies.

When writing code it’s very likely that one ends up using types defined in reference assemblies such as System.Runtime and System.Runtime.Extensions, specially with .NET Standard  and .NET Core’s pay-for play model. These assemblies are just facades and the used types will eventually be forwarded to implementation assemblies in runtime, depending on the target framework.

Going back to Roslyn, the idea is that we’ll be compiling/generating code in runtime, so the question arises: which assemblies should I reference during the compilation? The only satisfying explanation I’ve found is a bit lost in a GItHub issue, hence this post.

There are two main approaches we can use during compilation: 1) use runtime (implementation) assemblies; and 2) use reference (contract) assemblies.

The remainder of this post is based on the following sample program that uses Roslyn. Note that the code to compile (code variable) uses DateTime (defined in reference assembly System.Runtime) and Math (defined in reference assembly System.Runtime.Extensions).

class Program
{
    static void Main(string[] args)
    {
        var code = @"
        using System;
        public class Test
        {
            public double MagicNumber => Math.Round(42.42 * DateTime.Now.Second);
        }
        ";

        var syntaxTree = CSharpSyntaxTree.ParseText(code);

        List references = // TODO define references here!
        var options = new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary, optimizationLevel: OptimizationLevel.Release, allowUnsafe: false);

        var compilation = CSharpCompilation.Create(Guid.NewGuid().ToString("N"), new[] { syntaxTree }, references, options);

        using (var ms = new MemoryStream())
        {
           var compilationResult = compilation.Emit(ms);
           if (compilationResult.Success)
           {
                ms.Position = 0;
                var assembly = AssemblyLoadContext.Default.LoadFromStream(ms);
                var instance = Activator.CreateInstance(assembly.DefinedTypes.First().UnderlyingSystemType);
               // Do something with "instance"
            }
            else
            {
                foreach (var error in compilationResult.Diagnostics)
                {
                   Console.WriteLine(error.ToString());
                }
            }
        }
     }
}

Option 1 – Runtime assemblies

Since we’re invoking Roslyn, we’re already in the context of an executing application targeting a specific framework. This framework includes the runtime (implementation) assemblies that we need, so the first option for compilation is referencing those assemblies. But how can we know their location in a flexible way?

The .NET Core runtime defines a so called “trusted platform assemblies” list, accessible via the AppContext class:

var trustedAssembliesPaths = ((string)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")).Split(Path.PathSeparator);

This list contains the locations of all the assemblies loaded from trusted locations, namely the ones on shared framework installations and NuGet caches. Example:

"C:\Program Files\dotnet\shared\Microsoft.NETCore.App\1.1.1\System.Linq.dll"
"C:\Users\luis\.nuget\packages\system.xml.xpath\4.3.0\lib\netstandard1.3\System.Xml.XPath.dll"

The aforementioned list contains a lot of assemblies that we probably don’t need (and don’t want to make available!) in our code, so it’s probably a good idea to filter the list. After that, creating a MetadataReference is straightforward. The full code to obtain the references that my code needs is:

var trustedAssembliesPaths = ((string)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")).Split(Path.PathSeparator);
var neededAssemblies = new[]
{
    "System.Runtime",
    "mscorlib",
};
List references = trustedAssembliesPaths
    .Where(p => neededAssemblies.Contains(Path.GetFileNameWithoutExtension(p)))
    .Select(p => MetadataReference.CreateFromFile(p))
    .ToList();

Pros:

  • All the needed assemblies are made available by the framework. No need to include additional resources on the application package.

Cons:

  • Need to know the actual runtime (implementation) assemblies. Note that my code depends on System.Runtime.Extensions facade, but I actually add a reference to mscorlib.
  • If an implementation assembly is updated (e.g. shared framework updated) the code might break (it’s unlikely, but possible). Even though .NET APIs should be backward compatible, “compiler overload resolution might prefer an API added in the new version instead of the one that it used to pick before

Option 2 – Reference assemblies

Instead of relying on runtime assemblies, we can compile the code against reference assemblies. The NuGet packages for System.Runtime and alike include both implementation and reference/contract assemblies (you can see this on your local NuGet cache).

System.Runtime/4.3.0/
                   \_ lib/net462/System.Runtime.dll
                   \_ ref/netstandard1.5/System.Runtime.dll

As these assemblies are in a well know location, we can easily create MetadataReferences:

var nugetCache = Environment.GetEnvironmentVariable("UserProfile") + @"\.nuget\packages\";
List references = return new List<MetadataReference>
{
  MetadataReference.CreateFromFile(nugetCache + @"System.Runtime\4.3.0\ref\netstandard1.5\System.Runtime.dll"),
  MetadataReference.CreateFromFile(nugetCache + @"System.Runtime.Extensions\4.3.0\ref\netstandard1.5\System.Runtime.Extensions.dll"),
}
.ToList();

Note that in the sample above I’m using the local NuGet cache. In a real-world scenario you’d include the reference assemblies for the chosen .NET Standard version in the application package (e.g. as resources).

When the assembly generated using Roslyn is loaded, its assembly references are resolved into the corresponding runtime assemblies.

Pros:

  • Assemblies are loaded from a well-known location.
  • Since the reference assemblies won’t change (unless you explicitly update them) there’s a guarantee that the code won’t break on future versions of the .NET Core runtime.

Cons:

  • Additional items on the application package.

Conclusion

Despite both approaches being valid and the first one being used by the C# Scripting API, I tend to prefer the second as it is more stable and less obscure. Hope this helps!

Customizing ASP.NET Core MVC: filters, constraints and conventions

ASP.NET Core – much like the not core predecessor – has many extension possibilities. In this post I’ll cover three of those possibilities that can be useful for application-wide and/or cross-cutting customization:

  • Filters (action filters).
  • Action constraints
  • Application model conventions.

This post is a condensed/revisited version of two posts I did last year about the application model on ASP.NET Core (part I and part II), which by that time was still on beta versions. If you are up to a more in-depth description, take a look at those two posts, as most of them still applies.

Action filters

ASP.NET Core MVC includes multiple types of filters, namely authorization, action and exception filters. All the filters extend IFilterMetadata, which is a marker interface used by the framework to discover filters applied as custom attributes.

The filters are executed within the so called action invocation pipeline or filter pipeline, which runs after MVC selects an action method to handle the request. Different types of filters execute at different stages in the pipeline, hence having different intended usages The official documentation has a decent guide on selecting the appropriate filter type. For this post I’ll stick with action filters.

Actions filters execute immediately before and/or after the action method. They have access to the results of model binding and action parameters (before the action executes) as well as the action result (after the action executes). These filters are represented by the IActionResult interface but the framework includes an utility base class for attribute action filters named ActionFilterAttribute.

A common scenario on action methods (namely Web APIs) is to check if the model state is valid and, if not, return a 400 status code. This is a rather cross-cutting concern and can be easily implemented via an action filter. Using the aforementioned ActionFilterAttribute base class, we can  write a filter that checks the model state validation status before the action method is executed:

public class ValidateModelState : ActionFilterAttribute
{
  public override void OnActionExecuting(ActionExecutingContext context)
  {
    if (!context.ModelState.IsValid)
    {
        context.Result = new BadRequestResult();
    }
  }
}

By setting the Result property during the OnActionExecuting method we prevent the execution of the action method and subsequent action filters. The filter can then be applied to an action method (or a controller class) just like any other custom attribute:

[HttpPost, ValidateModelState]
public IActionResult Test([FromBody] Model model)
{
  // ...
}

Action constraints

An important part of request handling is determining which action method should be executed. This might depend on different factors, such as the request’s HTTP method. Action constraints provide a means of determining if an action is suitable for a given request. They are represented by the IActionConstraint interface, depicted below. The context parameter can be used to access the route data, the HttpContext and the current candidate actions.

public interface IActionConstraint : IActionConstraintMetadata
{
    int Order { get; }
    bool Accept(ActionConstraintContext context);
}

As an example, lets say I want to have an API endpoint that accepts POST requests with two different media types. Instead of if/else on a single action method, we can add an action constraint based on the request’s content type. Here’s how I’d like to write my actions:

[HttpPost, MediaType("application/json")]
public void MyAction(MyModel model) { }

[HttpPost, MediaType("image/png")]
public void MyAction(Stream content) { }

Implementing MediaTypeAttribute  is pretty straight forward: the framework already includes a base class for action constraints implemented as attributes, which we can use in this case:

  public class MediaTypeAttribute : ActionMethodSelectorAttribute
  {
      private readonly string _mediaType;

      public MediaTypeAttribute(string mediaType)
      {
          this._mediaType = mediaType;
      }

      public override bool IsValidForRequest(RouteContext routeContext, ActionDescriptor action)
      {
          return routeContext.HttpContext.Request.ContentType.Equals(_mediaType, StringComparison.OrdinalIgnoreCase);
      }
  }

Application model conventions

If you’re familiar with previous versions of ASP.NET MVC you probably have heard about action descriptors. Action descriptors are discovered by MVC during application startup and represent the HTTP endpoints on the application. As with many other items on MVC, action descriptors are obtained via their own set of providers. While it is possible to add/replace IActionDescriptorProviders, the default implementation includes an intermediate mechanism to enable customization of action descriptors on most scenarios: the ApplicationModel.

The application model is a representation of the application components discovered by the framework, namely controllers and their actions. As illustrated below, this model includes a lot of metadata on the different components. Note that the aforementioned filters and action constraints are also part of the application model.

mvc-appmodel

After the whole model is discovered by the framework, we’re given a chance to customize it before the final action descriptors are created. This customizations are applied using application model conventions:

public interface IApplicationModelConvention
{
    void Apply(ApplicationModel application);
}

On previous versions of Web API, one could prefix action methods with the HTTP method name to make them apply to that method (e.g. GetNNNN, PostNNNN). This convention is not present on ASP.NET Core MVC, but the behavior is easy to mimic using an application model convention.

public class HttpMethodActionPrefixConvention : IApplicationModelConvention
{
    public void Apply(ApplicationModel application)
    {
        var actions = application.Controllers.SelectMany(c => c.Actions);
        foreach (var action in actions)
        {
            if (action.ActionName.StartsWith("get", StringComparison.OrdinalIgnoreCase))
            {
                var selector = action.Selectors.First();
                selector.ActionConstraints.Add(new HttpMethodActionConstraint(new[] { HttpMethod.Get.Method }));
                action.ActionName = action.ActionName.Substring(3);
            }
            // ...
        }
    }
}

In this example, for each action that was collected into the application model I’m checking if its name starts with an HTTP method. If so, a constraint is added to the ActionModel and the final action name is adjusted. This way, if we have a GetData action it will be constrained to HTTP GET and the action name throughout the application (e.g. route matching) will be Data. Finally, to use the convention we need to add it to MVC options in the Startup class:

services.AddMvc(options =>
{
    options.Conventions.Add(new HttpMethodActionPrefixConvention());
});

As a final note, one might ask why the application model was introduced, instead of having the developers directly modify action descriptors. The application model provides a customization model that is closer to how a developer organizes the application (e.g. controllers that contain actions). The final action descriptors are more complex and there’s a lot of plumbing to be done to create them that probably shouldn’t surface to the developer. For instance, a single ActionModel might result on multiple ActionDescriptors, depending on attribute routing. The application model is simpler and easier to modify, leaving all the final hard work to the framework.

This post ended up quite long, but it made sense to me to describe the three mechanisms together. Hope this helps!

Cookie-based TempData for ASP.NET Core

When developing web applications, it is common to persist some small state across a POST-redirect-GET flow (e.g. feedback message to a user after some action). In ASP.NET, TempData is the appropriate mechanism for this scenario: it persists a value until the next time it is fetched.

The default TempData implementation on ASP.NET MVC 5 and ASP.NET Core MVC is based on session storage. Another approach is to use a cookie as storage, making a client’s TempData flow with him. On ASP.NET MVC 5 I’ve been successfully using Brock Allen’s CookieTempData provider for the last few years. However, it doesn’t support ASP.NET Core.

I’ve implemented a cookie-based TempData provider for ASP.NET Core MVC. The code is available on GitHub and the library is published as a NuGet package. To replace the default provider with the cookie-based one, you just need to add a call to AddCookieTempData on your ConfigureServices method:

public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc();
    services.AddCookieTempData();
}

The cookies are serialized in BSON and protected using the new data protection APIs. Feel free to provide feedback. Hope this helps!

ASP.NET Core hosting (revisited) – Part II

Last year I did a series of posts following ASP.NET Core execution in depth. At that time the framework was still named ASP.NET 5, I was using betas 5/6 and DNX (.NET Execution Environment) was still around. A lot has changed since then, namely the usage of .NET CLI instead of DNX. Today I’ll continue my previous post on hosting and somehow revisit last year’s post about ASP.NET request processing.

The last post ended up right before the WebHost is initialized. This is where ASP.NET configures the application services and middlewares in order to build the final RequestDelegate. Finally, when started, the host fires up the registered server (IServer), linking the incoming requests to the RequestDelegate. Lets go through these steps.

The initialization of WebHost is where the Startup class plays its role. Internally, the startup logic is represented by the IStartup interface, depicted below:

public interface IStartup
{
    IServiceProvider ConfigureServices(IServiceCollection services);
    void Configure(IApplicationBuilder app);
}

This interface mimics the two application startup actions: 1) configure services on DI and 2) configure the middlewares. On the Main method, we invoked UseStartup<T>; if you take a look at the method’s source code you’ll see that it actually registers an implementation of IStartup that resolves the two actions by convention, by looking up methods on the supplied startup class.

Anyway, the first actions on the WebHost initialization are all about gathering the required services and configuring the application services (the following lines are a bit scattered on the source; I condensed the most relevant parts here):

_startup = _hostingServiceProvider.GetRequiredService<IStartup>();
_applicationServices = _startup.ConfigureServices(_applicationServiceCollection);
Server = _applicationServices.GetRequiredService<IServer>();
var builderFactory = _applicationServices.GetRequiredService<IApplicationBuilderFactory>();
IApplicationBuilder builder = builderFactory.CreateBuilder(Server.Features);

Notice how the host gets required services that were previously configured by the host builder and our code on the Main method.

At this point the application services are configured on the DI infrastructure and the host has an application builder ready to start configuring middlewares. There are three main players on that stage:

  • IApplicationBuilder – used to register middlewares. The interface is very simplistic:
    public interface IApplicationBuilder
    {
        // ...
        RequestDelegate Build();
        IApplicationBuilder Use(Func<RequestDelegate, RequestDelegate> middleware);
    }
  • The Startup.Configure method – where the application configures its middlewares. To that end, it gets an IApplicationBuilder as an argument, as shown below:
    public class Startup
    {
        public void Configure(IApplicationBuilder app) { ... }
    }
  • Startup filters – classes that implement IStartupFilter, allowing configurations to be done on the IApplicationBuilder before the Startup.Configure method is invoked. The interface is as follows:
    public interface IStartupFilter
    {
        Action<IApplicationBuilder> Configure(Action<IApplicationBuilder> next);
    }

The only usage of these filters I found on ASP.NET source is to register a middleware that handles the per-request lifetime on the DI infrastructure. It’s important to ensure that this middleware executes before application middlwares so that the latter can use per-request services.

The construction of the aplication RequestDelegate starts exactly by applying startup filters. The bootstrap Action<IApplicationBuilder> is the Startup.Configure method and filters successively wrap around the previous action. Note that filters are applied in reverse order of registration, so that the first to be registered is actually the outermost wrapper (the first to execute).

var startupFilters = _applicationServices.GetService<IEnumerable<IStartupFilter>>();
Action<IApplicationBuilder> configure = _startup.Configure;
foreach (var filter in startupFilters.Reverse())
{
    configure = filter.Configure(configure);
}

Having this composition of middleware configuration actions, it’s time to actually execute them on the IApplicationBuilder and then build the RequestDelegate.

configure(builder);
return builder.Build();

After the configure delegate is executed, the application builder contains and ordered list of middlewares, as shown below. For this example I have two middlewares on my Startup class; the other is the aforementioned middleware that handles per-request DI.

middlewares

Recall that a middleware is a function that composes a RequestDelegate around another (the next one in the chain). Therefore, the final RequestDelegate must be built from the last middleware to the first, as illustrated below. This ensures that the first middleware to be registered is actually the outermost wrapper (the first to execute).

Untitled.png

The application builder’s Build method does exactly that: reverse the middlewares list and invoke them one by one, passing the current RequestDelegate. To bootstrap the configuration, it uses a default RequestDelegate that returns an HTTP 404 status code, ensuring a default response when no other middleware handles the request.

public RequestDelegate Build()
{
    RequestDelegate app = context =>
    {
        context.Response.StatusCode = 404;
        return Task.FromResult(0);
    };
    foreach (var component in _components.Reverse())
    {
        app = component(app);
    }
    return app;
}

At this point the WebHost is initialized and we have a RequestDelegate that represents the whole application. When the host is started (its Run method is invoked on the console application) it’s time to bring the bring the server implementation in and have it deliver requests to the RequestDelegate. To that end, one last component is used: IHttpApplication.

public interface IHttpApplication<TContext>
{
    TContext CreateContext(IFeatureCollection contextFeatures);
    Task ProcessRequestAsync(TContext context);
    void DisposeContext(TContext context, Exception exception);
}

This interface is the glue between the server and our application. This is visible on the IServer interface:

public interface IServer : IDisposable
{
    void Start<TContext>(IHttpApplication<TContext> application);
    // ...
}

Just by analyzing these two interfaces we can imagine a skeleton of what’s happening under the covers, probably with some actions in between:

  1. When the server is started, it is given an IHttpApplication.
  2. When a new request arrives on the server, it asks the application  to create a context for the request.
  3. The context is supplied to the application for request processing. This means that the context must contain everything the application needs to handle the corresponding request.
  4. After the request is done, the server asks the application to dispose of the previously created context.

So, what’s that IFeatureCollection thing on CreateContext? Features are fine-grained representations of the capabilities exposed by the server for the current request. There are feature interfaces to represent the HTTP connection, the request, the response, and so on. You can take a look at all the features on the HttpAbstractions repository.

If you recall, the RequestDelegate gets an HttpContext and returns a Task. It is the hosting layer’s responsibility to wrap the low-level feature interfaces exposed by the server in a higher-level, strongly-typed HttpContext. This is done via the built-in implementation of IHttpApplication:

private readonly RequestDelegate _application;
private readonly IHttpContextFactory _httpContextFactory;

public Context CreateContext(IFeatureCollection contextFeatures)
{
    var httpContext = _httpContextFactory.Create(contextFeatures);
    // ...
    return new Context
    {
        HttpContext = httpContext,
        // ...
    };
}

public Task ProcessRequestAsync(Context context)
{
    return _application(context.HttpContext);
}

Going back a bit, when we start a WebHost, it actually starts the underlying server passing in the IHttpApplication and everything is ready for request processing. When a request comes in and the server invokes IHttpApplication.ProcessRequestAsync, the HttpContext is passed into the application RequestDelegate and we’re immediately executing middlewares’ code. There’s literally nothing else on the way of the request! Pretty slim, right?

execution

As a final note – and in case you’re wondering – it’s worth mentioning that HttpContext itself is an abstraction for applications to build on. The IHttpContextFactory implementation creates instances of DefaultHttpContext, which in turn uses DefaultHttpRequest and DefaultHttpResponse. These classes contain the logic of mapping feature interfaces into the high-level APIs.

The two sets of abstractions – feature interfaces and HTTP abstractions – allow applications to be hosted on different servers and different servers to be used without being tied to one specific hosting model, as illustrated below.

abstractions.PNG

And that’s it for this series. I revisited the execution of an ASP.NET Core application, starting from configuration via WebHostBuilder up to the moment where a request is handled to the application RequestDelegate. Hope this helps!