Referencing system assemblies in Roslyn compilations

With the recent stable release of the Roslyn packages (.NET Compiler Platform) I finally decided to give it a try. I’ve found it pretty simple to compile some code in runtime, load the resulting assembly and execute it. Overall it’s pretty impressive, given the amount of work that’s being done.

There are loads of resources on Roslyn online, namely Josh Varty’s Learn Roslyn Now series and Rolsyn’s documentation on GitHub. On this post, however, I’ll focus one aspect that wasn’t obvious to me in the first place: referencing shared framework assemblies.

When writing code it’s very likely that one ends up using types defined in reference assemblies such as System.Runtime and System.Runtime.Extensions, specially with .NET Standard  and .NET Core’s pay-for play model. These assemblies are just facades and the used types will eventually be forwarded to implementation assemblies in runtime, depending on the target framework.

Going back to Roslyn, the idea is that we’ll be compiling/generating code in runtime, so the question arises: which assemblies should I reference during the compilation? The only satisfying explanation I’ve found is a bit lost in a GItHub issue, hence this post.

There are two main approaches we can use during compilation: 1) use runtime (implementation) assemblies; and 2) use reference (contract) assemblies.

The remainder of this post is based on the following sample program that uses Roslyn. Note that the code to compile (code variable) uses DateTime (defined in reference assembly System.Runtime) and Math (defined in reference assembly System.Runtime.Extensions).

class Program
{
    static void Main(string[] args)
    {
        var code = @"
        using System;
        public class Test
        {
            public double MagicNumber => Math.Round(42.42 * DateTime.Now.Second);
        }
        ";

        var syntaxTree = CSharpSyntaxTree.ParseText(code);

        List references = // TODO define references here!
        var options = new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary, optimizationLevel: OptimizationLevel.Release, allowUnsafe: false);

        var compilation = CSharpCompilation.Create(Guid.NewGuid().ToString("N"), new[] { syntaxTree }, references, options);

        using (var ms = new MemoryStream())
        {
           var compilationResult = compilation.Emit(ms);
           if (compilationResult.Success)
           {
                ms.Position = 0;
                var assembly = AssemblyLoadContext.Default.LoadFromStream(ms);
                var instance = Activator.CreateInstance(assembly.DefinedTypes.First().UnderlyingSystemType);
               // Do something with "instance"
            }
            else
            {
                foreach (var error in compilationResult.Diagnostics)
                {
                   Console.WriteLine(error.ToString());
                }
            }
        }
     }
}

Option 1 – Runtime assemblies

Since we’re invoking Roslyn, we’re already in the context of an executing application targeting a specific framework. This framework includes the runtime (implementation) assemblies that we need, so the first option for compilation is referencing those assemblies. But how can we know their location in a flexible way?

The .NET Core runtime defines a so called “trusted platform assemblies” list, accessible via the AppContext class:

var trustedAssembliesPaths = ((string)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")).Split(Path.PathSeparator);

This list contains the locations of all the assemblies loaded from trusted locations, namely the ones on shared framework installations and NuGet caches. Example:

"C:\Program Files\dotnet\shared\Microsoft.NETCore.App\1.1.1\System.Linq.dll"
"C:\Users\luis\.nuget\packages\system.xml.xpath\4.3.0\lib\netstandard1.3\System.Xml.XPath.dll"

The aforementioned list contains a lot of assemblies that we probably don’t need (and don’t want to make available!) in our code, so it’s probably a good idea to filter the list. After that, creating a MetadataReference is straightforward. The full code to obtain the references that my code needs is:

var trustedAssembliesPaths = ((string)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")).Split(Path.PathSeparator);
var neededAssemblies = new[]
{
    "System.Runtime",
    "mscorlib",
};
List references = trustedAssembliesPaths
    .Where(p => neededAssemblies.Contains(Path.GetFileNameWithoutExtension(p)))
    .Select(p => MetadataReference.CreateFromFile(p))
    .ToList();

Pros:

  • All the needed assemblies are made available by the framework. No need to include additional resources on the application package.

Cons:

  • Need to know the actual runtime (implementation) assemblies. Note that my code depends on System.Runtime.Extensions facade, but I actually add a reference to mscorlib.
  • If an implementation assembly is updated (e.g. shared framework updated) the code might break (it’s unlikely, but possible). Even though .NET APIs should be backward compatible, “compiler overload resolution might prefer an API added in the new version instead of the one that it used to pick before

Option 2 – Reference assemblies

Instead of relying on runtime assemblies, we can compile the code against reference assemblies. The NuGet packages for System.Runtime and alike include both implementation and reference/contract assemblies (you can see this on your local NuGet cache).

System.Runtime/4.3.0/
                   \_ lib/net462/System.Runtime.dll
                   \_ ref/netstandard1.5/System.Runtime.dll

As these assemblies are in a well know location, we can easily create MetadataReferences:

var nugetCache = Environment.GetEnvironmentVariable("UserProfile") + @"\.nuget\packages\";
List references = return new List<MetadataReference>
{
  MetadataReference.CreateFromFile(nugetCache + @"System.Runtime\4.3.0\ref\netstandard1.5\System.Runtime.dll"),
  MetadataReference.CreateFromFile(nugetCache + @"System.Runtime.Extensions\4.3.0\ref\netstandard1.5\System.Runtime.Extensions.dll"),
}
.ToList();

Note that in the sample above I’m using the local NuGet cache. In a real-world scenario you’d include the reference assemblies for the chosen .NET Standard version in the application package (e.g. as resources).

When the assembly generated using Roslyn is loaded, its assembly references are resolved into the corresponding runtime assemblies.

Pros:

  • Assemblies are loaded from a well-known location.
  • Since the reference assemblies won’t change (unless you explicitly update them) there’s a guarantee that the code won’t break on future versions of the .NET Core runtime.

Cons:

  • Additional items on the application package.

Conclusion

Despite both approaches being valid and the first one being used by the C# Scripting API, I tend to prefer the second as it is more stable and less obscure. Hope this helps!

OAuth 2.0 library for .NET native apps

A few months ago I wrote about OAuth 2.0 for native apps, mentioning App Auth, an OAuth 2.0 client library for native Android and iOS applications. Recently I discovered that a similar library has been release for .NET! It is a PCL (Xamarin included) and it features the same security concerns as App Auth, following the IETF recommendations for native OAuth 2.0 clients. In addition, it also supports OpenId Connect features, such as validating identity  tokens. Very nice!

OAuth 2.0 mind map

Since the publication of The OAuth 2.0 Authorization Framework (RFC 6749) in 2012, both the IETF OAuth work group and the OpenID Foundation have released many other RFCs that extend/complement the base framework and the OAuth 2.0 ecosystem. I found this variety a bit confusing, so I figured I’d do a mind map and short summary of the specs for future reference. Here it goes.

oauth-mindmap

RFC 6749 – The OAuth 2.0 Authorization Framework

This is the core OAuth 2.0 standard. It defines the different roles (resource owner, client, authorization server, etc.) and the main authorization endpoints, flows and artifacts. Extensibility rules are also defined in this document. The “framework” on the RFC title is important: there are multiple authorization scenarios and a lot of aspects are left unspecified, which somehow justifies all the satellite specs. This was also the reason for heavy criticism on the early days of OAuth 2.0.

Tokens

RFC 6750 – Bearer Token Usage

The main spec doesn’t define the format of access tokens nor how they should be used to access protected resources. The most commonly deployed tokens, however, are bearer tokens. As the name implies, the possession of such tokens is sufficient to access the associated resources. RFC 6750 defines how bearer tokens should be sent to resource servers.

RFC 7009 – Token Revocation

Authorization servers usually allow resource owners to revoke previously granted authorizations. This RFC proposes an additional endpoint for authorization servers that clients can use to invalidate tokens they no longer need.

RFC 7662 – Token Introspection

The main spec doesn’t cover the actual usage of access tokens nor the procedure that resource servers should apply to verify them. More than verifying if a token is valid, resource servers are likely to need the associated metadata (e.g. the granted scopes) for authorization decisions. In a common deployment scenario, both authorization and resource servers are under the same domain, hence being able to somehow share the token validation logic. For more loose scenarios, RFC 7662 defines a token introspection endpoint that resource servers may use to query the authorization server to obtain the status and metadata of a token that is presented by an OAuth 2.0 client.

RFC 7519 – JSON Web Token (JWT)

JSON Web Token (JWT) is a compact, URL-safe means of representing claims to be transferred between two parties. The claims in a JWT are encoded as a JSON object.” JWTs can be used with JSON Web Signature (JWS) and JSON Web Encryption (JWE), enabling the claims to be integrity protected and/or encrypted. The spec includes a set of predefined claims – such as the issuer, subject and audience identifiers – as well as processing rules. JWTs can be used as self-contained access/refresh tokens.

More on obtaining authorization grants

RFC 7636 – Proof Key for Code Exchange by OAuth Public Clients (PKCE)

Clients that can’t keep their credentials private are called public clients. This is the case of most native applications. As such, the authorization code grant cannot be securely used by this type of clients. This spec defines a proof of possession extension that makes the code grant usable by these clients. Read this post for more details on PKCE and the underlying vulnerability.

Device Flow (draft)

Defines an authorization flow for clients executing on devices with limited input and/or browsing capabilities (e.g. TVs, STBs). It is based on out-of-band codes that the user provides to the authorization server using another device with better user-agent capabilities (e.g. a tablet).

Multiple Response Type Encoding Practices

Defines rules to encode authorization response parameters when multiple response types are used. Also defines response mode, a new authorization request parameter that allows the client to specify how the response should be delivered (query-string, fragment, etc.).

Form Post Response Mode

Based on the previous spec, this document defines a new response mode on which the authorization response is returned to the client via a auto-submit HTML form using an HTTP POST. This is different from the redirects (HTTP GET) defined on the core OAuth spec, with the advantage of reducing the likelihood of codes and tokens being logged as part of URLs.

Management

RFC 7591 – Dynamic Client Registration Protocol

Clients are usually registered by their developers in the intended authorization servers. This spec defines a means of dynamically registering clients Registration requests include items such as redirect URIs, intended grant types and web page URLs, and the response contains the assigned client ID and secret. There’s also a complementary experimental spec for subsequent changes to the client metadata.

Authorization Server Discovery Metadata (draft)

This draft defines the format of a JSON document that a client can use to obtain the different endpoints and capabilities of an authorization server. This metadata includes items such as the URL of the token endpoint and the supported response types. Besides this draft there’s also the OpenID Connect version, which is final and actually appeared first. The draft is compatible with the later.

Other types of authorization grants

RFC 7521 – Assertion Framework for Client Authentication and Authorization Grants

Defines mechanisms for OAuth 2.0 to work with other identity systems using assertions. The spec defines processing rules and message flows for using assertion-based client authentication and authorization grants when interacting with the token endpoint. This means that authorization grants are obtained by other means, instead of using the common OAuth flows. There’s a profile using SAML 2.0 assertions and another using JWT assertions.

Informational & best practices

RFC 6819 – Threat Model and Security Considerations

Defines a set of threats to the authorization flows and the corresponding countermeasures. Since OAuth 2.0 doesn’t include any “message security” on the base flows, being aware of those security aspects is very important to ensure the protocols’ security.

OAuth 2.0 for Native Apps (draft)

Native clients – such as mobile applications – can’t keep their credentials private. This spec defines a set of best practices for implementing such clients, namely the usage of the device’s native browser for authorization flows. I’ve discussed this subject in detail on a previous post.

Authentication

OpenID Connect Core 1.0

OpenID Connect 1.0 is a simple identity layer on top of the OAuth 2.0 protocol“. Actually it’s not that simple; it’s a whole new subject, but this post wouldn’t be complete without mentioning it. The spec defines extensions to OAuth 2.0 that enable clients to verify the identity of the user that authenticated on the authorization server.

Google’s Site Reliability Engineering

DevOps & Co

Here’s How Google Makes Sure It (Almost) Never Goes Down