ASP.NET Core on Azure Service Fabric: accessing Key Vault with Managed Identities (and the errors on the token endpoint!)

Managed Identities are an Azure feature that provides automatically managed identities to different Azure services (e.g. a VM) that can be used for authentication in other services (e.g. Key Vault). This avoids storing/managing credentials in the source code.

At the time of writing, VM Scale Sets support for Managed Identities is in preview. Since Service Fabric clusters are usually deployed on top of VM scale sets, it means that Managed Identities can be used with Service Fabric (tested with reliable services).

A common use-case is to have an ASP.NET Core service using Key Vault as a source of configuration values. If the cluster (actually the underlying VMs) are using a Managed Identity, you can authenticate into Key Vault using that identity. The easiest way to setup this scenario is using the Microsoft.Extensions.Configuration.AzureKeyVault package, which provides a AddAzureKeyVault extension method for IConfiguration. There are many articles on how to do this integration and configure the service identities and access policies.

I was recently doing this setup and I started getting the following exception when loading the configuration from Key Vault:

AzureServiceTokenProviderException
Parameters: Connectionstring: [No connection string specified], Resource: https://vault.azure.net, Authority: https://lo
gin.windows.net/72f988bf-86f1-41af-91ab-2d7cd011db47. Exception Message: Tried the following 3 methods to get an access
token, but none of them worked.

The error actually happens when the AzureServiceTokenProvider is trying to obtain a token to access the Vault using the service’s Managed Identity.

Lesson #1: pass “RunAs=App;” in the connectionString parameter  of AzureServiceTokenProvider. This way it will not try different modes to obtain a token, and the exception is a bit better.

The error now is something like: “cannot establish a connection because the target machine actively refused ir”.

Lesson #2: install/update the latest version of Microsoft.Azure.Services.AppAuthentication. The aforementioned package with the configuration extensions doesn’t depend on the latest version of this one. It happens that there’s an hard-coded URL of the endpoint that is used to get the Vault access token… and that endpoint changed since there was an “old way” of getting those tokens that is being deprecated.

After updating I got everything working. The versions I used for the packages are:

  • Microsoft.Extensions.Configuration.AzureKeyVault: 2.1.1
  • Microsoft.Azure.Services.AppAuthentication: 1.0.3 (version 1.0.1 won’t work with Managed Identities on Service Fabric).

All the “get the token” behavior is hidden in the AppAuthentication package and since it was an indirect dependency it was hard to spot where the problem was. Hope this helps!

Advertisement

2 thoughts on “ASP.NET Core on Azure Service Fabric: accessing Key Vault with Managed Identities (and the errors on the token endpoint!)

  1. Hey. This is the relevant part:

    // Service runs using an Azure Managed Identity, which can be used to access Azure resources.
    var azureTokenProvider = new AzureServiceTokenProvider(“RunAs=App;”);

    // Load secrets from Key Vault into configuration, using a prefix to filter out values
    var keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureTokenProvider.KeyVaultTokenCallback));
    configuration.AddAzureKeyVault($”https://{keyVaultName}.vault.azure.net/”, keyVaultClient, new PrefixKeyVaultSecretManager(keyVaultItemsPrefix));

    “keyVaultName” and “keyVaultItemsPrefix” are variables loaded from JSON configuration files. You can find “PrefixKeyVaultSecretManager” implementation on Microsoft docs.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s