Tuesday, 5 August 2025

Asp.Net - Using Polly .NET Resiliency V9

This article explains how using Polly .NET Resiliency library to add support for different resiliency strategies in Asp.net Core. The Polly library is created by Microsoft Community and its site is here:

https://www.pollydocs.org/

Important note about Polly library

The Polly library has changed its API quite a bit from v7 to v8 and now to v9. While Polly has been a Microsoft Community project, Microsoft now takes a bit more lead with Polly v9 being more tightly coupled to Microsoft's ecosystem. Future v10 is expected to be not so much changed as earlier major versions of Polly, only stabilize and
improve. The following Nuget libraries are added to the demo solution, SwashBuckle Nuget also added here for Swagger support used when testing.

<ItemGroup>
  <PackageReference Include="Microsoft.Extensions.Http.Polly" Version="9.0.7" />
  <PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="9.7.0" />
  <PackageReference Include="Swashbuckle.AspNetCore" Version="9.0.3" />
</ItemGroup>



The source code shown in this article is available to clone from my Github repo here: https://github.com/toreaurstadboss/HttpClientUsingPolly

Extension methods to add Polly v9 support http clients

The following extension methods offers creating a HttpClient via IHttpClientFactory or via a pipeline provided by ResiliencyProvider. Note that only the retry resilience strategy is shown here. There are multiple strategies available with Polly.

PollyExtensions.cs



using Microsoft.Extensions.Http.Resilience;
using Microsoft.Extensions.Logging;
using Polly;
using Polly.Retry;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;

namespace HttpClientUsingPolly
{

    /// <summary>
    /// Contains helper methods to add support for Polly V9 resilience strategies
    /// </summary>
    public static class PollyExtensions
    {
      
        public static void AddPollyHttpClient(this IServiceCollection services)
        {
            services.AddHttpClient(GithubEndpoints.HttpClientName, client =>
                {
                    client.DefaultRequestHeaders.UserAgent.Add(new ProductInfoHeaderValue("MyApp", "1.0"));
                })
                .ConfigurePrimaryHttpMessageHandler(() =>
                {
                    var handler = new RandomHttpErrorHandler(errorChance: 75);
                    handler.InnerHandler = new HttpClientHandler(); // Assign the terminal handler
                    return handler;
                }) //IMPORTANT to make Polly retry here using ConfigurePrimaryHttpMessageHandler and set the Innerhandler to HttpClientHandler
                .AddResilienceHandler(RetryResiliencePolicy, builder =>
                {
                    var serviceProvider = services.BuildServiceProvider();
                    var logger = serviceProvider.GetRequiredService<ILoggerFactory>().CreateLogger("NamedPolly");

                    var options = CreateRetryStrategyOptions(logger);
                    builder.AddRetry(options);
                });
        }    

        public static void AddNamedPollyPipelines(this IServiceCollection services)
        {
            services.AddResiliencePipeline<string>(RetryResiliencePolicy, builder =>
            {
                var serviceProvider = services.BuildServiceProvider();
                var logger = serviceProvider.GetRequiredService<ILoggerFactory>().CreateLogger("NamedPolly");

                builder.AddRetry(new RetryStrategyOptions
                {
                    MaxRetryAttempts = 3,
                    DelayGenerator = RetryDelaysClient,
                    ShouldHandle = new PredicateBuilder().Handle<HttpRequestException>(),
                    OnRetry = args =>
                    {
                        var httpEx = args.Outcome.Exception as HttpRequestException;
                        logger.LogInformation($"[NamedPolicy] Retrying due to: {httpEx?.Message}. Attempt: {args.AttemptNumber}");
                        return default;
                    }
                });
            });
        }

        private static HttpRetryStrategyOptions CreateRetryStrategyOptions(ILogger logger)
        {
            HttpRetryStrategyOptions options = new HttpRetryStrategyOptions
            {
                MaxRetryAttempts = 3,
                DelayGenerator = RetryDelaysPipeline,
                ShouldHandle = args =>
                {
                    if (args.Outcome.Exception is HttpRequestException ||
                        args.Outcome.Exception is TaskCanceledException)
                        return ValueTask.FromResult(true);

                    if (args.Outcome.Result is HttpResponseMessage response &&
                        TransientStatusCodes.Contains(response.StatusCode))
                        return ValueTask.FromResult(true);

                    return ValueTask.FromResult(false);
                },
                OnRetry = args =>
                {
                    logger.LogInformation($"Retrying... Attempt {args.AttemptNumber}");
                    return default;
                }
            };

            return options;
        }

        public const string RetryResiliencePolicy = "RetryResiliencePolicy";

        static readonly HttpStatusCode[] TransientStatusCodes = new[]
        {
            HttpStatusCode.RequestTimeout,      // 408
            HttpStatusCode.InternalServerError, // 500
            HttpStatusCode.BadGateway,          // 502
            HttpStatusCode.ServiceUnavailable,  // 503
            HttpStatusCode.GatewayTimeout       // 504
        };

        static Func<RetryDelayGeneratorArguments<HttpResponseMessage>, ValueTask<TimeSpan?>>? RetryDelaysPipeline =
             args => CommonDelayGenerator(args.AttemptNumber);

        static Func<RetryDelayGeneratorArguments<object>, ValueTask<TimeSpan?>>? RetryDelaysClient =
             args => CommonDelayGenerator(args.AttemptNumber);


        static ValueTask<TimeSpan?> CommonDelayGenerator(int attemptNumber)
        {
            var delay = attemptNumber switch
            {
                1 => TimeSpan.FromSeconds(1),
                2 => TimeSpan.FromSeconds(2),
                3 => TimeSpan.FromSeconds(4),
                _ => TimeSpan.FromSeconds(0) // fallback, shouldn't hit
            };
            return new ValueTask<TimeSpan?>(delay);
        }

    }
}



Note that the use of ConfigurePrimaryHttpMessageHandler here took a lot of time to find out. The Polly docs did not offer a good example of using IHttpClientFactory combined with throwing errors, which will be shown later in this article in the RandomHttpErrorHandler DelegatingHandler. It is also important to configure the primary http message handler here before the resilience handler is added using AddResilienceHandler. The following summary of resilience strategies shows that Polly offers more strategies than just the retry strategy. Every resilience strategy here can be combined to offer multiple resilience strategy to obtain the desired level of QoS = Quality of Service.
Polly Resilience Strategies
Strategy Description Can Be Combined?
Retry Retries a failed operation based on rules (e.g., delay, max attempts). ✅ Yes
Circuit Breaker Stops calls when failures exceed a threshold, allowing time to recover. ✅ Yes
Timeout Limits how long an operation can run before it's considered failed. ✅ Yes
Bulkhead Limits concurrent executions to prevent overload. ✅ Yes
Fallback Provides an alternative result or action when the primary fails. ✅ Yes
Rate Limiter Controls the rate of requests to avoid overwhelming a service. ✅ Yes


Adding support to test out random http request errors

To verify that the retry resilience strategy works, the following DelegatingHandler shown in the code in PollyExtensions will random throw HttpRequestException with a set estimated failure rate in percentage to be able to to end-to-end testing of retries. Debugging the endpoints to be defined later in this article shows the retries are working with this setup.

RandomHttpErrorHandler.cs



namespace HttpClientUsingPolly
{

    public class RandomHttpErrorHandler : DelegatingHandler
    {

        private readonly Random _random = new();
        private readonly double _errorChance;

        public RandomHttpErrorHandler(double errorChance)
        {
            _errorChance = errorChance;
        }

        protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
        {
            if (_random.NextDouble() < (_errorChance / 100))
            {
                // Pick a random transient status code
                var httpErrorsPossible = new[]
                {
                        System.Net.HttpStatusCode.RequestTimeout,        // 408
                        System.Net.HttpStatusCode.InternalServerError,   // 500
                        System.Net.HttpStatusCode.BadGateway,            // 502
                        System.Net.HttpStatusCode.ServiceUnavailable,    // 503
                        System.Net.HttpStatusCode.GatewayTimeout         // 504
                    };

                var chosenStatus = httpErrorsPossible[_random.Next(httpErrorsPossible.Length)];

                throw new HttpRequestException($"Simulated Http error: {(int)chosenStatus}", null, chosenStatus);

            }

            return await base.SendAsync(request, cancellationToken);
        }

    }
}




PollyPipelineHelper.cs

The following code makes it easier to use the ResiliencePipelineProvider that will look up configured resilience pipelines that are defined in the method AddNamedPollyPipelines further up in the article.



using Polly.Registry;

namespace HttpClientUsingPolly
{
    public static class PollyPipelineHelper
    {

        public static ValueTask<HttpResponseMessage> ExecuteWithPolicyAsync(
            this ResiliencePipelineProvider<string> pipelineProvider,
            string policyName,
            Func<CancellationToken, Task<HttpResponseMessage>> action,
            CancellationToken cancellationToken = default)
        {
            var pipeline = pipelineProvider.GetPipeline(policyName);

            return pipeline.ExecuteAsync(
                async ct => await action(ct),
                cancellationToken);
        }

    }
}





Program.cs

The folllowing setup is done in Program.cs, using the extension methods defined further up in the article. The setup below is the setup for the demo code, which is an Asp.net Web Api application. Minimal Api endpoints will be defined for the demo.



namespace HttpClientUsingPolly
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var builder = WebApplication.CreateBuilder(args);

            // Add services to the container.

            builder.Services.AddControllers();

            builder.Services.AddSwaggerGen();

            builder.Services.AddPollyHttpClient(); //set up polly v9 enabled http client (http client will be created via IHttpClientFactory)
            builder.Services.AddNamedPollyPipelines(); // set up polly v9 enabled resilience pipepline (http client will be created via ResiliencePipelineProvider 

            var app = builder.Build();

            // Configure the HTTP request pipeline.

            app.UseHttpsRedirection();

            app.UseAuthorization();

            app.MapGitHubUserEndpoints();

            app.MapControllers();

            if (app.Environment.IsDevelopment())
            {
                app.UseSwagger();
                app.UseSwaggerUI();
            }

            app.Run();
        }
    }
}




Finally, the following Minimal Api endpoint methods first uses the resilience provider. As we can see, there is quite a bit of setup to make use of Polly using a resilience pipeline.

Minimal Api Endpoints using Polly enabled resilience pipeline

The following Github endpoints uses Polly resilience pipeline defined further up in the article. Both the IHttpClientFactory and ResiliencePipelineProvider are injected from DI container using the [FromService] attribute. There is some juggling to get to the httpClient to perform the request.

GithubEndpoints.cs



using Microsoft.AspNetCore.Mvc;
using Polly.Registry;
using System.Net.Http.Headers;
using System.Text.Json;

namespace HttpClientUsingPolly
{

    public static class GithubEndpoints
    {

        public const string HttpClientName = "GitHubClient";

        public static void MapGitHubUserEndpoints(this WebApplication app)
        {
            app.MapGet("/github-v1/{username}", async (
                string username,
                [FromServices] IHttpClientFactory httpClientFactory,
                [FromServices] ResiliencePipelineProvider<string> resiliencePipelineProvider) =>
            {
                string url = $"https://api.github.com/users/{username}";

                using var client = httpClientFactory.CreateClient(HttpClientName);
                client.DefaultRequestHeaders.UserAgent.Add(new ProductInfoHeaderValue("MyApp", "1.0"));

                var response = await resiliencePipelineProvider.ExecuteWithPolicyAsync(
                    PollyExtensions.RetryResiliencePolicy,
                    ct => client.GetAsync(url, ct));

                response.EnsureSuccessStatusCode();
                var json = await response.Content.ReadAsStringAsync();
                var user = JsonSerializer.Deserialize<JsonElement>(json);

                return Results.Json(user);
            });

            app.MapGet("/test-retry-v1", async (
                [FromServices] IHttpClientFactory httpClientFactory,
                [FromServices] ResiliencePipelineProvider<string> resiliencePipelineProvider) =>
            {
                using var client = httpClientFactory.CreateClient(HttpClientName);

                var response = await resiliencePipelineProvider.ExecuteWithPolicyAsync(
                    PollyExtensions.RetryResiliencePolicy,
                    ct => client.GetAsync("https://example.com", ct));

                return Results.Json(response);
            });

        }

    }

}




Simpler use of Polly via IHttpClientFactory

The following Minimal API endpoints shows a simpler usage of Polly, where the use of Polly retry resilience strategy are implicit. It was defined in the method AddPollyHttpClient further up in this article.


    //use keyed httpclient and do not go via pipeline provider 

    app.MapGet("/github-v2/{username}", async (
       string username,
       [FromServices] IHttpClientFactory httpClientFactory) =>
    {
        string url = $"https://api.github.com/users/{username}";
        using var client = httpClientFactory.CreateClient(HttpClientName);

        var response = await client.GetAsync(url);
        response.EnsureSuccessStatusCode();

        var json = await response.Content.ReadAsStringAsync();
        var user = JsonSerializer.Deserialize<JsonElement>(json);
        return Results.Json(user);
    });

    app.MapGet("/test-retry-v2", async (
        [FromServices] IHttpClientFactory httpClientFactory) =>
    {
        using var client = httpClientFactory.CreateClient(HttpClientName);

        var response = await client.GetAsync("https://example.com");

        return Results.Json(response);
    });


As shown above, the Minimal API endpoints above uses IHttpClientFactory. The complexity of Polly is hidden and we can focus in the minimal API method to instantiate our HttpClient and make request and then retrieve the response content and return the result in the minimal API endpoint.

Further resources - Videos showing more information about Polly

Polly Resilience Strategy Video Resources
Title Description Length Link
Visual Studio Toolbox Live – Ensuring Resilience with Polly Martin Costello walks through Polly strategies like Retry, Circuit Breaker, Timeout, and how to use them in .NET apps. 59:29 Watch
Learn Live – Implement Resiliency in a Cloud-Native ASP.NET Core Microservice Deep dive into code-based and infrastructure-based resiliency using Polly and service meshes like Linkerd. 1:25:06 Watch
Implementing Resilience Strategies with Polly in ASP.NET Core Step-by-step guide to applying retry, circuit breaker, and timeout policies in ASP.NET Core using Polly. ~15 min Watch
Coding Short: Resilient .NET Apps with Polly Shawn Wildermuth gives a concise overview of Polly with practical examples of retry and circuit breaker usage. 14:07 Watch
Building Resilient Cloud Services with .NET 8 | .NET Conf 2023 Explores .NET 8 resiliency features and Polly integration in cloud-native applications. ~45 min Watch

Monday, 28 July 2025

Azure Keyvault and HMAC-SHA256 Pseudonymizer

In this article, using Azure Keyvault together with HMAC-SHA256.

Introduction to HMAC SHA-256

The HMAC-SHA256 is a SHA-256 cryptographic hashing function that combines SHA-256 with a secret key (that can be shared) to provide both data integrity (to ensure that data has not been altered) and data authenticity (that data originates from a sender that knows about the shared secret key of the HMAC-SHA256 hash). HMAC-SHA256 generates a Message Authentication Code (MAC)
that we can get both these benefits from, data authenticity and data integrity. Since it is 256 bits or 32 bytes, the hash can be represented as hexadecimal string of length 64. If it is represented as a base-64 string, it will be approximately 44 bytes long, given that a base-64 string will be padded in multiple of 4 characters.

Storing the shared secret key in Azure Keyvault

Azure Keyvault will be used to store the shared secret key. The secret key could be stored anywhere, but using a KeyVault in Azure makes it easier to keep it safe and apply rolling policies for the secret key. First off, to create an Azure Keyvault secret, make sure you have an Azure Keyvault created. Search for a resource called Key vaults with a key icon. Choose Create, enter a Key vault name and Region and resource group, Pay per use and Standard pricing tier. (Premium includes HSM backed keys, which will use dedicated physical devices - HSM = Hardware Security Module - for enhanced FIPS 140-2 Level 3 certified security). Once the keyvault is created, choose Object and Secrets. Click Generate/Import. Just enter a name for the secret and the secret value. It is suggested that they secret value you enter should be genereated from a cryptographically secure random generator and a length of at least 16 bytes and 32 bytes.


Generating a strong shared secret key for HMAC SHA-256

The following code can be used to generate a strong 256 (32-byte) key, and pasted into the Secret value
HmacSha256KeyGenerator.cs


using System.Security.Cryptography;

namespace MinimalApiSecurityExperimentsDotNet.Extensions
{

    public static class HmacSha256KeyGenerator
    {

        public static string GenerateStrongKey()
        {
            // Generate a 256-bit (32-byte) key
            byte[] key = new byte[32];
            using (var rng = RandomNumberGenerator.Create())
            {
                rng.GetBytes(key);
            }

            // Convert to Base64 for storage or display
            string base64Key = Convert.ToBase64String(key);
            return base64Key;
            //Console.WriteLine("Generated HMAC-SHA-256 Key (Base64):");
            //Console.WriteLine(base64Key);
        }

    }

}



MinimalApiSecurityDemo.csproj
The following Nuget packages can be added to work with Azure Keyvault secrets.


  <PackageReference Include="Azure.Identity" Version="1.14.2" />
  <PackageReference Include="Azure.Security.KeyVault.Keys" Version="4.8.0" />
  <PackageReference Include="Azure.Security.KeyVault.Secrets" Version="4.8.0" />


Note that you do not need the Keys nuget package here if you only want to retrieve secrets. Azure KeyVault got three different object types - Keys, secrets and certificates. Only secrets will be looked at in this article.
KeyVaultSecretRetriever.cs
Let's look at key vault retrieval.


using Azure.Identity;
using Azure.Security.KeyVault.Secrets;

namespace MinimalApiSecurityExperimentsDotNet
{
    
    public class KeyVaultSecretRetriever
    {
        private readonly IConfiguration _configuration;
        public KeyVaultSecretRetriever(IConfiguration configuration)
        {
            _configuration = configuration;
        }

        public async Task<KeyVaultSecret>? GetSecretAsync(string secretName)
        {
            try
            {
                string keyVaultUri = _configuration["AzureKeyVault:VaultUri"] ?? throw new ArgumentNullException(nameof(keyVaultUri));
                var client = new SecretClient(new Uri(keyVaultUri), new DefaultAzureCredential());
                KeyVaultSecret secret = await client.GetSecretAsync(secretName);
                return secret;
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Error retrieving the secret {secretName}: {ex.Message}");
                return null;
            }
        }     

    }

}


The SecretClient in the code above is the client used to retrieve the secret from Azure KeyVault. The client is from the Azure.Security.KeyVault.Secrets Nuget package. The DefaultAzureCredential used here means you will have to login in advance before retrieving the secret. There are different credential types you can use instead, but this default credential type is the easiest to use for testing out. If you run the code above from the console, you can for example install the Azure Powershell module and run az login. The following appsettings file can be used for the code above.
appsettings.json


{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "AllowedHosts": "*",

  "AzureKeyVault": {
    "VaultUri": "https://yourkeyvaultsomewhereinazure.vault.azure.net/",
    "HmacSha256SecretName":  "YourExampleHmacStrongKey"
  }

}


Pseudonymizer class

The following pseudonymizer class uses HMAC-SHA256 to generate a pseudonym for a given message. This can be used for generating a value for sensitive data with a key. The key can be shared between sender and receiver to avoid exposing sensitive values in clear text.
Pseudonymizer.cs


using System.Text;

namespace MinimalApiSecurityExperimentsDotNet
{

    public class Pseudonymizer
    {
        private readonly byte[]? _key;

        public Pseudonymizer(string key)
        {
            if (!string.IsNullOrWhiteSpace(key))
            {
                _key = Encoding.UTF8.GetBytes(key);
            }
        }

        /// <summary>
        /// Outputs a pseudonymized version of the input string using HMAC SHA256.       
        /// </summary>
        /// <param name="message"></param>
        /// <param name="outputToHexString">Output to hex string if true. If false, base-64 string is returned</param>
        /// <returns></returns>
        public string? Pseudonymize(string? message, bool outputToHexString = true)
        {
            if (string.IsNullOrWhiteSpace(message) || _key == null)
            {
                return message;
            }
            using var hmac = new System.Security.Cryptography.HMACSHA256(_key);
            var hash = hmac.ComputeHash(Encoding.UTF8.GetBytes(message));
            return outputToHexString ? Convert.ToHexString(hash) : Convert.ToBase64String(hash);
        }

        public bool Verify(string input, string pseudonym)
        {
            if (string.IsNullOrEmpty(input) || string.IsNullOrEmpty(pseudonym))
            {
                return false;
            }
            var computedPseudonym = Pseudonymize(input);
            return computedPseudonym == pseudonym;
        }

    }

}


The code above shows that we can both psedonymize and verify the pseudonym. Next up, a provider for creating an instance of this Pseudonymizer for use in ASP.net Core, for example.
PseudonymizerProvider.cs


namespace MinimalApiSecurityExperimentsDotNet.Extensions
{

    public class PseudonymizerProvider
    {

        private Pseudonymizer? _pseudonymizer;

        public void Set(Pseudonymizer pseudonymizer)
        {
            _pseudonymizer = pseudonymizer;
        }

        public Pseudonymizer Get()
        {
            return _pseudonymizer ?? throw new InvalidOperationException("Pseudonymizer not initalized");
        }

    }

}


The following Hosted service sets up the pseudonymizer service. It will start up when the Asp.net core application is starting, however after the DI container has been built.

PseudonymizerHostedService .cs



using Azure.Security.KeyVault.Secrets;

namespace MinimalApiSecurityExperimentsDotNet.Extensions
{

    public class PseudonymizerHostedService : IHostedService
    {

        private readonly IServiceProvider _provider;

        public PseudonymizerHostedService(IServiceProvider provider)
        {
            _provider = provider;
        }

        public async Task StartAsync(CancellationToken cancellationToken)
        {
            using var scope = _provider.CreateScope();
            var config = scope.ServiceProvider.GetRequiredService<IConfiguration>();
            var keyVaultRetriever = new KeyVaultSecretRetriever(config);
            var hmacSha256SecretName = config["AzureKeyVault:HmacSha256SecretName"];
            if (hmacSha256SecretName == null)
            {
                throw new InvalidOperationException($"Config '{hmacSha256SecretName}' not is missing.");
            }
            KeyVaultSecret? secret = await keyVaultRetriever.GetSecretAsync(hmacSha256SecretName);

            if (secret == null)
            {
                throw new InvalidOperationException($"Secret '{hmacSha256SecretName}' not found in Key Vault.");
            }

            var pseudonymizer = new Pseudonymizer(secret.Value);

            // Register the pseudonymizer in the pseudonymizerProvider
            var pseudonymizerProvider = scope.ServiceProvider.GetRequiredService<PseudonymizerProvider>();
            pseudonymizerProvider.Set(pseudonymizer);
        }

        public Task StopAsync(CancellationToken cancellationToken) => Task.CompletedTask;

    }

}


In the Program.cs of a sample Asp.net Web API with minimal API, the provider and the hosted service is registered before building the webapplication with .Build().

Program.cs



using MinimalApiSecurityExperimentsDotNet.Extensions;

namespace MinimalApiSecurityExperimentsDotNet
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var builder = WebApplication.CreateBuilder(args);

            // Add services to the container.

            builder.Services.AddControllers();

            // Add Swagger services
            builder.Services.AddSwaggerGen();
            builder.Services.AddEndpointsApiExplorer();

            // Adding keyvault access via registering PseudonymizerHostedService 
            builder.Services.AddSingleton<PseudonymizerProvider>();
            builder.Services.AddHostedService<PseudonymizerHostedService>();

            var app = builder.Build();

            // Configure the HTTP request pipeline.

            app.UseHttpsRedirection();

            app.UseAuthorization();

            app.MapControllers();

            // Enable Swagger middleware 
            if (app.Environment.IsDevelopment())
            {
                app.UseSwagger();
                app.UseSwaggerUI();
            }
          
            app.MapGet("/generatestrongkey", () =>
            {
                return HmacSha256KeyGenerator.GenerateStrongKey();

            });

            app.MapGet("/pseudonymize", (string? key, string? value) =>
            {
                var pseudonym = new Pseudonymizer(key).Pseudonymize(value, outputToHexString: true);
                return Results.Json(pseudonym);
            });

            app.MapGet("/pseudonymize/verify", (string? key, string? value, string? pseudonym) =>
            {
                var pseudonymizer = new Pseudonymizer(key);
                var isValid = pseudonymizer.Verify(value, pseudonym);
                return Results.Json(isValid);
            });

            //Example using the injected singleton PseudonymizerProvider

            app.MapGet("/pseudo", (string? input, PseudonymizerProvider pseudonymizerProvider) =>
            {
                var pseudonymizer = pseudonymizerProvider.Get();
                var result = pseudonymizer.Pseudonymize(input); //pseudonymize the input using the inject Pseudonymizer instance. This will use HMAC SHA256 under the hood.
                return Results.Ok(new { message = input, pseudonym = result });
            });

            app.MapGet("/getkeyvaultsecret", async (string secretName) =>
            {
                var keyvaultRetriever = new KeyVaultSecretRetriever(builder.Configuration);
                var secret = await keyvaultRetriever.GetSecretAsync(secretName);
                return Results.Json(secret);
            });          

            app.Run(); 
        }
    }


}


The benefits of registering the pseudonymizer's hosted service and provider in the container is shown in the code above. We can easily pseudonymize and use Azure Key vault stored secret for the HMAC-SHA256 hash / pseudonoym.


 //Example using the injected singleton PseudonymizerProvider

app.MapGet("/pseudo", (string? input, PseudonymizerProvider pseudonymizerProvider) =>
{
    var pseudonymizer = pseudonymizerProvider.Get();
    var result = pseudonymizer.Pseudonymize(input); //pseudonymize the input using the inject Pseudonymizer instance. This will use HMAC SHA256 under the hood.
    return Results.Ok(new { message = input, pseudonym = result });
});


Summary

The pseudonymizer is initialized and registered as a singleton, it got a dependency against Azure Keyvault service at startup, but further on pseudonymizing can now use the singleton instance of the pseudonymizer that is registered by the hosted service at application startup. The pseudonymizer can for example pseudonymize sensitive values and hide their content. A receiver can be shown this pseudonymized value in the response. There pseudonymized value could also be described as for example one of these three descriptions:
  • Key identifier / alias
  • Obfuscated key named
  • Derived key label
If the receiver got the shared secret key, since it is a one-way cryptographic algorithm, obtaining the original value is still only possible via a brute-force dictionary attack. However, it can be used for example as a pseudo-id that is shared and also identify for example a shared identifier, for example a patient identifier for multiple registered documents in a database for example. The pseudo id can be even stored in the database as a way of obfuscating sensitive values such a SSN / PID (Patient Id, SSN = Social Security Number).

Note about Keyvault access

Just because you know the url to an Azure keyvault and possible the secret's name, does of course not mean you got access to the secret. Inside the Azure Keyvault, clik on Access Control (IAM) to get started. To just test things out, follow the guide by clicking the button Add role assignment. Follow the wizard to gain access to your key vault. Search for role Key Vault Administrator. Click Next. Choose Select members to choose for example your own user. Click Next Click Review and assign. Finally you have added access to your Key vault. This must be done even for the Key vault you have created yourself. Of course, if you use Azure AD, you can choose different members here and authorize fine-grained access.

Sunday, 20 July 2025

Object detection with Machine Learning

In this article, a demo performing object detection in images using machine learning is presented. ML.net will be used for the machine training in the Blazor serverside app the demo is made with. The demo will describe how you can train a machine learning model to detect stop signs in United States from some 50 images, downloaded from Unsplash website. You can of course choose whatever object to detect here and obtain sample images from for example sites such as Unsplash. There are no official lower limit on the needed images you need for reliable detection of objects in images using machine learning. Obviously, your images should include viewing the object from different angles and being partly obscured and similar variations, such as weathering effects (snow et cetera) and different lighting conditions (day, night, dusk, dawn and so on). High quality object detection will use thousands of images for training and use powerful GPUs in the cloud. I have trained the learning model on my own PC using a CPU. It took 830 seconds, some 13 minutes in all. ML.net supports computing machine learning models in the clouds, some cost may of course be expected instead of running in on cheaper hardware such as your own PC (CPU). The benefit of using GPUs in the cloud is that the machine learning model can use more diffusion models and compare which machine learning model is optimal. This demands a lot of computing. Note that the Modelbuilder of ML.net is used in VS 2022. This is available after installing the ML.net workload in VS 2022. I have also used VoTT to tag the 50+ images I used for training set. I have added the source code in my Github repo here:

https://github.com/toreaurstadboss/ObjectDetectionMachineLearning

VOTT - Visual Object Tagging Tool

ML.net object detection uses input from VoTT tool where a human (you!) has tagged the object(s) of interest in the image. Note that a stopsign along a road or street can be multiple places in an image. Some places for example, we see an object 2-3 times of course. Your objective in doing a good interpretation in training data (pictures!) of tagging and labelling objects of interest in Vott is a straight forward process. VoTT is available for download here, click in the releases of the Github repo to find latest stable installer (exe):


https://github.com/microsoft/VoTT/releases/tag/v2.2.0


https://github.com/microsoft/VoTT Vott repo on Github
Here is a screenshot of Vott while I am tagging pictures : You can open a folder with several pictures, I am using .png format, and then use the polygon tool to 'lasso around' the contours of each object of interest in the image. A label must be designated also. This process is manually repeated. A minimum of about 50 images such be considered, but a reliable trained machine learning model in productions should probably be trained with a lot more, in the several thousands. However, in the beginning, it would be benificial to keep the count of images low so finding the optimum machine learning algorithm is easier without such a large training set. In Ml.net, the algorithm ObjectDetectionMulti won, however also only one model was explored. Using GPUs in the cloud would at least reduce computation time and yield higher quality diffusion models.

Screenshot of the demo

The following screenshot shows three detected objects in the test image. The machine learned model is trained to detect Stop signs, used in road signs of roads and streets in United States. Interestingly, the machine learning model also detected the stop sign poining the other way for meeting traffic, so a total of three stop signs were detected in the rail road crossing sample image. The fourth stop sign was not detected, however, I have trained the data not on road signs rotated away from camera. Training also to detect stop signs in these cases would perfect this even better. But I am quite pleased with the results when I test out the machine learned model.

First, let's look at the code behind the demo shown in the screenshot above.

Machine learning model exposed via Web API

Using ML.net we can not only train a machine learning models to detect object in images, we can also via using ML.net generate the API that will process images and returns objects found in images, if any. ML.net object detection supports images of format .png, .jpeg and .bmp. In the demo, .png images were used. Actually the demo used high-res images from the web site Unsplash, which provides beautiful and royalty free of use images to use with for example meachine learning.

The API and endpoint for the processing of images is shown next. The client specifies the file name of the image to analyse and after processing, the resulted detected objects are returned. If any objects were found, we will get these objects in summary format of bounding boxes and detected labels from the machine learned model. ML.Net generated the Web api code here. Note that the POST endpoint was made manually by myself. We turn off caching here through response headers and note that the bounding box coordinates are returned scaled to a 800x600 virtual image. The bounding boxes are returned as quadruples (four floats per object) in the float array returned. The POST endpoing in the minimal API shown below shows how these bounding boxes are calculated and returned. I have not refactored this code yet into a helper method, but you can use this code as a reference to how to rescale the calculated bounding boxes from the virtual 800x600 image back to the original image pixel width and height.


Program.cs | StopsignDetection_webapi1 project




<!-- This file was auto-generated by ML.NET Model Builder. -->
using Microsoft.AspNetCore.Builder;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.ML;
using Microsoft.OpenApi.Models;
using Microsoft.ML.Data;
using System.Drawing;
using System.IO;
using System.Threading.Tasks;
using StopSignDetection_WebApi1;
using Microsoft.AspNetCore.Mvc;

<!-- Configure app -->
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddPredictionEnginePool<StopSignDetection.ModelInput, StopSignDetection.ModelOutput>()
    .FromFile("StopSignDetection.mlnet");

builder.Services.AddEndpointsApiExplorer();

builder.Services.AddSwaggerGen(c =>
{
    c.SwaggerDoc("v1", new OpenApiInfo { Title = "Object detection - Stop sign detection", Description = "Docs for my API", Version = "v1" });
});
var app = builder.Build();

app.UseSwagger();

if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI(c =>
    {
        c.SwaggerEndpoint("/swagger/v1/swagger.json", "Object detection - Machine learning MLN.");
    });
}

<!-- Define prediction route & handler -->
app.MapPost("/predict",
    async (HttpContext context, PredictionEnginePool<StopSignDetection.ModelInput, StopSignDetection.ModelOutput> predictionEnginePool, [FromBody] PredictRequest request) =>
    {
        context.Response.Headers["Cache-Control"] = "no-store, no-cache, must-revalidate, max-age=0";
        context.Response.Headers["Pragma"] = "no-cache";
        context.Response.Headers["Expires"] = "0";

        var image = MLImage.CreateFromFile(request.ImagePath);

        var input = new StopSignDetection.ModelInput()
        {
            Image = image,
        };

        int originalWidth = image.Width;
        int originalHeight = image.Height;

        const int virtualWidth = 800;
        const int virtualHeight = 600;

        var prediction = predictionEnginePool.Predict(input);
        var boxes = prediction.PredictedBoundingBoxes;

        for (int i = 0; i < boxes.Length; i += 4)
        {
            float left = boxes[i];
            float top = boxes[i + 1];
            float width = boxes[i + 2];
            float height = boxes[i + 3];

            float scaledLeft = left * originalWidth / virtualWidth;
            float scaledTop = top * originalHeight / virtualHeight;
            float scaledWidth = width * originalWidth / virtualWidth;
            float scaledHeight = height * originalHeight / virtualHeight;

            Console.WriteLine($"Box {i / 4}: X={scaledLeft}, Y={scaledTop}, Width={scaledWidth}, Height={scaledHeight}");

            (boxes[i], boxes[i + 1], boxes[i + 2], boxes[i + 3]) = (scaledLeft, scaledTop, scaledWidth, scaledHeight);  //assign using tuples and update the float array per object
        }

        return await Task.FromResult(prediction);
    });

app.Run();



PredictRequest is a simple class using as the request object while POST-ing.


PredictRequest.cs | StopsignDetection_webapi1 project

namespace StopSignDetection_WebApi1 { public class PredictRequest { public string ImagePath { get; set; } = string.Empty; } }

Over to the client, which is a Blazor serverside. The following UI is in the component that shows the UI with code-behind.

Home.razor




@page "/"
@using Microsoft.AspNetCore.Components.Forms

<PageTitle>Home</PageTitle>

<h1>Object detection using Machine learning</h1>

<script src="js/home.js" type="text/javascript"></script>

<p>
    Upload an image to use the Object detection demo. The machine-learned ML.Net model will detect <em>Stop signs</em> and
    display bounding boxes around each stop sign in the image. The stop sign is trained to use those used as traffic signs in United States
    along streets and roads.
</p>

<div class="container">

    <div class="row align-items-start">
        <div class="col">
            <label><b>Select a picture to run stop sign object detection</b></label><br />
            <InputFile OnChange="@OnInputFile" accept=".jpeg,.jpg,.png" />
            <br />
            <code class="alert-secondary">Supported file formats: .jpeg, .jpg and .png. (.bmp also supported) Max image file upload size : 10 MB</code>
            <br />
        </div>
    </div>

    <div class="row align-items-start">
        <div class="col">
            <label><b>Detected objects (stop-signs) in the loaded image:</b></label><br />

            @if (LatestPrediction?.predictedLabel?.Count > 0)
            {
                <table class="table table-bordered table-striped table-hover mt-3">
                    <thead class="table-dark">
                        <tr>
                            <th>#</th>
                            <th>Label</th>
                            <th>X1</th>
                            <th>Y1</th>
                            <th>X2</th>
                            <th>Y2</th>
                            <th>Confidence</th>
                        </tr>
                    </thead>
                    <tbody>
                        @for (int i = 0; i < LatestPrediction.predictedLabel.Count; i++)
                        {
                            var label = LatestPrediction.predictedLabel.ElementAt(i);
                            var bbox = LatestPrediction.predictedBoundingBoxes.Skip(i * 4).Take(4).ToArray();
                            var score = LatestPrediction.score.ElementAt(i);

                            <tr>
                                <td>@(i + 1)</td>
                                <td>@label</td>
                                <td>@bbox[0].ToString("0.00")</td>
                                <td>@bbox[1].ToString("0.00")</td>
                                <td>@bbox[2].ToString("0.00")</td>
                                <td>@bbox[3].ToString("0.00")</td>
                                <td>@score.ToString("0.0000")</td>
                            </tr>
                        }
                    </tbody>
                </table>
            }
            else
            {
                <p class="text-muted">No predictions available.</p>
            }
        </div>
    </div>

    <div class="row align-items-start">
        <div class="col overflow-scroll">
            <label class="alert-info">Preview of the selected image</label>
            <div>
                <img id="PreviewImage" style="border:1px solid black;" src="@UploadedImagePreview" /><br />
            </div>
        </div>
        <div class="col overflow-scroll">
            <label class="alert-info">Image with bounding boxes</label>
            <canvas height="400" id="PreviewImageBbox" style="border:solid 1px black">
            </canvas>
            <br />
        </div>
    </div>

</div>



The image you want to analyze is uploaded using an InputFile component in Blazor. This uses HTML file upload control. After file is uploaded, the API will be called in the event handler OnInputFile. The code behind of the home component is shown next.

Home.razor.cs



using Microsoft.AspNetCore.Components;
using Microsoft.AspNetCore.Components.Forms;
using Microsoft.JSInterop;
using ObjectDetectionMachineLearning.Web.Models;
using System.Text;
using System.Text.Json;

namespace ObjectDetectionMachineLearning.Web.Components.Pages
{
    partial class Home
    {

        [Inject]
        private HttpClient Http { get; set; } = default!;

        [Inject]
        private IJSRuntime JsRunTime { get; set; } = default!;

        private string? UploadedImagePreview;

        private MLPrediction LatestPrediction = default!;

        /// <summary>
        /// Uploads an image and sets the imagePreview property to display it
        /// </summary>
        /// <param name="e"></param>
        /// <returns></returns>

        private async Task OnInputFile(InputFileChangeEventArgs e)
        {
            var file = e.File;

            if (file != null && (file.ContentType == "image/jpeg" || file.ContentType == "image/png"))
            {
                string savedUploadedImageFullPath = await SaveUploadedImage(file);

                // Optional: Set preview if you still want to show it in the UI
                using var ms = new MemoryStream();
                using var previewStream = file.OpenReadStream(maxAllowedSize: 10 * 1024 * 1024);
                await previewStream.CopyToAsync(ms);
                var bytes = ms.ToArray();
                UploadedImagePreview = $"data:{file.ContentType};base64,{Convert.ToBase64String(bytes)}";

                string? prediction = await CallPredictApiAsync(savedUploadedImageFullPath);

                var jsonBboxes = CreateBoundingBoxJson(prediction);
                await JsRunTime.InvokeVoidAsync("InitLoadBoundingBoxes", jsonBboxes);

                Console.WriteLine($"Prediction {prediction}");

                //StateHasChanged();
            }
        }

        private string CreateBoundingBoxJson(string? prediction)
        {
            if (string.IsNullOrEmpty(prediction))
                return "[]";
            try
            {
                var mlPrediction = JsonSerializer.Deserialize<MLPrediction>(prediction);
                LatestPrediction = mlPrediction;
                return ConvertMLPredictionToBoundingBoxJson(mlPrediction!);
            }
            catch (JsonException ex)
            {
                Console.WriteLine($"Error deserializing prediction: {ex.Message}");
                return "[]";
            }

        }

        public static string ConvertMLPredictionToBoundingBoxJson(MLPrediction prediction)
        {
            var boxes = prediction.predictedBoundingBoxes;
            var labels = prediction.predictedLabel ?? new List<string>();
            var scores = prediction.score ?? new List<float>();

            if (boxes == null || boxes.Count % 4 != 0)
                return "[]";

            var results = new List<object>();

            for (int i = 0; i < boxes.Count; i += 4)
            {
                float x1 = boxes[i];
                float y1 = boxes[i + 1];
                float x2 = boxes[i + 2];
                float y2 = boxes[i + 3];

                float width = x2 - x1;
                float height = y2 - y1;

                results.Add(new
                {
                    Name = i / 4 < labels.Count ? labels[i / 4] : "Unknown",
                    X = x1,
                    Y = y1,
                    Width = width,
                    Height = height,
                    Confidence = i / 4 < scores.Count ? scores[i / 4].ToString("0.0000") : "0.0000"
                });
            }

            return JsonSerializer.Serialize(results, new JsonSerializerOptions { WriteIndented = true });
        }

        private static async Task<string> SaveUploadedImage(IBrowserFile file)
        {
            var uploadsFolder = Path.Combine(Environment.CurrentDirectory, "UploadedImages");
            Directory.CreateDirectory(uploadsFolder); // Ensure folder exists

            var fileName = $"{Guid.NewGuid()}_{file.Name}";
            var filePath = Path.Combine(uploadsFolder, fileName);

            using (var stream = file.OpenReadStream(maxAllowedSize: 10 * 1024 * 1024))
            using (var fileStream = new FileStream(filePath, FileMode.Create))
            {
                await stream.CopyToAsync(fileStream);
            }

            return filePath;
        }

        private async Task<string?> CallPredictApiAsync(string imagePath)
        {
            try
            {
                var payload = new { imagePath = imagePath };
                var content = new StringContent(JsonSerializer.Serialize(payload), Encoding.UTF8, "application/json");

                var response = await Http.PostAsync("https://localhost:65194/predict", content);

                if (response.IsSuccessStatusCode)
                {
                    var result = await response.Content.ReadAsStringAsync();
                    Console.WriteLine("Prediction result: " + result);
                    return result;
                }
                else
                {
                    Console.WriteLine($"API call failed: {response.StatusCode}");
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("Error calling API: " + ex.Message);
            }

            return null;
        }

    }
}



home.js

Over to the client side javascript next, which handles the loading of bounding boxes using Canvas in HTML5.


<script type="text/javascript">
var colorPalette = [
    "red", "yellow", "blue", "green", "fuchsia",
    "moccasin", "purple", "magenta", "aliceblue",
    "lightyellow", "lightgreen"
];

function rescaleCanvas() {
    var img = document.getElementById('PreviewImage');
    var canvas = document.getElementById('PreviewImageBbox');
    var displayWidth = img.clientWidth;
    var displayHeight = img.clientHeight;
    canvas.width = displayWidth;
    canvas.height = displayHeight;
}

function LoadBoundingBoxes(objectDescriptions) {
    if (!objectDescriptions) {
        alert('No objects found in image.');
        return;
    }

    console.log(new Date() + ' ' + 'home.js : Loading bounding boxes from returned results ..');

    var objectDesc = typeof objectDescriptions === "string"
        ? JSON.parse(objectDescriptions)
        : objectDescriptions;

    var canvas = document.getElementById('PreviewImageBbox');
    var img = document.getElementById('PreviewImage');
    var ctx = canvas.getContext('2d');

    var scaleX = canvas.width / img.naturalWidth;
    var scaleY = canvas.height / img.naturalHeight;

    ctx.clearRect(0, 0, canvas.width, canvas.height);
    ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
    ctx.font = "10px Verdana";

    console.log(`ctx.drawImage Canvas width: ${canvas.width} Canvas height: ${canvas.height} ScaleX ${scaleX} ScaleY ${scaleY}`);

    for (var i = 0; i < objectDesc.length; i++) {
        const obj = objectDesc[i];
        const x = obj.X * scaleX;
        const y = obj.Y * scaleY;
        const width = obj.Width * scaleX;
        const height = obj.Height * scaleY;

        ctx.beginPath();
        ctx.strokeStyle = "black";
        ctx.lineWidth = 1;
        ctx.fillText(obj.Name, x + width / 2, y + height / 2);
        ctx.fillText("Confidence: " + obj.Confidence, x + width / 2, 10 + y + height / 2);
    }

    for (var i = 0; i < objectDesc.length; i++) {
        const obj = objectDesc[i];
        const x = obj.X * scaleX;
        const y = obj.Y * scaleY;
        const width = obj.Width * scaleX;
        const height = obj.Height * scaleY;

        ctx.fillStyle = getColor();
        ctx.globalAlpha = 0.2;
        ctx.fillRect(x, y, width, height);

        ctx.globalAlpha = 1.0;
        ctx.lineWidth = 3;
        ctx.strokeStyle = "blue";
        ctx.strokeRect(x, y, width, height);

        ctx.fillStyle = "black";
        ctx.fillText("Color: " + getColor(), x + width / 2, 20 + y + height / 2);
    }

    console.log('Bounding boxes:', objectDesc);
}

function getColor() {
    var colorIndex = Math.floor(Math.random() * colorPalette.length);
    return colorPalette[colorIndex];
}

function InitLoadBoundingBoxes(objectDescriptions) {
    const img = document.getElementById('PreviewImage');

    const draw = () => {
        setTimeout(() => {
            LoadBoundingBoxes(objectDescriptions);
        }, 1000);
    };

    if (!img.complete) {
        img.onload = draw;
    } else {
        draw();
    }
}
</script>



ML.net project for Object detection

The repo also contains a project with the Object detection with the training data. You will need the ML.net workload here to work ML.Net in VS 2022.


Model builder in VS 2022

Training with pictures - training data

The ML.net project allows you to via the built-in ModelBuilder UI select the pictures that you can train the machine learning model.



Concluding notes

We have seen how we fairly conveniently can train a machine learning model to analyze images and detect objects. In this demo, stop signs along roads and streets in United States is used for the training. Note that you with using Vott, the Visual Object Tagging Tool, you can enter multiple labels. i.e. you can train the machine learning model to detect and interpret multiple types of objects. For example speed signs and stop signs to take an example. A self-driving car would use machine learning to intelligently interpret in real time such road signs and use additional input such as GPS, road databases and LIDAR to ultimately achieve a situation awareness that is needed to drive a car. Of course, we have just trained a machine learning model to just detect stop signs from 50 training images, but now you should have a better understanding how you can train machine learning models to detect objects in images.