Showing posts with label MAUI. Show all posts
Showing posts with label MAUI. Show all posts

Saturday, 1 August 2026

Kartverket Matrikkel Eiendoms API with MAUI

This article will show how looking up data from Eiendoms API (v1) from Kartverket. The data is retrieved from Matrikkelen, the official registry of properties and land parcels in Norway. The demo will show using a MAUI app and the source code with the demo is available in my Github repository to be cloned from here :


Screenshots

Clicking on a property in the map will show the matrikkelen information for the first hit of the properties around the clicked point.

Using Kartverket's Eiendom API v1

The Eiendom API offers basic Eiendomsinformasjon, property information. A Swagger page and the endpoint of the API is available here:

The service, that is a client to contact and retrieved the Eiendomsinformation for a given point that uses the /punkt API method is shown below. Please note that the json naming dictates we must set the Property naming policy to camelCase, as we can see from the JsonProperty naming convention used further down in the Json POCO classes shown in KartverketResponse.cs code snippet.

KartverketService.cs


using System.Globalization;
using System.Net.Http.Json;
using System.Text.Json;

namespace MauiMapAppDemo.Services
{

    public class KartverketService
    {

        private const string _apiKartverketEiendomV1BaseUrl = "https://api.kartverket.no/eiendom/v1/";

        private readonly HttpClient _httpClient = new HttpClient
        {
            BaseAddress = new Uri(_apiKartverketEiendomV1BaseUrl)
        };

        /// <summary>
        /// Retrieves matrikkel informasjon from given point (Punkt) from Kartverket's Eiendom API v1
        /// </summary>
        /// <param name="latitude">Latitude</param>
        /// <param name="longitude">Longitude</param>
        /// <param name="koordSys">Defaulting here to EUREF89 = 4258 as the coordinate system id, which Kartverket uses and is also what Google maps coords are using</param>
        /// <returns></returns>
        public async Task<KartverketPunktResponse?> GetMatrikkelInformationFromLocationAsync(double latitude, double longitude, int koordSys = 4258)
        {
            string url = $"punkt?ost={longitude.ToString(CultureInfo.InvariantCulture)}&nord={latitude.ToString(CultureInfo.InvariantCulture)}&koordsys={koordSys}&radius=10&utkoordsys={koordSys}&treffPerSide=1&side=1";

            var kartverketResponseForLocation = await _httpClient.GetFromJsonAsync<KartverketPunktResponse>(url, options: new System.Text.Json.JsonSerializerOptions
            {
                PropertyNamingPolicy = JsonNamingPolicy.CamelCase
            });
            return kartverketResponseForLocation;
        }

    }
}

About the choice of EUREF 89 = 4258 as the default coordinate system in the call above. Google maps uses actually WGS 84 which is EPSG coordinate system id 4326, but the difference is less than a meter. Since a 10 meter radius of the clicked point is used, we pass in the clicked Location point's latitude and longitude to the north and east parameters in the url call above in the KartverketService.cs sample code shown, with a given radius of 10 and although we get a result of multiple hits within the 10 meter radius, the first if any of properties or parcels found is returned from the service, by setting the side and treffPerSide parameters.

🌍 What are EUREF89 and EPSG?

🗺️ EUREF89 is a geographic coordinate reference system used across Europe, including by Kartverket in Norway. It defines how latitude and longitude positions are represented on the Earth's surface.

🔢 EPSG:4258 is the unique identifier for the EUREF89 coordinate system. EPSG codes are standardized reference numbers that mapping software, APIs, and GIS tools use to identify coordinate systems.

📍 In this project, coordinates are obtained from Google Maps and sent to Kartverket's Eiendom API v1 using EUREF89 (EPSG:4258). Using the same coordinate reference system helps ensure accurate property lookups in the Norwegian Matrikkel.

🏠 The result is that a location selected on the map can be translated directly into property and cadastral information from Kartverket.

The following Json classes will map the returned Json data from the service to objects. To create these classes, I just copied example output response from the API method /punkt and used Paste Special => Paste Json as classes in VS 2026.

KartverketPunktResponse.cs


using System.Text;
using System.Text.Json.Serialization;

namespace MauiMapAppDemo.Services
{
    public class KartverketPunktResponse
    {
        [JsonPropertyName("eiendom")]
        public List<Punkt>? Eiendom { get; set; }

        [JsonPropertyName("metadata")]
        public Metadata? Metadata { get; set; }
    }

    public class Metadata
    {
        [JsonPropertyName("side")]
        public int Side { get; set; }

        [JsonPropertyName("sokeStreng")]
        public string? SokeStreng { get; set; }

        [JsonPropertyName("totaltAntallTreff")]
        public int TotaltAntallTreff { get; set; }

        [JsonPropertyName("treffPerSide")]
        public int TreffPerSide { get; set; }

        [JsonPropertyName("viserFra")]
        public int ViserFra { get; set; }

        [JsonPropertyName("viserTil")]
        public int ViserTil { get; set; }
    }

    public class Punkt
    {
        [JsonPropertyName("bruksnummer")]
        public int Bruksnummer { get; set; }

        [JsonPropertyName("festenummer")]
        public int Festenummer { get; set; }

        [JsonPropertyName("gardsnummer")]
        public int Gardsnummer { get; set; }

        [JsonPropertyName("hovedområde")]
        public bool HovedOmrade { get; set; }

        [JsonPropertyName("kommunenummer")]
        public string? Kommunenummer { get; set; }

        [JsonPropertyName("lokalid")]
        public int Lokalid { get; set; }

        [JsonPropertyName("matrikkelnummertekst")]
        public string? Matrikkelnummertekst { get; set; }

        [JsonPropertyName("meterFraPunkt")]
        public int MeterFraPunkt { get; set; }

        [JsonPropertyName("nøyaktighetsklasseteig")]
        public string? Noyaktighetsklasseteig { get; set; }

        [JsonPropertyName("objekttype")]
        public string? Objekttype { get; set; }

        [JsonPropertyName("oppdateringsdato")]
        public DateTime? Oppdateringsdato { get; set; }

        [JsonPropertyName("representasjonspunkt")]
        public Representasjonspunkt? Representasjonspunkt { get; set; }

        [JsonPropertyName("seksjonsnummer")]
        public int Seksjonsnummer { get; set; }

        [JsonPropertyName("teigmedflerematrikkelenheter")]
        public bool Teigmedflerematrikkelenheter { get; set; }

        [JsonPropertyName("uregistrertjordsameie")]
        public bool Uregistrertjordsameie { get; set; }

        public string FullstendigMatrikkelNummer
        {
            get
            {
                var sb = new StringBuilder();
                if (!string.IsNullOrWhiteSpace(Kommunenummer))
                {
                    sb.Append(Kommunenummer?.ToString().PadLeft(4, '0'));
                }
                if (Gardsnummer > 0)
                {
                    sb.Append("/" + Gardsnummer);
                }
                if (Bruksnummer > 0)
                {
                    sb.Append("/" + Bruksnummer);
                }
                if (Festenummer > 0)
                {
                    sb.Append("/" + Festenummer);
                }
                if (Seksjonsnummer > 0)
                {
                    sb.Append("/" + Seksjonsnummer);
                }

                return sb.ToString();
            }
        }
    }

    public class Representasjonspunkt
    {
        [JsonPropertyName("koordsys")]
        public int Koordsys { get; set; }

        [JsonPropertyName("nord")]
        public double Nord { get; set; }

        [JsonPropertyName("øst")]
        public double Ost { get; set; }
    }

}



Groundwork on the parts of property information for Fullstendig matrikkelnummer

🏠 Components of a Full Matrikkel Number

Icon Component Abbreviation Example Description
🏛️ Municipality Number Knr. 5001 Identifies the municipality where the property is located. An overview of Kommunenummer can be seen here : https://www.kartverket.no/til-lands/fakta-om-norge/norske-fylke-og-kommunar
🌾 Farm Number - Gårdnummer / Gardsnummer Gnr. 138 Identifies the original farm or cadastral area within the municipality.
🏠 Usage Number / Bruksnummer Bnr. 4515 Identifies a specific property or parcel within a farm number.
📜 Leasehold Number / Festenummer Fnr. 7 Used when the property is a leased parcel of land (festetomt). The festetomt / lease agreements typically lasts very long, in the area of 80-100 years typically and can be renewed.
🏢 Section Number Snr. 12 Identifies a specific section or apartment within a sectioned property.

📋 Example

5001 / 138 / 4515 / 7 / 12

Note that the first three numbers are ALWAYS set for all properties in Norway. That is Kommunenummer, Gårdsnummer (Gardsnummer) and Bruksnummer. And some properties also are leased, so they got a lease number or sectioned too. The numbers are read left to right in descending order of the overview shown above.
Value Meaning
🏛️ 5001 Municipality Number
🌾 138 Farm Number (Gårdsnummer)
🏠 4515 Usage Number (Bruksnummer)
📜 7 Leasehold Number (Festenummer)
🏢 12 Section Number (Seksjonsnummer)

💡 A full matrikkel number is built in the following order:

🏛️ Municipality Number → 🌾 Farm Number → 🏠 Usage Number → (📜 Leasehold Number → 🏢 Section Number )



📐 Getting Property Boundary Area Information (Grenseinformasjon)

You can retrieve boundary information for a selected property using the /punkt/omrader endpoint in Eiendoms API (v1).

The API returns a GeoJSON geometry describing the property's boundaries. The coordinates are provided as latitude and longitude values, so they must be projected into a metric coordinate system before calculating the property's area.

🗺️ In a GeoJSON polygon, the coordinates are returned as a multi-dimensional array:

  • ✅ The first array represents the property's outer boundary.
  • ➖ Any additional arrays represent holes or excluded areas within the property.

📏 To calculate the total area:

  1. Calculate the area of the outer boundary polygon.
  2. Subtract the area of each interior polygon (hole).

💡 This follows the standard GeoJSON polygon specification and ensures that any gaps within the property are excluded from the final area calculation.

The following screenshots shows selecting a property for testing out the precision of the area calculations using of a polygon of property's boundary using Eiendoms API v1.
As the screenshots show, the demo shows an agreement with Trondheimskart solution, which is the property map that Trondheim kommune is using : The calculated area in Trondheimskartet is 1125 square metres and the app calculate it to 1126 square metres.

🗺️ Area and polygon overview

This note explains how the app turns Kartverket omrade data into a visible polygon and a simple area summary. The goal is to keep the UI readable while still showing the geometry work behind the scenes.

The flow is: tap a point, fetch Kartverket data, build a closed polygon from the returned coordinates, and show the area in square metres. The main page binds the polygon path and the area label separately so the overlay and the text can update independently.

📐 Area calculation

The shared geometry helper uses the shoelace formula. If the incoming ring is still in WGS84 degrees, it first transforms the coordinates to UTM Zone 33N so the result comes out in square metres. The shoelace formulae is also known as Gauss area formula and is visually described with an example here from University of Waterloo in Canada great article:

Note that the code below uses Nuget package ProjNet
using ProjNet.CoordinateSystems;
using ProjNet.CoordinateSystems.Transformations;

namespace MauiMapAppDemo.Services
{
    public static class GeometryUtils
    {
        public static double CalculatePolygonArea(double[][] ring, bool transformToMetricCoordinateSystem = true)
        {
            if (ring == null || ring.Length < 3)
            {
                throw new ArgumentException("Polygon must contain at least three points.", nameof(ring));
            }

            var projectedRing = new double[ring.Length][];

            if (transformToMetricCoordinateSystem)
            {
                var ctFactory = new CoordinateTransformationFactory();
                var wgs84 = GeographicCoordinateSystem.WGS84;
                var utm33 = ProjectedCoordinateSystem.WGS84_UTM(33, true);
                var transform = ctFactory.CreateFromCoordinateSystems(wgs84, utm33);

                for (int i = 0; i < ring.Length; i++)
                {
                    var projected = transform.MathTransform.Transform(ring[i][0], ring[i][1]);
                    projectedRing[i] = new[] { projected.x, projected.y };
                }
            }

            double area = 0;

            for (int i = 0; i < projectedRing.Length; i++)
            {
                int next = (i + 1) % projectedRing.Length;
                area += projectedRing[i][0] * projectedRing[next][1];
                area -= projectedRing[next][0] * projectedRing[i][1];
            }

            return Math.Abs(area) / 2.0;
        }

        public static double CalculateTotalArea(double[][][] coordinates, bool transformToMetricCoordinateSystem = true)
        {
            if (coordinates == null || coordinates.Length == 0)
            {
                return 0;
            }

            double totalArea = 0;

            for (int ringIndex = 0; ringIndex < coordinates.Length; ringIndex++)
            {
                var ringArea = CalculatePolygonArea(coordinates[ringIndex], transformToMetricCoordinateSystem);

                if (ringIndex == 0)
                {
                    totalArea += ringArea;
                }
                else
                {
                    totalArea -= ringArea;
                }
            }

            return totalArea;
        }
    }
}

For GeoJSON, the first ring is the outer boundary and later rings are holes. That matches how Kartverket polygons are usually interpreted when the app calculates total area from an omrade response.

🛰️ Kartverket omrade fetch

The service builds the omrader URL from the clicked latitude and longitude, then deserializes the response with a shared camel-case serializer option. The resulting geometry is what the app later converts into a polygon overlay.

public async Task<KartverketOmraadeResponse?> GetGeoJsonFromLocationAsync(double latitude, double longitude, int koordSys = 4258)
{
    string url = $"punkt/omrader?ost={longitude.ToString(CultureInfo.InvariantCulture)}&nord={latitude.ToString(CultureInfo.InvariantCulture)}&koordsys={koordSys}&radius=10&utkoordsys={koordSys}&treffPerSide=1&side=1";

    var kartverketResponseForLocation = await _httpClient.GetFromJsonAsync<KartverketOmraadeResponse>(url, options: s_camelCaseJsonOptions);
    return kartverketResponseForLocation;
}

🧾 Kartverket omrade response model

The response model is where the geometry turns into a calculated area value. The top-level object exposes features, and the area is derived from the nested polygon coordinates through a helper property.

public class KartverketOmraadeResponse
{
    [JsonPropertyName("features")]
    public Feature[] Features { get; set; }

    [JsonPropertyName("type")]
    public string Type { get; set; }

    public double? TotalAreaOfAllAreas
    {
        get
        {
            try
            {
                double totalArea = Features.Sum(f => GeometryUtils.CalculateTotalArea(f.Geometry.Coordinates));
                return totalArea;
            }
            catch (Exception)
            {
                return null;
            }
        }
    }
}

public class Geometry
{
    [JsonPropertyName("coordinates")]
    public double[][][] Coordinates { get; set; }

    [JsonPropertyName("type")]
    public string Type { get; set; }

    public double? TotalArea
    {
        get
        {
            try
            {
                double totalArea = GeometryUtils.CalculateTotalArea(Coordinates);
                return totalArea;
            }
            catch (Exception)
            {
                return null;
            }
        }
    }
}

public class Feature
{
    [JsonPropertyName("geometry")]
    public Geometry Geometry { get; set; }

    [JsonPropertyName("properties")]
    public Properties Properties { get; set; }

    [JsonPropertyName("type")]
    public string Type { get; set; }
}

The nice part here is that the area calculation is pushed into the model layer through read-only properties. That makes the popup logic simpler because it can ask for the total area instead of re-running geometry math in the UI layer. 📦

🧩 Polygon refresh behavior

The map behavior listens to the bound polygon path and rebuilds the overlay whenever the path changes. It clears the old polygon first, then adds the latest path back to the map.

private void RefreshMatrikkelPolygon()
{
    if (_map?.MapElements == null)
    {
        return;
    }

    ClearMatrikkelPolygon();

    if (!IsMatrikkelMode || !MatrikkelPolygonPath.Any())
    {
        return;
    }

    var polygon = new Polygon
    {
        StrokeColor = Colors.Red,
        StrokeWidth = 5,
        FillColor = Color.FromArgb("#22FF0000")
    };

    foreach (var location in MatrikkelPolygonPath)
    {
        polygon.Geopath.Add(location);
    }

    if (polygon.Geopath.Count < 3)
    {
        return;
    }

    var firstLocation = polygon.Geopath.First();
    var lastLocation = polygon.Geopath.Last();

    if (firstLocation.Latitude != lastLocation.Latitude || firstLocation.Longitude != lastLocation.Longitude)
    {
        polygon.Geopath.Add(firstLocation);
    }

    _matrikkelPolygon = polygon;
    _map.MapElements.Add(polygon);
}

That final closure step matters. If the last point does not match the first point, the behavior appends the first point again so the polygon is visually closed and the shape is easier to reason about. 🔁

📊 Area presentation

The viewmodel keeps the user-facing area text separate from the map geometry. When Kartverket returns a usable polygon, the app formats the result as square metres and also shows the same value in mål.

if (omraadeResponse?.TotalAreaOfAllAreas is double totalArea && totalArea > 0)
{
    MatrikkelAreaText = FormatMatrikkelAreaText(totalArea);
}
else
{
    MatrikkelAreaText = "Areal: ukjent";
}

private static string FormatMatrikkelAreaText(double areaSquareMetres)
{
    var maal = areaSquareMetres / 1000d;
    return $"Areal: {areaSquareMetres:N0} m² ({maal:N2} mål)";
}

The bound properties behind that flow are the polygon path and the area label. One drives the overlay, the other is what the user sees on the page.

public IEnumerable<Location> MatrikkelPolygonPath
{
    get => (IEnumerable<Location>)GetValue(MatrikkelPolygonPathProperty);
    set => SetValue(MatrikkelPolygonPathProperty, value);
}

public static readonly BindableProperty MatrikkelPolygonPathProperty =
    BindableProperty.Create(
        nameof(MatrikkelPolygonPath),
        typeof(Location[]),
        typeof(MapPinsBehavior),
        defaultValue: Array.Empty<Location>(),
        propertyChanged: OnMatrikkelPolygonPathChanged);

public static readonly BindableProperty IsMatrikkelModeProperty =
    BindableProperty.Create(
        nameof(IsMatrikkelMode),
        typeof(bool),
        typeof(MapPinsBehavior),
        false);

📱 Map page wiring

The map page binds the behavior directly. In this project the page is MapsDemo.xaml, and the snippet below shows the same behavior wiring that drives the map, measurement mode, and matrikkel polygon overlay.

<maps:Map Grid.Row="3"
          Margin="12"
          x:Name="MapCtrl"
          MapType="Street">
    <maps:Map.Behaviors>
        <behaviors:MapPinsBehavior
            BindingContext="{Binding Source={x:Reference MapCtrl}, Path=BindingContext}"
            Center="{Binding MapCenter}"
            PinItems="{Binding CabinPins}"
            IsMeasuringMode="{Binding IsMeasuringMode}"
            IsMatrikkelMode="{Binding IsMatrikkelMode}"
            MeasureStart="{Binding FirstLocationMeasureMode}"
            MatrikkelPolygonPath="{Binding MatrikkelPolygonPath}"
            MeasureEnd="{Binding SecondLocationMeasureMode}"
            MapClickedCommand="{Binding MapClickedCommand}"
            PinClickedCommand="{Binding PinClickedCommand}" />
    </maps:Map.Behaviors>
</maps:Map>

✨ Short takeaway

The app gets geometry from Kartverket, calculates area in a metric-friendly way, closes the polygon before drawing it, and keeps the user-facing square-metre text separate from the map overlay. That makes the map interaction clearer and easier to explain in a blog post.

Sunday, 12 July 2026

Android and Google Maps using .NET Maui

MAUI Maps Demo Notes 🗺️

A short summary of the map demo, measurement mode, and pin icon adjustments.
The source code can be cloned from this Github repo of mine : 👉 MauiMapAppDemo Github repo 💾

This demo shows a .NET MAUI map page with two main ideas: regular cabin pins and a measure mode that lets the user tap two points to draw a red line and calculate distance. The page is driven by a view model, while a map behavior keeps the map logic reusable and keeps the XAML clean. In use is also CommunityToolkit.Mvvm to make MVVM easier to implement. The use of Behaviors to implement view specific behavior / functionality is a common pattern when working with client based logic and MVVM and UI controls.

🔒 The app reads Google Maps and Azure Maps keys from user secrets during local development. The keys are not exposed in the solution or this article.

What the solution does

  • Loads cabin pin data from the view model and inits the location around Trondheim in Norway.
  • Supports a measure mode that tracks start and end taps (by clicking a button to activate this)
  • Draws a polyline (line segment) between the two points when both locations are set.
  • Uses custom measurement marker icons so the start and end pins are easier to spot.
  • Keeps the layout polished with spacing around the logo, button, and map.

Behavior

The behavior listens for map clicks, manages the temporary measure pins, and clears the graphics when measure mode is turned off. It also wires the pin click command for normal cabin markers.

using MauiMapAppDemo.ViewModels;
using Microsoft.Maui.Controls.Maps;
using Microsoft.Maui.Maps;
using System.Windows.Input;

namespace MauiMapAppDemo.Behaviors
{
    public class MapPinsBehavior : Behavior<Microsoft.Maui.Controls.Maps.Map>
    {
        public static readonly BindableProperty IsMeasuringModeProperty =
            BindableProperty.Create(
                nameof(IsMeasuringMode),
                typeof(bool),
                typeof(MapPinsBehavior),
                false,
                propertyChanged: OnMeasurementStateChanged);

        private void RefreshMeasurementLine()
        {
            if (_map == null)
            {
                return;
            }

            ClearMeasurementGraphics();

            if (MeasureStart == null || MeasureEnd == null)
            {
                return;
            }

            _startPin = new MeasurementPin
            {
                Label = "Start",
                Address = "Measurement Start",
                Location = MeasureStart,
                IconResourceName = "startmarkerv2"
            };

            _endPin = new MeasurementPin
            {
                Label = "End",
                Address = "Measurement End",
                Location = MeasureEnd,
                IconResourceName = "endmarkerv2"
            };
        }
    }
}

View model

The view model owns the state: measurement mode, the first and second tapped locations, the computed distance, and the cabin pins. That keeps the behavior focused on map rendering and interaction.

using CommunityToolkit.Mvvm.ComponentModel;
using CommunityToolkit.Mvvm.Input;
using System.Collections.ObjectModel;

namespace MauiMapAppDemo.ViewModels
{
    public partial class MapsViewModel : ObservableObject
    {
        [ObservableProperty]
        public bool _isMeasuringMode;

        [ObservableProperty]
        private Location? _firstLocationMeasureMode;

        [ObservableProperty]
        private Location? _secondLocationMeasureMode;

        [ObservableProperty]
        private double _distanceMeasuredKm;

        public ObservableCollection<MapPinModel> CabinPins { get; } = [];

        [RelayCommand]
        private async Task MapClicked(Location location)
        {
            if (IsMeasuringMode)
            {
                HandleMeasuringMode(location);
                return;
            }

            await HandleDefaultMapClicked(location);
        }
    }
}

Marker icons 📍

The biggest visual improvement was the measurement markers. The solution now uses small transparent SVG assets for the start and end points, which avoids oversized default-looking icons and keeps the map readable.

The important part was keeping those marker assets compact and making sure the Android handler only applies them to the measurement pins.

Layout cleanup

The page also got a small spacing pass: the button gained more breathing room and a larger corner radius, the logo and map got margins, and the Eudem-25 info label was moved higher so the top of the page reads more naturally.

MAUI Maps Demo - Code Details

This is a compact code detail listing for the two core pieces behind the demo: the map behavior and the view model. It is meant as a quick reference for how the map, measurement flow, and pin handling fit together.

MapPinsBehavior

The behavior owns map interaction, pin rendering, measurement graphics, and the custom marker icon hookup used for the start and end measurement pins.

using MauiMapAppDemo.ViewModels;
using Microsoft.Maui.Controls.Maps;
using Microsoft.Maui.Maps;
using System.Windows.Input;

namespace MauiMapAppDemo.Behaviors
{

    public class MapPinsBehavior : Behavior<Microsoft.Maui.Controls.Maps.Map>
    {

        private Microsoft.Maui.Controls.Maps.Map? _map;

        private Microsoft.Maui.Controls.Maps.Polyline? _measurementLine;

        private Microsoft.Maui.Controls.Maps.Pin? _startPin;
        private Microsoft.Maui.Controls.Maps.Pin? _endPin;


        public static readonly BindableProperty IsMeasuringModeProperty =
            BindableProperty.Create(
                nameof(IsMeasuringMode),
                typeof(bool),
                typeof(MapPinsBehavior),
                false,
                propertyChanged: OnMeasurementStateChanged);

        public bool IsMeasuringMode
        {
            get => (bool)GetValue(IsMeasuringModeProperty);
            set => SetValue(IsMeasuringModeProperty, value);
        }


        public static readonly BindableProperty MeasureStartProperty =
            BindableProperty.Create(
                nameof(MeasureStart),
                typeof(Location),
                typeof(MapPinsBehavior),
                propertyChanged: OnMeasurementChanged
            );

        public Location MeasureStart
        {
            get => (Location)GetValue(MeasureStartProperty);
            set => SetValue(MeasureStartProperty, value);
        }

        public static readonly BindableProperty MeasureEndProperty =
            BindableProperty.Create(
                nameof(MeasureEnd),
                typeof(Location),
                typeof(MapPinsBehavior),
                propertyChanged: OnMeasurementChanged
            );

        public Location MeasureEnd
        {
            get => (Location)GetValue(MeasureEndProperty);
            set => SetValue(MeasureEndProperty, value);
        }


        public static readonly BindableProperty CenterProperty =
         BindableProperty.Create(
             nameof(Center),
             typeof(Location),
             typeof(MapPinsBehavior),
             propertyChanged: OnCenterChanged
             );

        public Location Center
        {
            get => (Location)GetValue(CenterProperty);
            set => SetValue(CenterProperty, value);
        }

        public static readonly BindableProperty PinItemsProperty =
            BindableProperty.Create(
                nameof(PinItems),
                typeof(IEnumerable<MapPinModel>),
                typeof(MapPinsBehavior),
                propertyChanged: OnPinItemsChanged
                );      

        public IEnumerable<MapPinModel>? PinItems
        {
            get => (IEnumerable<MapPinModel>?)GetValue(PinItemsProperty);
            set => SetValue(PinItemsProperty, value);
        }

        public static readonly BindableProperty MapClickedCommandProperty =
            BindableProperty.Create(
                nameof(MapClickedCommand),
                typeof(ICommand),
                typeof(MapPinsBehavior),
                defaultValue: null);

        public ICommand MapClickedCommand
        {
            get => (ICommand)GetValue(MapClickedCommandProperty);
            set => SetValue(MapClickedCommandProperty, value);
        }

        public static readonly BindableProperty PinClickedCommandProperty =
           BindableProperty.Create(
               nameof(PinClickedCommand),
               typeof(ICommand),
               typeof(MapPinsBehavior),
               defaultValue: null);

        public ICommand PinClickedCommand
        {
            get => (ICommand)GetValue(PinClickedCommandProperty);
            set => SetValue(PinClickedCommandProperty, value);
        }

        protected override void OnAttachedTo(Microsoft.Maui.Controls.Maps.Map bindable)
        {
            _map = bindable;

            WireUpMapClickedCommand(bindable);

            base.OnAttachedTo(bindable);

            RefreshPins();

            if (Center is not null)
            {
                _map.MoveToRegion(MapSpan.FromCenterAndRadius(Center, Distance.FromKilometers(8)));
            }
        }

        private void WireUpMapClickedCommand(Microsoft.Maui.Controls.Maps.Map map)
        {
            map.MapClicked += (object? sender, MapClickedEventArgs e) =>
            {
                if (MapClickedCommand?.CanExecute(e.Location) == true)
                {
                    MapClickedCommand.Execute(e.Location);
                }
            };
        }

        protected override void OnDetachingFrom(Microsoft.Maui.Controls.Maps.Map bindable)
        {
            _map = null;

            base.OnDetachingFrom(bindable);
        }

        private static void OnCenterChanged(
            BindableObject bindable,
            object oldValue,
            object newValue)
        {
            var behavior = (MapPinsBehavior)bindable;

            if (behavior._map is not null && newValue is Location location)
            {
                behavior._map.MoveToRegion(MapSpan.FromCenterAndRadius(location,
                    Distance.FromKilometers(8)));
            }
        }

        private static void OnMeasurementChanged(
            BindableObject bindable,
            object oldValue,
            object newValue)
        {
            ((MapPinsBehavior)bindable).RefreshMeasurementLine();
        }


        private static void OnMeasurementStateChanged(
            BindableObject bindable,
            object oldValue,
            object newValue)
        {
            var behavior = (MapPinsBehavior)bindable;

            if (!(bool)newValue)
            {
                behavior.ClearMeasurementGraphics();
            }
        }


        private void RefreshMeasurementLine()
        {
            if (_map == null)
            {
                return;
            }

            ClearMeasurementGraphics();

            if (MeasureStart == null)
            {
                return;
            }
            else
            {
                _startPin = new MeasurementPin
                {
                    Label = "Start",
                    Address = "Measurement Start",
                    Location = MeasureStart,
                    IconResourceName = "startmarkerv2"
                };

                _map.Pins.Add(_startPin);
            }

            if (MeasureEnd == null)
            {
                return;
            }
            else
            {


                _endPin = new MeasurementPin
                {
                    Label = "End",
                    Address = "Measurement End",
                    Location = MeasureEnd,
                    IconResourceName = "endmarkerv2"
                };

                _map.Pins.Add(_endPin);
            }

            _measurementLine = new Polyline
            {
                StrokeColor = Colors.Red,
                StrokeWidth = 5
            };

            _measurementLine.Geopath.Add(MeasureStart);
            _measurementLine.Geopath.Add(MeasureEnd);

            _map.MapElements.Add(_measurementLine);
        }

        private static void OnPinItemsChanged(
            BindableObject bindable,
            object oldValue,
            object newValue)
        {
            ((MapPinsBehavior)bindable).RefreshPins();
        }

        private void ClearMeasurementGraphics()
        {
            if (_map == null)
            {
                return;
            }

            if (_startPin != null)
            {
                _map.Pins.Remove(_startPin);
            }

            if (_endPin != null)
            {
                _map.Pins.Remove(_endPin);
            }

            if (_measurementLine != null)
            {
                _map.MapElements.Remove(_measurementLine);
            }

            _startPin = null;
            _endPin = null;
            _measurementLine = null;
        }

        private void RefreshPins()
        {
            if (_map is null || PinItems is null)
                return;

            _map.Pins.Clear();

            foreach (var item in PinItems.OfType<MapPinModel>())
            {
                var pin = new Pin
                {
                    Label = item.Label,
                    Address = item.Address,
                    Location = new Location(
                        item.Latitude,
                        item.Longitude)
                };

                pin.MarkerClicked += (_, _) =>
                {
                    if (PinClickedCommand?.CanExecute(item) == true)
                    {
                        PinClickedCommand.Execute(item);
                    }
                };

                _map.Pins.Add(pin);
            }
        }




    }
}

MapsViewModel

The view model owns the measurement state, the list of cabin pins, and the commands invoked by the page and the behavior.

using CommunityToolkit.Mvvm.ComponentModel;
using CommunityToolkit.Mvvm.Input;
using MauiMapAppDemo.Repositories.PinLocations;
using MauiMapAppDemo.Services;
using System.Collections.ObjectModel;

namespace MauiMapAppDemo.ViewModels
{

    public partial class MapsViewModel : ObservableObject
    {
        private readonly OpenTopoService _openTopoService;
        private readonly GeocodingService _geocodingService;
        private readonly DialogService _dialogService;

        private bool _pinClickInProgress = false;

        [ObservableProperty]
        public bool _isMeasuringMode;

        [ObservableProperty]
        private Location? _firstLocationMeasureMode;

        [ObservableProperty]
        private Location? _secondLocationMeasureMode;

        [ObservableProperty]
        private double _distanceMeasuredKm;

        public ObservableCollection<MapPinModel> CabinPins { get; } = [];

        public Location MapCenter { get; } = new(63.4305, 10.3951);

        public MapsViewModel(OpenTopoService openTopoService, GeocodingService geocodingService, DialogService dialogService)
        {
            InitCabinPins();

            _openTopoService = openTopoService;
            _geocodingService = geocodingService;
            _dialogService = dialogService;
        }

        [RelayCommand]
        private async Task PinClicked(MapPinModel pin)
        {
            _pinClickInProgress = true;
            try
            {
                var elevation =
                    await _openTopoService.GetElevationAsync(
                        pin.Latitude,
                        pin.Longitude);

                var placementInfo =
                    await _geocodingService.GetGeocodingPlacemark(
                        pin.Latitude,
                        pin.Longitude);

                await _dialogService.ShowAlertAsync(
                    pin.Label,
                    $"{pin.Address}\n\nElevation: {elevation}m\n\n{placementInfo}",
                    "OK");
            }
            finally
            {
                _pinClickInProgress = false;
            }
        }

        [RelayCommand]
        private async Task ToggleMeasureMode()
        {
            IsMeasuringMode = !IsMeasuringMode; //toggle the measuring mode
        }

        [RelayCommand]
        private async Task MapClicked(Location location)
        {
            if (_pinClickInProgress)
            {
                return;
            }

            if (IsMeasuringMode)
            {
                HandleMeasuringMode(location);
                return;
            }

            await HandleDefaultMapClicked(location);
        }

        private void HandleMeasuringMode(Location location)
        {
            if (FirstLocationMeasureMode == null)
            {
                FirstLocationMeasureMode = location;
                return;
            }

            if (SecondLocationMeasureMode == null)
            {
                SecondLocationMeasureMode = location;

                var distance = Location.CalculateDistance(
                    FirstLocationMeasureMode,
                    SecondLocationMeasureMode,
                    DistanceUnits.Kilometers
                    );

                DistanceMeasuredKm = Math.Round(distance, 1);

                return;
            }

            //Third click restarts over 
            FirstLocationMeasureMode = location;
            SecondLocationMeasureMode = null;
        }      

        private async Task HandleDefaultMapClicked(Location location)
        {
            var elevationOfPoint = await _openTopoService.GetElevationAsync(location.Latitude, location.Longitude);
            await ShowLocationInformationAlert($"Clicked point in the map:", $"Showing elevation of clicked point:", location.Latitude, location.Longitude);
        }

        private async Task ShowLocationInformationAlert(string label, string address, double latitude, double longitude)
        {
            var elevationOfPoint = await _openTopoService.GetElevationAsync(latitude, longitude);

            var placementInfo = await _geocodingService.GetGeocodingPlacemark(latitude, longitude);

            await _dialogService.ShowAlertAsync(
                    label,
                    address + $"\n\nElevation: {elevationOfPoint} m\n\nGeocoding (Placement) info:\n {placementInfo ?? ""}",
                    "OK"
                ); //on click , alert the pin data also via this marker clicked callback 
        }

        private void InitCabinPins()
        {
            foreach (var cabin in TrondheimCabins.GetSampleData())
            {
                CabinPins.Add(
                    new MapPinModel
                    {
                        Label = cabin.Name,
                        Address = cabin.Description,
                        Latitude = cabin.Latitude,
                        Longitude = cabin.Longitude
                    });
            }
        }

    }
}

MauiProgram Adjustments for custom markers on Android platform with Maui

This section shows the startup wiring that makes the demo work on Android: it registers MAUI Maps, reads the maps keys from user secrets, and appends the custom marker icon mapping used by the measurement pins.

using CommunityToolkit.Maui;
using MauiMapAppDemo.Behaviors;
using MauiMapAppDemo.Services;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using Microsoft.Maui.Maps.Handlers;

namespace MauiMapAppDemo
{
    public static class MauiProgram
    {
        public static MauiApp CreateMauiApp()
        {
            var builder = MauiApp.CreateBuilder();
            builder
                .UseMauiApp<App>()
                .UseMauiMaps()
                .UseMauiCommunityToolkit()
                .ConfigureFonts(fonts =>
                {
                    fonts.AddFont("OpenSans-Regular.ttf", "OpenSansRegular");
                    fonts.AddFont("OpenSans-Semibold.ttf", "OpenSansSemibold");
                });

#if ANDROID
            MapPinHandler.Mapper.AppendToMapping("MeasurementPinIcons", (handler, mapPin) =>
            {
                if (mapPin is not MeasurementPin measurementPin)
                {
                    return;
                }

                var resourceId = GetDrawableResourceId(measurementPin.IconResourceName);
                if (resourceId == 0)
                {
                    return;
                }

                handler.PlatformView.SetIcon(Android.Gms.Maps.Model.BitmapDescriptorFactory.FromResource(resourceId));
            });
#endif

#if DEBUG
            builder.Configuration.AddUserSecrets<App>();
#endif

            string azureMapsKey = builder.Configuration["AzureMapsKey"] ?? string.Empty;
            builder.ConfigureEssentials(essentials => essentials.UseMapServiceToken(azureMapsKey));

#if DEBUG
            builder.Logging.AddDebug();
#endif

            builder.Services.AddSingleton<OpenTopoService>();
            builder.Services.AddSingleton<GeocodingService>();
            builder.Services.AddSingleton<DialogService>();

            return builder.Build();
        }

#if ANDROID
        private static int GetDrawableResourceId(string resourceName)
        {
            if (string.IsNullOrWhiteSpace(resourceName))
            {
                return 0;
            }

            var field = typeof(Resource.Drawable).GetField(
                resourceName,
                System.Reflection.BindingFlags.Public |
                System.Reflection.BindingFlags.Static |
                System.Reflection.BindingFlags.IgnoreCase);

            if (field?.GetValue(null) is int resourceId)
            {
                return resourceId;
            }

            return 0;
        }
#endif
    }
}

Screenshots of the demo running with Android emulator from VS 2026 📸

Here are images of the demo running with the Android emulator. Pixel 7 with API v36 is being used here.

Saturday, 21 October 2023

Using Azure Health Information extraction in Azure Cognitive Services

This article presents code how to extract Health information from arbitrary text using Azure Health Information extraction in Azure Cognitive Services. This technology uses NLP - natural language processing combined with AI techniques. A Github repo exists with the code for a running .NET MAUI Blazor demo in .NET 7 here:

https://github.com/toreaurstadboss/HealthTextAnalytics

A screenshot from the demo shows how it works below. The demo uses Azure AI Healthcare information extraction to extract entities of the text, such as a person's age, gender, employment and medical history and condition such as diagnosises, procedures and so on. The returned data in the demo is shown at the bottom of the demo, the raw data shows it is in the format as a json and in a FHIR format. Since we want FHIR format, we must use the REST api to get this information. Azure AI Healthcare information also extracts relations, which is connecting the entities together for semantic analysis of the text. Also, links exist for each entity for further reading. These are external systems such as Snomed CT and Snomed codes for each entity. Let's look at the source code for the demo next. We define a named http client in the MauiProgram.cs file which starts the application. We could move the code into a middleware extension method, but the code is kept simple in the demo.

MauiProgram.cs


  var azureEndpoint = Environment.GetEnvironmentVariable("AZURE_COGNITIVE_SERVICES_LANGUAGE_SERVICE_ENDPOINT");
  var azureKey = Environment.GetEnvironmentVariable("AZURE_COGNITIVE_SERVICES_LANGUAGE_SERVICE_KEY");

  if (string.IsNullOrWhiteSpace(azureEndpoint))
  {
      throw new ArgumentNullException(nameof(azureEndpoint), "Missing system environment variable: AZURE_COGNITIVE_SERVICES_LANGUAGE_SERVICE_ENDPOINT");
  }
  if (string.IsNullOrWhiteSpace(azureKey))
  {
      throw new ArgumentNullException(nameof(azureKey), "Missing system environment variable: AZURE_COGNITIVE_SERVICES_LANGUAGE_SERVICE_KEY");
  }

  var azureEndpointHost = new Uri(azureEndpoint);

  builder.Services.AddHttpClient("Az", httpClient =>
  {
      string baseUrl = azureEndpointHost.GetLeftPart(UriPartial.Authority); //https://stackoverflow.com/a/18708268/741368
      httpClient.BaseAddress = new Uri(baseUrl);
      //httpClient..Add("Content-type", "application/json");
      //httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));//ACCEPT header
      httpClient.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", azureKey);
  });



The content-type header will be specified instead inside the HttpRequestMessage shown further below and not in this named client. As we see, we must add both the endpoint base url and also the key in the Ocp-Apim-Subscription-key http header. Let's next look at how to create a POST request to the language resource endpoint that offers the health text analysis below.

HealthAnalyticsTextClientService.cs




using HealthTextAnalytics.Models;
using System.Diagnostics;
using System.Text;
using System.Text.Json.Nodes;

namespace HealthTextAnalytics.Util
{

    public class HealthAnalyticsTextClientService : IHealthAnalyticsTextClientService
    {

        private readonly IHttpClientFactory _httpClientFactory;
        private const int awaitTimeInMs = 500;
        private const int maxTimerWait = 10000;

        public HealthAnalyticsTextClientService(IHttpClientFactory httpClientFactory)
        {
            _httpClientFactory = httpClientFactory;
        }

        public async Task<HealthTextAnalyticsResponse> GetHealthTextAnalytics(string inputText)
        {
            var client = _httpClientFactory.CreateClient("Az");
            string requestBodyRaw = HealthAnalyticsTextHelper.CreateRequest(inputText);
            //https://learn.microsoft.com/en-us/azure/ai-services/language-service/text-analytics-for-health/how-to/call-api?tabs=ner
            var stopWatch = Stopwatch.StartNew();
            HttpRequestMessage request = CreateTextAnalyticsRequest(requestBodyRaw);
            var response = await client.SendAsync(request);
            var result = new HealthTextAnalyticsResponse();
            var timer = new PeriodicTimer(TimeSpan.FromMilliseconds(awaitTimeInMs));
            int timeAwaited = 0;

            while (await timer.WaitForNextTickAsync())
            {
                if (response.IsSuccessStatusCode)
                {
                    result.IsSearchPerformed = true;
                    var operationLocation = response.Headers.First(h => h.Key?.ToLower() == Constants.Constants.HttpHeaderOperationResultAvailable).Value.FirstOrDefault();

                    var resultFromHealthAnalysis = await client.GetAsync(operationLocation);
                    JsonNode resultFromService = await resultFromHealthAnalysis.GetJsonFromHttpResponse();
                    if (resultFromService.GetValue<string>("status") == "succeeded")
                    {
                        result.AnalysisResultRawJson = await resultFromHealthAnalysis.Content.ReadAsStringAsync();
                        result.ExecutionTimeInMilliseconds = stopWatch.ElapsedMilliseconds;
                        result.Entities.AddRange(HealthAnalyticsTextHelper.GetEntities(result.AnalysisResultRawJson));
                        result.CategorizedInputText = HealthAnalyticsTextHelper.GetCategorizedInputText(inputText, result.AnalysisResultRawJson);
                        break;
                    }
                }
                timeAwaited += 500;
                if (timeAwaited >= maxTimerWait)
                {
                    result.CategorizedInputText = $"ERR: Timeout. Operation to analyze input text using Azure HealthAnalytics language service timed out after waiting for {timeAwaited} ms.";
                    break;
                }
            }

            return result;
        }

        private static HttpRequestMessage CreateTextAnalyticsRequest(string requestBodyRaw)
        {
            var request = new HttpRequestMessage(HttpMethod.Post, Constants.Constants.AnalyzeTextEndpoint);
            request.Content = new StringContent(requestBodyRaw, Encoding.UTF8, "application/json");//CONTENT-TYPE header
            return request;
        }
    }

}



The code is using some helper methods to be shown next. As the code above shows, we must poll the Azure service until we get a reply from the service. We poll every 0.5 second up to a maxium of 10 seconds from the service. Typical requests takes about 3-4 seconds to process. Longer input text / 'documents' would need more processing time than 10 seconds, but for this demo, it works great.

HealthAnalyticsTextHelper.CreateRequest method


  public static string CreateRequest(string inputText)
  {
      //note - the id 1 here in the request is a 'local id' that must be unique per request. only one text is supported in the 
      //request genreated, however the service allows multiple documents and id's if necessary. in this demo, we only will send in one text at a time
      var request = new
      {
          analysisInput = new
          {
              documents = new[]
              {
                  new { text = inputText, id = "1", language = "en" }
              }
          },
          tasks = new[]
          {
              new { id = "analyze 1", kind = "Healthcare", parameters = new { fhirVersion = "4.0.1" } }
          }
      };
      return JsonSerializer.Serialize(request, new JsonSerializerOptions { WriteIndented = true });
  }



Creating the body of POST we use a template via a new anonymized object shown above which is what the REST service excepts. We could have multiple documents here, that is input texts, in this demo only one text / document is sent in. Note the use of id='1' and 'analyze 1' here. We have some helper methods in System.Text.Json here to extract the JSON data sent in the response.

JsonNodeUtil


 public static class JsonNodeUtil
 {

     public static async Task<JsonNode> GetJsonFromHttpResponse(this HttpResponseMessage response)
     {
         var resultFromService = JsonSerializer.Deserialize<JsonNode>(await response.Content.ReadAsStringAsync());
         return resultFromService;
     }

     public static T? GetValue<T>(this JsonNode jsonNode, string key)
     {
         if (jsonNode == null)
         {
             return default;
         }
         return jsonNode[key] != null ? jsonNode[key].GetValue<T>() : default;
     }

 }

 

More code exists for returning a categorized colored input text showing the entities of the input text in the helper below.

HealthAnalyticsTextHelper.cs - methods GetCategorizedInputText and GetBackgroundColor


 public static string GetCategorizedInputText(string inputText, string analysisText)
 {
     var sb = new StringBuilder(inputText);
     try
     {
         Root doc = JsonSerializer.Deserialize<Root>(analysisText);

         //try loading up the documents inside of the analysisText
         var entities = doc?.tasks?.items.FirstOrDefault()?.results?.documents?.SelectMany(d => d.entities)?.ToList();
         if (entities != null)
         {
             foreach (var row in entities.OrderByDescending(r => r.offset))
             {
                 sb.Insert(row.offset + row.length, "</b></span>");
                 sb.Insert(row.offset, $"<span style='color:{GetBackgroundColor(row)}' title='{row.category}: {row.text} Confidence: {row.confidenceScore} {row.name}'><b>");
             }
         }
     }
     catch (Exception err)
     {

         Console.WriteLine("Got an error while trying to load in analysis healthcare json: " + err.ToString());
     }
     return $"<pre style='text-wrap:wrap; max-height:500px;font-size: 10pt;font-family:Verdana, Geneva, Tahoma, sans-serif;'>{sb}</pre>";
 }

 private static string GetBackgroundColor(Entity row)
 {
     var cat = row?.category?.ToLower();
     string backgroundColor = cat switch
     {
         "age" => "purple",
         "diagnosis" => "orange",
         "gender" => "purple",
         "symptomorsign" => "purple",
         "direction" => "blue",
         "symptom" => "purple",
         "symptoms" => "purple",
         "bodystructure" => "blue",
         "body" => "purple",
         "structure" => "purple",
         "examinationname" => "green",
         "procedure" => "green",
         "treatmentname" => "green",
         "conditionqualifier" => "lightgreen",
         "time" => "lightgreen",
         "date" => "lightgreen",
         "familyrelation" => "purple",
         "employment" => "purple",
         "livingstatus" => "purple",
         "administrativeevent" => "darkgreen",
         "careenvironment" => "darkgreen",
         _ => "darkgray"
     };
     return backgroundColor;
 }




I have added the Domain classes from the service using the https://json2csharp.com/ website on the intial responses I got from the REST service using Postman. The REST Api might change in the future, that is, the JSON returned. In that case, you might want to adjust the domain classes here if the deserialization fails. It seems relatively stable though, I have tested the code for some weeks now. Finally, the categorized colored text code here had to remove newlines to get a correct indexing of the different entities found in the text. This code shows how to get rid of newlines of the inputted text.


 public static class StringExtensions
 {
     
     public static string CleanupAllWhiteSpace(this string input) => Regex.Replace(input ?? string.Empty, @"\s+", " ");
     
 }


Let's look at the UI in the Index.razor file below.

Index.razor


@page "/"
@using HealthTextAnalytics.Models;
@inject IHttpClientFactory _httpClientFactory;
@inject IHealthAnalyticsTextClientService _healthAnalyticsTextClientService;

<h3>Azure HealthCare Text Analysis - Azure Cognitive Services</h3>

<EditForm Model="@Model" OnValidSubmit="@Submit">
    <DataAnnotationsValidator />
    <ValidationSummary />

    <InputWatcher @ref="inputWatcher" FieldChanged="@FieldChanged" />

    <div class="form-group row">
        <label><strong>Text input</strong></label>
        <InputTextArea @onkeyup="@removeWhitespace" class="overflow-scroll" style="max-height:500px;max-width:900px;font-size: 10pt;font-family:Verdana, Geneva, Tahoma, sans-serif" @bind-Value="@Model.InputText" rows="5" />
    </div>

    <div class="form-group row"> 
        <div class="col">
            <br />
            <button class="btn btn-outline-primary" disabled="@isInvalid" type="submit">Run</button>
        </div>
        <div class="col">
        </div>
        <div class="col">
        </div>
    </div>

    <br />

@if (isProcessing)
{

        <div class="progress" style="max-width: 90%">
            <div class="progress-bar progress-bar-striped progress-bar-animated"
                 style="width: 100%; background-color: green">
                 Retrieving result from Azure HealthCare Text Analysis. Processing..
            </div>
        </div>
        <br />

}

    <div class="form-group row">
        <label><strong>Analysis result</strong></label>

        @if (isSearchPerformed)
    {
        <br />
        <b>Execution time took: @Model.ExecutionTime ms (milliseconds)</b><br />
        <br />

        <b>Categorized and analyzed Health Analysis of inputted text</b>
        @ms
        <br />
     
        <table class="table table-striped table-dark table-hover">
                <th>Category</th>
                <th>Text</th>
                <th>Name</th>
                <th>ConfidenceScore</th>
                <th>Offset</th>
                <th>Length</th>
                <th>Links</th>
            <tbody>
            @foreach (var entity in Model.EntititesInAnalyzedResult)
        {
            <tr>
                    <td>@entity.category</td>
                    <td>@entity.text</td>
                    <td>@entity.name</td>
                    <td>@entity.confidenceScore</td>
                    <td>@entity.offset</td>
                    <td>@entity.length</td>
                    <td>@string.Join(Environment.NewLine, (@entity.links ?? new List<Link>()).Select(l => l?.dataSource + " " + l?.id + " | "))</td>
                </tr>
            
        }
            </tbody>
            </table>

        <b>Health Analysis raw text from Azure service</b>
        <InputTextArea class="overflow-scroll" readonly="readonly" style="max-height:500px; max-width:900px;font-size: 10pt;font-family:Verdana, Geneva, Tahoma, sans-serif" @bind-Value="@Model.AnalysisResult" rows="1000" />

    }
   </div>

</EditForm>


The code-behind of Index.razor , looks like this.


using HealthTextAnalytics.Models;
using HealthTextAnalytics.Util;
using Microsoft.AspNetCore.Components;
using Microsoft.AspNetCore.Components.Web;

namespace HealthTextAnalytics.Pages
{
    public partial class Index
    {

        private IndexModel Model = new();
        MarkupString ms = new();
        private bool isProcessing = false;
        private bool isSearchPerformed = false;   

        private InputWatcher inputWatcher = new InputWatcher();
        private bool isInvalid = false;

        private void FieldChanged(string fieldName)
        {
            isInvalid = !inputWatcher.Validate();
        }
        
        protected override void OnParametersSet()
        {
            Model.InputText = SampleData.Sampledata.SamplePatientTextNote2.CleanupAllWhiteSpace();
            StateHasChanged();
        }

        private void removeWhitespace(KeyboardEventArgs eventArgs)
        {
            Model.InputText = Model.InputText.CleanupAllWhiteSpace();
            StateHasChanged();
        }

        private async Task Submit()
        {
            try
            {
                ResetFieldsForBeforeSearch();

                HealthTextAnalyticsResponse response = await _healthAnalyticsTextClientService.GetHealthTextAnalytics(Model.InputText);
                Model.EntititesInAnalyzedResult = response.Entities;
                Model.ExecutionTime = response.ExecutionTimeInMilliseconds;
                Model.AnalysisResult = response.AnalysisResultRawJson;

                ms = new MarkupString(response.CategorizedInputText);              
            }
            catch (Exception err)
            {
                Console.WriteLine(err);
            }
            finally
            {
                ResetFieldsAfterSearch();
                StateHasChanged();
            }
        }

        private void ResetFieldsForBeforeSearch()
        {
            isProcessing = true;
            isSearchPerformed = false;
            ms = new MarkupString(string.Empty);
            Model.EntititesInAnalyzedResult.Clear();
            Model.AnalysisResult = string.Empty;
        }

        private void ResetFieldsAfterSearch()
        {
            isProcessing = false;
            isSearchPerformed = true;
        }

    }
}


Saturday, 14 October 2023

Using Image Analysis in Azure AI Cognitive Services

I have added a demo .NET MAUI Blazor app that uses Image Analysis in Computer Vision in Azure Cognitive Services. Note that Image Analysis is not available in all Azure data centers. For example, Norway East does not have this feature. However, North Europe Azure data center do have the feature, the data center i Ireland. A Github repo exists for this demo here:

https://github.com/toreaurstadboss/Image.Analyze.Azure.Ai

A screen shot for this demo is shown below: Demo screenshot The demo allows you to upload a picture (supported formats are .jpeg, .jpg and .png, but Azure AI Image Analyzer supports a lot of other image formats too). The demo shows a preview of the selected image and to the right an image of bounding boxes of objects in the image. A list of tags extracted from the image are also shown. Raw data from the Azure Image Analyzer service is shown in the text box area below the pictures, with a list of tags to the right. The demo is written with .NET Maui Blazor and .NET 6. Let us look at some code for making this demo. ImageSaveService.cs


using Image.Analyze.Azure.Ai.Models;
using Microsoft.AspNetCore.Components.Forms;

namespace Ocr.Handwriting.Azure.AI.Services
{

    public class ImageSaveService : IImageSaveService
    {

        public async Task<ImageSaveModel> SaveImage(IBrowserFile browserFile)
        {
            var buffers = new byte[browserFile.Size];
            var bytes = await browserFile.OpenReadStream(maxAllowedSize: 30 * 1024 * 1024).ReadAsync(buffers);
            string imageType = browserFile.ContentType;

            var basePath = FileSystem.Current.AppDataDirectory;
            var imageSaveModel = new ImageSaveModel
            {
                SavedFilePath = Path.Combine(basePath, $"{Guid.NewGuid().ToString("N")}-{browserFile.Name}"),
                PreviewImageUrl = $"data:{imageType};base64,{Convert.ToBase64String(buffers)}",
                FilePath = browserFile.Name,
                FileSize = bytes / 1024,
            };

            await File.WriteAllBytesAsync(imageSaveModel.SavedFilePath, buffers);

            return imageSaveModel;
        }

    }
}

//Interface defined inside IImageService.cs shown below
using Image.Analyze.Azure.Ai.Models;
using Microsoft.AspNetCore.Components.Forms;

namespace Ocr.Handwriting.Azure.AI.Services
{
  
    public interface IImageSaveService
    {

        Task<ImageSaveModel> SaveImage(IBrowserFile browserFile);

    }

}


The ImageSaveService saves the uploaded image from the IBrowserFile into a base-64 string from the image bytes of the uploaded IBrowserFile via OpenReadStream of the IBrowserFile. This allows us to preview the uploaded image. The code also saves the image to the AppDataDirectory that MAUI supports - FileSystem.Current.AppDataDirectory. Let's look at how to call the analysis service itself, it is actually quite straight forward. ImageAnalyzerService.cs


using Azure;
using Azure.AI.Vision.Common;
using Azure.AI.Vision.ImageAnalysis;

namespace Image.Analyze.Azure.Ai.Lib
{

    public class ImageAnalyzerService : IImageAnalyzerService
    {

        public ImageAnalyzer CreateImageAnalyzer(string imageFile)
        {
            string key = Environment.GetEnvironmentVariable("AZURE_COGNITIVE_SERVICES_VISION_SECONDARY_KEY");
            string endpoint = Environment.GetEnvironmentVariable("AZURE_COGNITIVE_SERVICES_VISION_SECONDARY_ENDPOINT");
            var visionServiceOptions = new VisionServiceOptions(new Uri(endpoint), new AzureKeyCredential(key));

            using VisionSource visionSource = CreateVisionSource(imageFile);

            var analysisOptions = CreateImageAnalysisOptions();

            var analyzer = new ImageAnalyzer(visionServiceOptions, visionSource, analysisOptions);
            return analyzer;

        }

        private static VisionSource CreateVisionSource(string imageFile)
        {
            using var stream = File.OpenRead(imageFile);
            using var reader = new StreamReader(stream);
            byte[] imageBuffer;
            using (var streamReader = new MemoryStream())
            {
                stream.CopyTo(streamReader);
                imageBuffer = streamReader.ToArray();
            }

            using var imageSourceBuffer = new ImageSourceBuffer();
            imageSourceBuffer.GetWriter().Write(imageBuffer);
            return VisionSource.FromImageSourceBuffer(imageSourceBuffer);
        }

        private static ImageAnalysisOptions CreateImageAnalysisOptions() => new ImageAnalysisOptions
        {
            Language = "en",
            GenderNeutralCaption = false,
            Features =
              ImageAnalysisFeature.CropSuggestions
            | ImageAnalysisFeature.Caption
            | ImageAnalysisFeature.DenseCaptions
            | ImageAnalysisFeature.Objects
            | ImageAnalysisFeature.People
            | ImageAnalysisFeature.Text
            | ImageAnalysisFeature.Tags
        };

    }

}

//interface shown below 

 public interface IImageAnalyzerService
 {
     ImageAnalyzer CreateImageAnalyzer(string imageFile);
 }



We retrieve environment variables here and we create an ImageAnalyzer. We create a Vision source from the saved picture we uploaded and open a stream to it using File.OpenRead method on System.IO. Since we saved the file in the AppData folder of the .NET MAUI app, we can read this file. We set up the image analysis options and the vision service options. We then call the return the image analyzer. Let's look at the code-behind of the index.razor file that initializes the Image analyzer, and runs the Analyze method of it. Index.razor.cs
 
 
 using Azure.AI.Vision.ImageAnalysis;
using Image.Analyze.Azure.Ai.Extensions;
using Image.Analyze.Azure.Ai.Models;
using Microsoft.AspNetCore.Components.Forms;
using Microsoft.JSInterop;
using System.Text;

namespace Image.Analyze.Azure.Ai.Pages
{
    partial class Index
    {

        private IndexModel Model = new();

        //https://learn.microsoft.com/en-us/azure/ai-services/computer-vision/how-to/call-analyze-image-40?WT.mc_id=twitter&pivots=programming-language-csharp

        private string ImageInfo = string.Empty;

        private async Task Submit()
        {
            if (Model.PreviewImageUrl == null || Model.SavedFilePath == null)
            {
                await Application.Current.MainPage.DisplayAlert($"MAUI Blazor Image Analyzer App", $"You must select an image first before running Image Analysis. Supported formats are .jpeg, .jpg and .png", "Ok", "Cancel");
                return;
            }

            using var imageAnalyzer = ImageAnalyzerService.CreateImageAnalyzer(Model.SavedFilePath);

            ImageAnalysisResult analysisResult = await imageAnalyzer.AnalyzeAsync();

            if (analysisResult.Reason == ImageAnalysisResultReason.Analyzed)
            {
                Model.ImageAnalysisOutputText = analysisResult.OutputImageAnalysisResult();
                Model.Caption = $"{analysisResult.Caption.Content} Confidence: {analysisResult.Caption.Confidence.ToString("F2")}";
                Model.Tags = analysisResult.Tags.Select(t => $"{t.Name} (Confidence: {t.Confidence.ToString("F2")})").ToList();
                var jsonBboxes = analysisResult.GetBoundingBoxesJson();
                await JsRunTime.InvokeVoidAsync("LoadBoundingBoxes", jsonBboxes);
            }
            else
            {
                ImageInfo = $"The image analysis did not perform its analysis. Reason: {analysisResult.Reason}";
            }

            StateHasChanged(); //visual refresh here
        }

        private async Task CopyTextToClipboard()
        {
            await Clipboard.SetTextAsync(Model.ImageAnalysisOutputText);
            await Application.Current.MainPage.DisplayAlert($"MAUI Blazor Image Analyzer App", $"The copied text was put into the clipboard. Character length: {Model.ImageAnalysisOutputText?.Length}", "Ok", "Cancel");
        }

        private async Task OnInputFile(InputFileChangeEventArgs args)
        {
            var imageSaveModel = await ImageSaveService.SaveImage(args.File);
            Model = new IndexModel(imageSaveModel);
            await Application.Current.MainPage.DisplayAlert($"MAUI Blazor ImageAnalyzer app App", $"Wrote file to location : {Model.SavedFilePath} Size is: {Model.FileSize} kB", "Ok", "Cancel");
        }


    }
}
 
 
In the code-behind above we have a submit handler called Submit. We there analyze the image and send the result both to the UI and also to a client side Javascript method using IJSRuntime in .NET MAUI Blazor. Let's look at the two helper methods of ImageAnalysisResult next. ImageAnalysisResultExtensions.cs
 
 
 using Azure.AI.Vision.ImageAnalysis;
using System.Text;

namespace Image.Analyze.Azure.Ai.Extensions
{
    public static class ImageAnalysisResultExtensions
    {

        public static string GetBoundingBoxesJson(this ImageAnalysisResult result)
        {
            var sb = new StringBuilder();
            sb.AppendLine(@"[");

            int objectIndex = 0;
            foreach (var detectedObject in result.Objects)
            {
                sb.Append($"{{ \"Name\": \"{detectedObject.Name}\", \"Y\": {detectedObject.BoundingBox.Y}, \"X\": {detectedObject.BoundingBox.X}, \"Height\": {detectedObject.BoundingBox.Height}, \"Width\": {detectedObject.BoundingBox.Width}, \"Confidence\": \"{detectedObject.Confidence:0.0000}\" }}");
                objectIndex++;
                if (objectIndex < result.Objects?.Count)
                {
                    sb.Append($",{Environment.NewLine}");
                }
                else
                {
                    sb.Append($"{Environment.NewLine}");
                }
            }
            sb.Remove(sb.Length - 2, 1); //remove trailing comma at the end
            sb.AppendLine(@"]");
            return sb.ToString();
        }

        public static string OutputImageAnalysisResult(this ImageAnalysisResult result)
        {
            var sb = new StringBuilder();

            if (result.Reason == ImageAnalysisResultReason.Analyzed)
            {

                sb.AppendLine($" Image height = {result.ImageHeight}");
                sb.AppendLine($" Image width = {result.ImageWidth}");
                sb.AppendLine($" Model version = {result.ModelVersion}");

                if (result.Caption != null)
                {
                    sb.AppendLine(" Caption:");
                    sb.AppendLine($"   \"{result.Caption.Content}\", Confidence {result.Caption.Confidence:0.0000}");
                }

                if (result.DenseCaptions != null)
                {
                    sb.AppendLine(" Dense Captions:");
                    foreach (var caption in result.DenseCaptions)
                    {
                        sb.AppendLine($"   \"{caption.Content}\", Bounding box {caption.BoundingBox}, Confidence {caption.Confidence:0.0000}");
                    }
                }

                if (result.Objects != null)
                {
                    sb.AppendLine(" Objects:");
                    foreach (var detectedObject in result.Objects)
                    {
                        sb.AppendLine($"   \"{detectedObject.Name}\", Bounding box {detectedObject.BoundingBox}, Confidence {detectedObject.Confidence:0.0000}");
                    }
                }

                if (result.Tags != null)
                {
                    sb.AppendLine($" Tags:");
                    foreach (var tag in result.Tags)
                    {
                        sb.AppendLine($"   \"{tag.Name}\", Confidence {tag.Confidence:0.0000}");
                    }
                }

                if (result.People != null)
                {
                    sb.AppendLine($" People:");
                    foreach (var person in result.People)
                    {
                        sb.AppendLine($"   Bounding box {person.BoundingBox}, Confidence {person.Confidence:0.0000}");
                    }
                }

                if (result.CropSuggestions != null)
                {
                    sb.AppendLine($" Crop Suggestions:");
                    foreach (var cropSuggestion in result.CropSuggestions)
                    {
                        sb.AppendLine($"   Aspect ratio {cropSuggestion.AspectRatio}: "
                            + $"Crop suggestion {cropSuggestion.BoundingBox}");
                    };
                }

                if (result.Text != null)
                {
                    sb.AppendLine($" Text:");
                    foreach (var line in result.Text.Lines)
                    {
                        string pointsToString = "{" + string.Join(',', line.BoundingPolygon.Select(pointsToString => pointsToString.ToString())) + "}";
                        sb.AppendLine($"   Line: '{line.Content}', Bounding polygon {pointsToString}");

                        foreach (var word in line.Words)
                        {
                            pointsToString = "{" + string.Join(',', word.BoundingPolygon.Select(pointsToString => pointsToString.ToString())) + "}";
                            sb.AppendLine($"     Word: '{word.Content}', Bounding polygon {pointsToString}, Confidence {word.Confidence:0.0000}");
                        }
                    }
                }

                var resultDetails = ImageAnalysisResultDetails.FromResult(result);
                sb.AppendLine($" Result details:");
                sb.AppendLine($"   Image ID = {resultDetails.ImageId}");
                sb.AppendLine($"   Result ID = {resultDetails.ResultId}");
                sb.AppendLine($"   Connection URL = {resultDetails.ConnectionUrl}");
                sb.AppendLine($"   JSON result = {resultDetails.JsonResult}");
            }
            else
            {
                var errorDetails = ImageAnalysisErrorDetails.FromResult(result);
                sb.AppendLine(" Analysis failed.");
                sb.AppendLine($"   Error reason : {errorDetails.Reason}");
                sb.AppendLine($"   Error code : {errorDetails.ErrorCode}");
                sb.AppendLine($"   Error message: {errorDetails.Message}");
            }

            return sb.ToString();
        }

    }
}


  
 
Finally, let's look at the client side Javascript function that we call and send the bounding boxes json to draw the boxes. We will use Canvas in HTML 5 to show the picture and the bounding boxes of objects found in the image. index.html
 
 
 	<script type="text/javascript">

		var colorPalette = ["red", "yellow", "blue", "green", "fuchsia", "moccasin", "purple", "magenta", "aliceblue", "lightyellow", "lightgreen"];

		function rescaleCanvas() {
			var img = document.getElementById('PreviewImage');
			var canvas = document.getElementById('PreviewImageBbox');
			canvas.width = img.width;
			canvas.height = img.height;
		}

		function getColor() {
			var colorIndex = parseInt(Math.random() * 10);
			var color = colorPalette[colorIndex];
			return color;
		}

		function LoadBoundingBoxes(objectDescriptions) {
			if (objectDescriptions == null || objectDescriptions == false) {
				alert('did not find any objects in image. returning from calling load bounding boxes : ' + objectDescriptions);
				return;
			}

			var objectDesc = JSON.parse(objectDescriptions);
			//alert('calling load bounding boxes, starting analysis on clientside js : ' + objectDescriptions);

			rescaleCanvas();
			var canvas = document.getElementById('PreviewImageBbox');
			var img = document.getElementById('PreviewImage');

			var ctx = canvas.getContext('2d');
			ctx.drawImage(img, img.width, img.height);

			ctx.font = "10px Verdana";

			for (var i = 0; i < objectDesc.length; i++) {
				ctx.beginPath();
				ctx.strokeStyle = "black";
				ctx.lineWidth = 1;
				ctx.fillText(objectDesc[i].Name, objectDesc[i].X + objectDesc[i].Width / 2, objectDesc[i].Y + objectDesc[i].Height / 2);
				ctx.fillText("Confidence: " + objectDesc[i].Confidence, objectDesc[i].X + objectDesc[i].Width / 2, 10 + objectDesc[i].Y + objectDesc[i].Height / 2);
			}

			for (var i = 0; i < objectDesc.length; i++) {
				ctx.fillStyle = getColor();
				ctx.globalAlpha = 0.2;
				ctx.fillRect(objectDesc[i].X, objectDesc[i].Y, objectDesc[i].Width, objectDesc[i].Height);
				ctx.lineWidth = 3;
				ctx.strokeStyle = "blue";
				ctx.rect(objectDesc[i].X, objectDesc[i].Y, objectDesc[i].Width, objectDesc[i].Height);
				ctx.fillStyle = "black";
				ctx.fillText("Color: " + getColor(), objectDesc[i].X + objectDesc[i].Width / 2, 20 + objectDesc[i].Y + objectDesc[i].Height / 2);

				ctx.stroke();
			}

			ctx.drawImage(img, 0, 0);


			console.log('got these object descriptions:');
			console.log(objectDescriptions);

		}
	</script>

 
  
The index.html file in wwwroot is the place we usually put extra css and js for MAUI Blazor apps and Blazor apps. I have chosen to put the script directly into the index.html file and not in a .js file, but that is an option to be chosen to tidy up a bit more. So there you have it, we can relatively easily find objects in images using Azure analyze image service in Azure Cognitive Services. We can get tags and captions of the image. In the demo the caption is shown above the picture loaded. Azure Computer vision service is really good since it has got a massive training set and can recognize a lot of different objects for different usages. As you see in the source code, I have the key and endpoint inside environment variables that the code expects exists. Never expose keys and endpoints in your source code.

Friday, 22 September 2023

Using Azure Computer Vision to perform Optical Character Recognition (OCR)

This article shows how you can use Azure Computer vision in Azure Cognitive Services to perform Optical Character Recognition (OCR). The Computer vision feature is available by adding a Computer Vision resource in Azure Portal. I have made a .NET MAUI Blazor app and the Github Repo for it is available here : https://github.com/toreaurstadboss/Ocr.Handwriting.Azure.AI.Models
Let us first look at the .csproj of the Lib project in this repo.


<Project Sdk="Microsoft.NET.Sdk.Razor">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>
  <ItemGroup>
    <SupportedPlatform Include="browser" />
  </ItemGroup>

	<ItemGroup>
		<PackageReference Include="Microsoft.Azure.CognitiveServices.Vision.ComputerVision" Version="7.0.1" />
		<PackageReference Include="Microsoft.AspNetCore.Components.Web" Version="6.0.19" />
	</ItemGroup>

</Project>


The following class generates ComputerVision clients that can be used to extract different information from streams and files containing video and images. We are going to focus on images and extracting text via OCR. Azure Computer Vision can also extract handwritten text in addition to regular text written by typewriters or text inside images and similar. Azure Computer Vision also can detect shapes in images and classify objects. This demo only focuses on text extraction form images. ComputerVisionClientFactory


using Microsoft.Azure.CognitiveServices.Vision.ComputerVision;

namespace Ocr.Handwriting.Azure.AI.Lib
{

    public interface IComputerVisionClientFactory
    {
        ComputerVisionClient CreateClient();
    }

    /// <summary>
    /// Client factory for Azure Cognitive Services - Computer vision.
    /// </summary>
    public class ComputerVisionClientFactory : IComputerVisionClientFactory
    {
        // Add your Computer Vision key and endpoint
        static string? _key = Environment.GetEnvironmentVariable("AZURE_COGNITIVE_SERVICES_VISION_KEY");
        static string? _endpoint = Environment.GetEnvironmentVariable("AZURE_COGNITIVE_SERVICES_VISION_ENDPOINT");

        public ComputerVisionClientFactory() : this(_key, _endpoint)
        {
        }

        public ComputerVisionClientFactory(string? key, string? endpoint)
        {
            _key = key;
            _endpoint = endpoint;
        }

        public ComputerVisionClient CreateClient()
        {
            if (_key == null)
            {
                throw new ArgumentNullException(_key, "The AZURE_COGNITIVE_SERVICES_VISION_KEY is not set. Set a system-level environment variable or provide this value by calling the overloaded constructor of this class.");
            }
            if (_endpoint == null)
            {
                throw new ArgumentNullException(_key, "The AZURE_COGNITIVE_SERVICES_VISION_ENDPOINT is not set. Set a system-level environment variable or provide this value by calling the overloaded constructor of this class.");
            }

            var client = Authenticate(_key!, _endpoint!);
            return client;
        }

        public static ComputerVisionClient Authenticate(string key, string endpoint) =>
            new ComputerVisionClient(new ApiKeyServiceClientCredentials(key))
            {
                Endpoint = endpoint
            };

    }
}



The setup of the endpoint and key of the Computer Vision resource is done via system-level envrionment variables. Next up, let's look at retrieving OCR text from images. Here we use ComputerVisionClient. We open up a stream of a file, an image, using File.OpenReadAsync and then the method ReadInStreamAsync of Computer vision client. The image we will load up in the app is selected by the user and the image is previewed and saved using MAUI Storage lib (inside the Appdata folder). OcrImageService.cs


using Microsoft.Azure.CognitiveServices.Vision.ComputerVision;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;
using Microsoft.Extensions.Logging;
using System.Diagnostics;
using ReadResult = Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models.ReadResult;

namespace Ocr.Handwriting.Azure.AI.Lib
{

    public interface IOcrImageService
    {
        Task<IList<ReadResult?>?> GetReadResults(string imageFilePath);
        Task<string> GetReadResultsText(string imageFilePath);
    }

    public class OcrImageService : IOcrImageService
    {
        private readonly IComputerVisionClientFactory _computerVisionClientFactory;
        private readonly ILogger<OcrImageService> _logger;

        public OcrImageService(IComputerVisionClientFactory computerVisionClientFactory, ILogger<OcrImageService> logger)
        {
            _computerVisionClientFactory = computerVisionClientFactory;
            _logger = logger;
        }

        private ComputerVisionClient CreateClient() => _computerVisionClientFactory.CreateClient();

        public async Task<string> GetReadResultsText(string imageFilePath)
        {
            var readResults = await GetReadResults(imageFilePath);
            var ocrText = ExtractText(readResults?.FirstOrDefault());
            return ocrText;
        }

        public async Task<IList<ReadResult?>?> GetReadResults(string imageFilePath)
        {
            if (string.IsNullOrWhiteSpace(imageFilePath))
            {
                return null;
            }

            try
            {
                var client = CreateClient();

                //Retrieve OCR results 

                using (FileStream stream = File.OpenRead(imageFilePath))
                {
                    var textHeaders = await client.ReadInStreamAsync(stream);
                    string operationLocation = textHeaders.OperationLocation;
                    string operationId = operationLocation[^36..]; //hat operator of C# 8.0 : this slices out the last 36 chars, which contains the guid chars which are 32 hexadecimals chars + four hyphens

                    ReadOperationResult results;

                    do
                    {
                        results = await client.GetReadResultAsync(Guid.Parse(operationId));
                        _logger.LogInformation($"Retrieving OCR results for operationId {operationId} for image {imageFilePath}");
                    }
                    while (results.Status == OperationStatusCodes.Running || results.Status == OperationStatusCodes.NotStarted);

                    IList<ReadResult?> result = results.AnalyzeResult.ReadResults;
                    return result;

                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
                return null;
            }
        }

        private static string ExtractText(ReadResult? readResult) => string.Join(Environment.NewLine, readResult?.Lines?.Select(l => l.Text) ?? new List<string>());

    }

}
                                           

Let's look at the MAUI Blazor project in the app. The MauiProgram.cs looks like this. MauiProgram.cs


using Ocr.Handwriting.Azure.AI.Data;
using Ocr.Handwriting.Azure.AI.Lib;
using Ocr.Handwriting.Azure.AI.Services;
using TextCopy;

namespace Ocr.Handwriting.Azure.AI;

public static class MauiProgram
{
    public static MauiApp CreateMauiApp()
    {
        var builder = MauiApp.CreateBuilder();
        builder
            .UseMauiApp<App>()
            .ConfigureFonts(fonts =>
            {
                fonts.AddFont("OpenSans-Regular.ttf", "OpenSansRegular");
            });

        builder.Services.AddMauiBlazorWebView();
#if DEBUG
        builder.Services.AddBlazorWebViewDeveloperTools();
        builder.Services.AddLogging();
#endif

        builder.Services.AddSingleton<WeatherForecastService>();
        builder.Services.AddScoped<IComputerVisionClientFactory, ComputerVisionClientFactory>();
        builder.Services.AddScoped<IOcrImageService, OcrImageService>();
        builder.Services.AddScoped<IImageSaveService, ImageSaveService>();

        builder.Services.InjectClipboard();

        return builder.Build();
    }
}



We also need some code to preview and save the image an end user chooses. The IImageService looks like this. ImageSaveService


using Microsoft.AspNetCore.Components.Forms;
using Ocr.Handwriting.Azure.AI.Models;

namespace Ocr.Handwriting.Azure.AI.Services
{

    public class ImageSaveService : IImageSaveService
    {

        public async Task<ImageSaveModel> SaveImage(IBrowserFile browserFile)
        {
            var buffers = new byte[browserFile.Size];
            var bytes = await browserFile.OpenReadStream(maxAllowedSize: 30 * 1024 * 1024).ReadAsync(buffers);
            string imageType = browserFile.ContentType;

            var basePath = FileSystem.Current.AppDataDirectory;
            var imageSaveModel = new ImageSaveModel
            {
                SavedFilePath = Path.Combine(basePath, $"{Guid.NewGuid().ToString("N")}-{browserFile.Name}"),
                PreviewImageUrl = $"data:{imageType};base64,{Convert.ToBase64String(buffers)}",
                FilePath = browserFile.Name,
                FileSize = bytes / 1024,
            };

            await File.WriteAllBytesAsync(imageSaveModel.SavedFilePath, buffers);

            return imageSaveModel;
        }

    }
}


Note the use of maxAllowedSize of IBrowserfile.OpenReadStream method, this is a good practice since IBrowserFile only supports 512 kB per default. I set it in the app to 30 MB to support some high res images too. We preview the image as base-64 here and we also save the image also. Note the use of FileSystem.Current.AppDataDirectory as base path here. Here we use nuget package Microsoft.Maui.Storage. These are the packages that is used for the MAUI Blazor project of the app. Ocr.Handwriting.Azure.AI.csproj



    <ItemGroup>
      <PackageReference Include="Microsoft.Azure.CognitiveServices.Vision.ComputerVision" Version="7.0.1" />
      <PackageReference Include="TextCopy" Version="6.2.1" />
    </ItemGroup>


The GUI looks like this : Index.razor


@page "/"
@using Ocr.Handwriting.Azure.AI.Models;
@using Microsoft.Azure.CognitiveServices.Vision.ComputerVision;
@using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;
@using Ocr.Handwriting.Azure.AI.Lib;
@using Ocr.Handwriting.Azure.AI.Services;
@using TextCopy;

@inject IImageSaveService ImageSaveService
@inject IOcrImageService OcrImageService 
@inject IClipboard Clipboard

<h1>Azure AI OCR Text recognition</h1>


<EditForm Model="Model" OnValidSubmit="@Submit" style="background-color:aliceblue">
    <DataAnnotationsValidator />
    <label><b>Select a picture to run OCR</b></label><br />
    <InputFile OnChange="@OnInputFile" accept=".jpeg,.jpg,.png" />
    <br />
    <code class="alert-secondary">Supported file formats: .jpeg, .jpg and .png</code>
    <br />
    @if (Model.PreviewImageUrl != null) { 
        <label class="alert-info">Preview of the selected image</label>
        <div style="overflow:auto;max-height:300px;max-width:500px">
            <img class="flagIcon" src="@Model.PreviewImageUrl" /><br />
        </div>

        <code class="alert-light">File Size (kB): @Model.FileSize</code>
        <br />
        <code class="alert-light">File saved location: @Model.SavedFilePath</code>
        <br />

        <label class="alert-info">Click the button below to start running OCR using Azure AI</label><br />
        <br />
        <button type="submit">Submit</button> <button style="margin-left:200px" type="button" class="btn-outline-info" @onclick="@CopyTextToClipboard">Copy to clipboard</button>
        <br />
        <br />
        <InputTextArea style="width:1000px;height:300px" readonly="readonly" placeholder="Detected text in the image uploaded" @bind-Value="Model!.OcrOutputText" rows="5"></InputTextArea>
    }
</EditForm>


@code {

    private IndexModel Model = new();

    private async Task OnInputFile(InputFileChangeEventArgs args)
    {       
        var imageSaveModel = await ImageSaveService.SaveImage(args.File);
        Model = new IndexModel(imageSaveModel);
        await Application.Current.MainPage.DisplayAlert($"MAUI Blazor OCR App", $"Wrote file to location : {Model.SavedFilePath} Size is: {Model.FileSize} kB", "Ok", "Cancel");
    }

    public async Task CopyTextToClipboard()
    {
        await Clipboard.SetTextAsync(Model.OcrOutputText);
        await Application.Current.MainPage.DisplayAlert($"MAUI Blazor OCR App", $"The copied text was put into the clipboard. Character length: {Model.OcrOutputText?.Length}", "Ok", "Cancel");

    }

    private async Task Submit()
    {
        if (Model.PreviewImageUrl == null || Model.SavedFilePath == null)
        {
            await Application.Current.MainPage.DisplayAlert($"MAUI Blazor OCR App", $"You must select an image first before running OCR. Supported formats are .jpeg, .jpg and .png", "Ok", "Cancel");
            return;
        }
        Model.OcrOutputText = await OcrImageService.GetReadResultsText(Model.SavedFilePath);
        StateHasChanged(); //visual refresh here
    }

}


The UI works like this. The user selects an image. As we can see by the 'accept' html attribute, the .jpeg, .jpg and .png extensions are allowed in the file input dialog. When the user selects an image, the image is saved and previewed in the UI. By hitting the Submit button, the OCR service in Azure is contacted and text is retrieved and displayed in the text area below, if any text is present in the image. A button allows copying the text into the clipboard. Here are some screenshots of the app.