Writing and reading unmapped properties in Azure Cosmos DB

This article presents code that shows how you can read and write unmapped properties in Azure Cosmos DB. An unmapped property in Azure Cosmos is a property that is NOT part of a model in your domain models, for example you could set a LastUpdateBy or other metadata and not expose this in your domain models. This is similar to shadow properties in Entity Framework, but these do exist in your domain models and are ignored actively using
Fluent API when configuring your db context. In Azure Cosmos, the RAW json is exposed in the "__json" property. This can be read using Newtonsoft library and you can add properties that are unmapped and read them afterwards. You should not only save the changes to persist them in Azure Cosmos but also re-read the data again to get an updated item in your container (table) in Azure Cosmos DB. The following extension methods allows to write and read such unmapped properties easier using Azure Cosmos DB.

public static class AzureCosmosEntityExtensions
{

    public static TResult? GetUnmappedProperty<TResult, T>(this T entity, string propname, DbContext context) where T : class {
        if (entity == null)
        {
            return default(TResult);
        }
        var entry = context.Entry(entity);
        var rawJson = entry.Property<JObject>("__jObject");
        var currentValueProp = rawJson.CurrentValue[propname];
        if (currentValueProp == null)
        {
            return default(TResult);
        }
        var currentValuePropCasted = currentValueProp.ToObject<TResult?>();
        return currentValuePropCasted;
    }

    public static void SetUnmappedProperty<T>(this T entity, string propname, object value, DbContext context) where T : class
    {
        if (entity == null)
        {
            return;
        }
        var entry = context.Entry(entity);
        var rawJson = entry.Property<JObject>("__jObject");
        rawJson.CurrentValue[propname] = JToken.FromObject(value);
        entry.State = EntityState.Modified;
    }

}

Let's see some sample code to set this up. Consider the following model :

 public class Address
  {
    public string AddressId { get; set; }
    public string State { get; set; }
    public string City { get; set; }
    public string Street { get; set; }
    public string HouseNumber { get; set; }
  }

Let's add another unmapped property "LastUpdate" that are not exposed in the domain model, but is an unmapped model. We must as mentioned make sure to reload the data we read here after saving the entity to test out reading the unmapped property. Note that we set ModelState to Modified to trigger the saving of these unmapped properties, since the ChangeTracker is not tracking them and EF will not save the updates to these unmapped properties if this is not done.

     //Create a db context and for an item entity in a container (table) in Azure Cosmos DB,
     //set the unmapped property to a value and also read out this property after saving it and reloading the entity

     using var context = await contextFactory.CreateDbContextAsync();

     var address = await context.Addresses.FirstAsync();
     const string unmappedPropKey = "LastUpdate";
     address.SetUnmappedProperty(unmappedPropKey, DateTime.UtcNow, context);
     await context.SaveChangesAsync();
     address = await context.Addresses.FirstAsync();
     var unnmappedProp = address.GetUnmappedProperty<DateTime, Address>(unmappedPropKey, context);

The following screenshot below shows the unmapped property LastUpdate was written to Azure Cosmos DB item in the container (table).

Loading references (navigational properties) for an item in Azure Cosmos DB

The extensions methods shown here can be applied in general in Entity Framework Core. In Azure Cosmos DB, if you use the FindAsync method for example, you will not load the references of the item automatically. Instead, you must explicitly go via Entry method and then to LoadAsync on each Reference. A Reference is also called a navigational property in EF. Let's first consider this code finding some Trip data item (data is stored in json format in Azure Cosmos DB since it is schema less and non-relational db or a 'document DB') and related data inside the Driver, Address. The POCO for Trip looks like this:

#region Info and license

#endregion

namespace TransportApp.Domain
{
  public class Trip
  {
    public string TripId { get; set; }
    public DateTime BeginUtc { get; set; }
    public DateTime? EndUtc { get; set; }
    public short PassengerCount { get; set; }

    public string DriverId { get; set; }
    public Driver Driver { get; set; }
    public string VehicleId { get; set; }
    public Vehicle Vehicle { get; set; }
    public string FromAddressId { get; set; }
    public Address FromAddress { get; set; }
    public string ToAddressId { get; set; }
    public Address ToAddress { get; set; }
  }
}

We create a db context for Azure Cosmos DB like this :

 private readonly IDbContextFactory _contextFactory;

 public SomeService(IDbContextFactory contextFactory)
 {
  _contextFactory = contextFactory; 
  
  //..
  
 }
 
 public async Task RunSomeDemoCode(){
    using var context = await contextFactory.CreateDbContextAsync();
 
 }
 	

Once we have our context object we can get data from Azure Cosmos DB.

 
 public async Task RunSomeDemoCode(){
    using var context = await contextFactory.CreateDbContextAsync();
    
    var trip1 = await otherContext.Trips.FindAsync($"{nameof(Trip)}-1");
    await otherContext.LoadEntityWithAllReferences(trip1!);
 
 }
 	

Note that we probably want to check that trip1 object is not null here. We want to get the relations also here. This is not automatically loaded in Azure Cosmos DB ! We can load the relations or navigational properties using the following extensions methods listed below. If you use the method accepting an EntityEntry, you must use the Entry method first, like shown in the other method.

 public static class EntityEntryExtensions
    {
        public static async Task LoadAllReferences<T>(this EntityEntry<T> entry) where T : class
        {
            foreach (var reference in entry.References)
            {
                await reference.LoadAsync();
            }
        }
    }

    public static class EntityExtensions
    {
        public static async Task LoadEntityWithAllReferences<T>(this DbContext dbContext, T dataItem) where T : class
        {
            if (dataItem == null)
            {
                return;
            }
            var entity = dbContext.Entry(dataItem!);
            foreach (var reference in entity.References)
            {
                await reference.LoadAsync();
            }
        }
    }

I have added two screenshots here, first before calling the method LoadEntityWithAllReferences, and afterwards.

Before: 




After:

---

As we can see from the screen shots, the references has been loaded and now you avoid to manually have to load one and one reference property / navigation property. Note - about some of the sample code - it came from a Pluralsight Course. I have looked more into the extension methods here myself.

/*
  This demo application accompanies Pluralsight course 'Using EF Core 6 with Azure Cosmos DB', 
  by Jurgen Kevelaers. See https://pluralsight.pxf.io/efcore6-cosmos.

  MIT License

  Copyright (c) 2022 Jurgen Kevelaers

  Permission is hereby granted, free of charge, to any person obtaining a copy
  of this software and associated documentation files (the "Software"), to deal
  in the Software without restriction, including without limitation the rights
  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  copies of the Software, and to permit persons to whom the Software is
  furnished to do so, subject to the following conditions:

  The above copyright notice and this permission notice shall be included in all
  copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  SOFTWARE.
*/

Forced cancellation of a CancellationToken

CancellationToken are used to signal that an asynchronous task should cancelled, at some given point in the code from here on a cancellation has been signaled and downstream. Methods downstream are methods and sub methods / sub routines. We can pass a cancellation token into for example Entity Framework Core to cancel a heavy database I/O process. Cancelling a method this way can be invoked on many different ways.

Examples of ways to cancel a cancellation tokens

1. By user interface actions. Like hitting a Cancel button in the UI. For example, the REST client Insomnia allows you to do this. 2. Other means of stopping a task. Inside a browser you can stop requests by reloading a window / tab. For example - If you use Swagger API in a browser, you can refresh the Swagger web page in a tab to indicate a cancellation is desired. The suggested way to programatically cancelling a cancellation token in code is to throw a OperationCanceledException in your code. If you have the Cancellation Token Source - the CTS - you can cancel the cancellation token as you like. Most often you do not have the CTS. You can always throw an OperationCanceledException. The source code can then listen to such a cancellation if you call ThrowIfCancellationIsRequested on the cancellation token. Another way to cancel a cancellation token is to create a linked cancellation token and cancellation the cancellation token source you created for it. This is an alternative way that directly updates the cancellation of a token to be cancelled downstream in case you have some logic further upstream that still should be called instead of directly throwing an OperationCancelledException. Is it a good approach to programatically create a new cancellation token and overwrite it or should you instead just throw an OperationCancelledException ? And why not just stick to the same object ? I am overwriting the token here using ref keyword, since CancellationToken is a struct object. This makes it harder to overwite, since structs are copied by value into methods, such as an extension method. However if this is a good idea or not - I include the code here for completeness.

Defining an extension method on cancellation tokens which can cancel them

CancellationTokenExtension.cs

namespace CarvedRock.Api
{
    public static class CancellationTokenExtensions
    {
        public static void ForceCancel(ref this CancellationToken cancellationToken,
        Func<bool>? condition = null)
        {
            if (condition == null || condition.Invoke())
            {
                var cts = CancellationTokenSource.CreateLinkedTokenSource(
                cancellationToken);
                cancellationToken = cts.Token;
                cts.Cancel();
            }
        }
    }
}

Here is some sample code that shows how we can use this extension method.

Using an extension method that is cancellation cancellation tokens

ProductController.cs

    [HttpGet]
    //[ResponseCache(Duration = 90, VaryByQueryKeys = new[] { "category" })]
    public async Task> Get(CancellationToken cancellationToken, string category = "all")
    {
        cancellationToken.ForceCancel(() => category == "kayak");
        using (_logger.BeginScope("ScopeCat: {ScopeCat}", category))
        {     
            _logger.LogInformation( "Getting products in API.");
            return await _productLogic.GetProductsForCategoryAsync(category, cancellationToken);
        }
    }

So there we have it, we can either use an approach like in this article, creating a temporary new cancellation token source and then created a linked cancellation token from the original cancellation token, overwriting it, and at the same time cancel it, possible by supplying a condition to decide if we want to cancel the cancellation token or not. Or we could just throw an OperationCanceledException. In the example code above I finally make the EF code communicating with the database and supply the cancelation token into a ToListAsync method here. This makes our code cancellable, in case we for example hit big data in the database that is slow and user wants to cancel.

Using an extension method that is cancellation cancellation tokens - downstream code making use of same cancellation token passed into sub method

CarvedRockRepository.cs

  public async Task<List<Product>> GetProductsAsync(string category, CancellationToken cancellationToken)
        { 

//.. Inside GetProductAsync method receiving token - more code above here inside the method

 var productsToSerialize = await _ctx.Products.Where(p => p.Category == category || category == "all")
  .Include(p => p.Rating).ToListAsync(cancellationToken);

// more code inside method below

}
  

Note ! Remember to pass down the cancellation token to your methods and sub methods /sub routines.