Monday 1 March 2021

Implementing ToDictionary in Typescript

In this article I will present some code I just did in my SimpleTsLinq library, which you can easily install using Npm. The library is here on Npmjs.com : The ToDictionary method looks like this:

  
  if (!Array.prototype.ToDictionary) {
  Array.prototype.ToDictionary = function <T>(keySelector: (arg: T) => any): any {
    let hash = {};
    this.map(item => {
      let key = keySelector(item);
      if (!(key in hash)) {
        hash[key] = item;
      }
      else {
        if (!(Array.isArray(hash[key]))) {
          hash[key] = [hash[key]];
        }
        hash[key].push(item);
      }
    });
    return hash;
  }
}
  

Here is a unit test (spec) for this method :

  
    it('can apply method ToDictionary on an array, allowing specificaton of a key selector for the dictionary object', () => {
    let heroes = [{ name: "Han Solo", age: 47, gender: "M" }, { name: "Leia", age: 29, gender: "F" }, { name: "Luke", age: 24, gender: "M" }, { name: "Lando", age: 47, gender: "M" }];
    let dictionaryOfHeroes = heroes.ToDictionary<Hero>(x => x.gender);

    let expectedDictionary = {
      "F": {
        name: "Leia", age: 29, gender: "F"
      },
      "M": [
        { name: "Han Solo", age: 47, gender: "M" },
        { name: "Luke", age: 24, gender: "M" },
        { name: "Lando", age: 47, gender: "M" }
      ]
    };
    expect(dictionaryOfHeroes).toEqual(expectedDictionary);
  });
  
  

You can also test out this library using Npm RunKit here: We can make a dictionary with different keys, image example:

Friday 5 February 2021

Overcoming limitations in Contains in Entity Framework

Entity Framework will hit a performance penalty bottleneck or crash if Contains contains a too large of a list. Here is how you can avoid this, using Marc Gravell's excellent approach in this. I am including some tests of this. I also suggest you consider LinqKit to use expandable queries to make this all work. First off, this class contains the extension methods for Entity Framework for this:
 
 
 public class EntityExtensions {
 
 /// <summary>
        /// This method overcomes a weakness with Entity Framework with Contains where you can partition the values to look for into 
        /// blocks or partitions, it is modeled after Marc Gravell's answer here:
        /// https://stackoverflow.com/a/568771/741368
        /// Entity Framework hits a limit of 2100 parameter limit in the DB but probably comes into trouble before this limit as even
        /// queries with several 100 parameters are slow.
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <typeparam name="TValue"></typeparam>
        /// <param name="source">Source, for example DbSet (table)</param>
        /// <param name="selector">Selector, key selector</param>
        /// <param name="blockSize">Size of blocks (chunks/partitions)</param>
        /// <param name="values">Values as parameters</param>
        /// 
        /// <example>
        ///   /// <[!CDATA[
        /// /// The following EF query will hit a performance penalty or time out if EF gets a too large list of operationids:
        /// ///
        /// /// var patients = context.Patients.Where(p => operationsIds.Contains(p.OperationId)).Select(p => new {
        /// ///  p.OperationId,
        /// ///  p.
        /// /// });
        /// ///
        /// /// 
        /// /// var patients = context.Patients.AsExpandable().InRange(p => p.OperationId, 1000, operationIds)
        /// //.Select(p => new
        /// //{
        /// //    p.OperationId,
        /// //    p.IsDaytimeSurgery
        /// //}).ToList();
        /// //]]
        /// </example>
        /// <returns></returns>
        public static IEnumerable<T> InRange<T, TValue>(
                this IQueryable<T> source,
                Expression<Func<T, TValue>> selector,
                int blockSize,
                IEnumerable<TValue> values)
        {
            MethodInfo method = null;
          
            foreach (MethodInfo tmp in typeof(Enumerable).GetMethods(
                    BindingFlags.Public | BindingFlags.Static))
            {
                if (tmp.Name == "Contains" && tmp.IsGenericMethodDefinition
                        && tmp.GetParameters().Length == 2)
                {
                    method = tmp.MakeGenericMethod(typeof(TValue));
                    break;
                }
            }

            if (method == null) throw new InvalidOperationException(
                   "Unable to locate Contains");
            foreach (TValue[] block in values.GetBlocks(blockSize))
            {
                var row = Expression.Parameter(typeof(T), "row");
                var member = Expression.Invoke(selector, row);
                var keys = Expression.Constant(block, typeof(TValue[]));
                var predicate = Expression.Call(method, keys, member);
                var lambda = Expression.Lambda<Func<T, bool>>(
                      predicate, row);
                foreach (T record in source.Where(lambda))
                {
                    yield return record;
                }
            }

        }

        /// <summary>
        /// Similar to Chunk, it partitions the IEnumerable source and returns the chunks or blocks by given blocksize. The last block can have variable length
        /// between 0 to blocksize since the IEnumerable can have of course variable size not evenly divided by blocksize. 
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="source"></param>
        /// <param name="blockSize"></param>
        /// <returns></returns>
        public static IEnumerable<T[]> GetBlocks<T>(
                this IEnumerable<T> source, int blockSize)
        {
            List<T> list = new List<T>(blockSize);
            foreach (T item in source)
            {
                list.Add(item);
                if (list.Count == blockSize)
                {
                    yield return list.ToArray();
                    list.Clear();
                }
            }
            if (list.Count > 0)
            {
                yield return list.ToArray();
            }
        }
        
  }

 
 
 
Linqkit allows us to rewrite queries for EF using expression trees. One class is ExpandableQuery. See the links here for further info about Linqkit and Linq-Expand.
 
  	/// <summary>Refer to http://www.albahari.com/nutshell/linqkit.html and
	/// http://tomasp.net/blog/linq-expand.aspx for more information.</summary>
	public static class Extensions
	{
		public static IQueryable<T> AsExpandable<T> (this IQueryable<T> query)
		{
			if (query is ExpandableQuery<T>) return (ExpandableQuery<T>)query;
			return new ExpandableQuery<T> (query);
		}
 
This all seems to look a bit cryptic, so lets see an integration test of mine instead:
 


        [Test]
        [Category(TestCategories.IntegrationTest)]
        public void GetDataChunkedDoesNotFail()
        {
            using (var context = DbContextManager.ScopedOpPlanDataContext)
            {
                int[] operationalUnitIds = new int[]{ 107455, 105431, 107646, 107846 };
                var reportItems = context.OperationalUnits.AsExpandable().InRange(ou => ou.FreshOrganizationalUnitId, 2, operationalUnitIds).ToList();
                Assert.IsNotNull(reportItems);           
                CollectionAssert.IsNotEmpty(reportItems); 
            }
        }

 
 
 
This shows how to use the InRange method of Marc Gravell. We use the AsExpandable method to allow us to hack into the expression tree of Entity Framework and the InRange method allows us to partition the work for EF. We do not know the siz of operational unit ids (usually it is low and another entity - operation Ids is of variable length and will in production blow up since we in some cases surpass the 2100 limit of Contains). And as I said before, Entity Framework will hit a performance bottleneck before 2100 parameteters are sent into the Contains method. This way of fixing it up will allow you to get stable running code in production again against large data and variable length. This code is tested with Entity Framework 6.2.0. Another article considers performance considerations for Contains and different approaches here: https://www.toptal.com/dot-net/entity-framework-performance-using-contains IMHO this approach has proven stable in a production environment for several years with large data and can be considered a stable workaround for EF slow Contains performance. I have made the LinqKit fork LinqKit.AsyncSupport available on Nuget here now: https://www.nuget.org/packages/ToreAurstadIt.LinqKit.AsyncSupport/1.1.0 This makes it possible to perform Async calls and expandable queries, i.e. queries with inline method calls for example. The nuget package now also sports symbol package for easier debugging experience. The source code for LinqKit.AsyncSupport is available here: https://github.com/toreaurstadboss/LinqKit.AsyncSupport

Thursday 31 December 2020

Simple property grid in Blazor

This artile will present a simple property grid in Blazor I have made. The component relies on standard stuff like Bootstrap, jQuery, Twitter Bootstrap and Font Awesome. But the repo url shown here links to the Github repo of mine which can be easily forked if you want to add features (such as editing capabilities). The component already supports nested levels, so if the object you inspect has a hierarchical structure, this is shown in this Blazor component. Having a component to inspect objects in Blazor is great as Blazor lacks inspect tools (since the app is compiled into a web assembly, we cannot easily inspect state of objects in the app other than the DOM and Javascript objects. With this component we can get basic inspection support to inspect state of the object in the app you desire to inspect). The Github repo contains also a bundled application which uses the component and shows a sample use-case (also shown in Gif video below). I have tested the component with three levels of depth for a sample object (included in the repo). The component is available here on my Github repo:
 
 git clone https://github.com/toreaurstadboss/BlazorPropertyGrid/tree/main/BlazorPropertyGrid
 
https://github.com/toreaurstadboss/BlazorPropertyGrid/tree/main/BlazorPropertyGrid
The component consists of two components where one of them is used in a recursive manner to support nested object structure. The top level component got this code-behind class.
PropertyGridComponentBase.cs
using System.Collections.Generic; using System.Reflection; using Microsoft.AspNetCore.Components; using Microsoft.AspNetCore.Components.Web; using Microsoft.JSInterop; namespace BlazorPropertyGridComponents.Components { public class PropertyGridComponentBase : ComponentBase { [Inject] public IJSRuntime JsRuntime { get; set; } [Parameter] public object DataContext { get; set; } public Dictionary<string, PropertyInfoAtLevelNodeComponent> Props { get; set; } public PropertyGridComponentBase() { Props = new Dictionary<string, PropertyInfoAtLevelNodeComponent>(); } protected override void OnParametersSet() { Props.Clear(); if (DataContext == null) return; Props["ROOT"] = MapPropertiesOfDataContext(string.Empty, DataContext, null); StateHasChanged(); } private bool IsNestedProperty(PropertyInfo pi) => pi.PropertyType.IsClass && pi.PropertyType.Namespace != "System"; private PropertyInfoAtLevelNodeComponent MapPropertiesOfDataContext(string propertyPath, object parentObject, PropertyInfo currentProp) { if (parentObject == null) return null; var publicProperties = parentObject.GetType() .GetProperties(BindingFlags.Public | BindingFlags.Instance); var propertyNode = new PropertyInfoAtLevelNodeComponent { PropertyName = currentProp?.Name ?? "ROOT", PropertyValue = parentObject, PropertyType = parentObject.GetType(), FullPropertyPath = TrimFullPropertyPath($"{propertyPath}.{currentProp?.Name}") ?? "ROOT", IsClass = parentObject.GetType().IsClass && parentObject.GetType().Namespace != "System" }; foreach (var p in publicProperties) { var propertyValue = p.GetValue(parentObject, null); if (!IsNestedProperty(p)) { propertyNode.SubProperties.Add(p.Name, new PropertyInfoAtLevelNodeComponent { IsClass = false, FullPropertyPath = TrimFullPropertyPath($"{propertyPath}.{p.Name}"), PropertyName = p.Name, PropertyValue = propertyValue, PropertyType = p.PropertyType //note - SubProperties are default empty if not nested property of course. } ); } else { //we need to add the sub property but recurse also call to fetch the nested properties propertyNode.SubProperties.Add(p.Name, new PropertyInfoAtLevelNodeComponent { IsClass = true, FullPropertyPath = propertyPath + p.Name, PropertyName = p.Name, PropertyValue = MapPropertiesOfDataContext(TrimFullPropertyPath($"{propertyPath}.{p.Name}"), propertyValue, p), PropertyType = p.PropertyType //note - SubProperties are default empty if not nested property of course. } ); } } return propertyNode; } protected void toggleExpandButton(MouseEventArgs e, string buttonId) { JsRuntime.InvokeVoidAsync("toggleExpandButton", buttonId); } private string TrimFullPropertyPath(string fullpropertypath) { if (string.IsNullOrEmpty(fullpropertypath)) return fullpropertypath; return fullpropertypath.TrimStart('.').TrimEnd('.'); } } }
And its razor file looks like this:
PropertyGridComponentBase.razor
@inherits PropertyGridComponentBase @using BlazorPropertyGridComponents.Components <table class="table table-striped col-md-4 col-lg-3 col-sm-6"> <thead> <tr> <th scope="col">Property</th> <th scope="col">Value</th> </tr> </thead> <tbody> @foreach (KeyValuePair<string, PropertyInfoAtLevelNodeComponent> prop in Props) { @if (!prop.Value.IsClass) { @* <tr> <td>@prop.Key</td> <td>@prop.Value</td> </tr>*@ } else { var currentNestedDiv = "currentDiv_" + prop.Key; var currentProp = prop.Value.PropertyValue; //must be a nested class property <tr> <td colspan="2"> <button type="button" id="@prop.Key" class="btn btn-info fas fa-minus" @onclick="(e) => toggleExpandButton(e,prop.Key)" data-toggle="collapse" data-target="#@currentNestedDiv"> </button> <div id="@currentNestedDiv" class="collapse show"> <PropertyRowComponent Depth="1" PropertyInfoAtLevel="@prop.Value" /> </div> </td> </tr> } } </tbody> </table> @code { }
We also have this helper class to model each property in the nested structure:
PropertyInfoAtLevelNodeComponent.cs
using System; using System.Collections.Generic; namespace BlazorPropertyGridComponents.Components { /// <summary> /// Node class for hierarchical structure of property info for an object of given object graph structure. /// </summary> public class PropertyInfoAtLevelNodeComponent { public PropertyInfoAtLevelNodeComponent() { SubProperties = new Dictionary<string, PropertyInfoAtLevelNodeComponent>(); } public string PropertyName { get; set; } public object PropertyValue { get; set; } public Type PropertyType { get; set; } public Dictionary<string, PropertyInfoAtLevelNodeComponent> SubProperties { get; private set; } public string FullPropertyPath { get; set; } public bool IsClass { get; set; } } }
Our lower component used by the top component code-behind looks like this:
PropertyRowComponentBase.cs
using System.Collections.Generic; using Microsoft.AspNetCore.Components; using Microsoft.AspNetCore.Components.Web; using Microsoft.JSInterop; namespace BlazorPropertyGridComponents.Components { public class PropertyRowComponentBase : ComponentBase { public PropertyRowComponentBase() { DisplayedFullPropertyPaths = new List<string>(); } [Parameter] public PropertyInfoAtLevelNodeComponent PropertyInfoAtLevel { get; set; } [Parameter] public int Depth { get; set; } [Parameter] public List<string> DisplayedFullPropertyPaths { get; set; } [Inject] protected IJSRuntime JsRunTime { get; set; } protected void toggleExpandButton(MouseEventArgs e, string buttonId) { JsRunTime.InvokeVoidAsync("toggleExpandButton", buttonId); } } }
The razor file looks like this:
PropertyRowComponent.razor
@using BlazorPropertyGridComponents.Components @inherits PropertyRowComponentBase @foreach (var item in PropertyInfoAtLevel.SubProperties.Keys) { var propertyInfoAtLevel = PropertyInfoAtLevel.SubProperties[item]; if (propertyInfoAtLevel != null) { @* if (DisplayedFullPropertyPaths.Contains(propertyInfoAtLevel.FullPropertyPath)){ continue; //the property is already displayed. }*@ DisplayedFullPropertyPaths.Add(propertyInfoAtLevel.FullPropertyPath); @* <span class="text-white bg-dark">@propertyInfoAtLevel.FullPropertyPath</span>*@ @* <em> @propertyInfoAtLevel </em>*@ } if (!propertyInfoAtLevel.PropertyType.IsClass || propertyInfoAtLevel.PropertyType.Namespace.StartsWith("System")) { <tr> <td> <span title="@propertyInfoAtLevel.FullPropertyPath" class="font-weight-bold">@propertyInfoAtLevel.PropertyName</span> </td> <td> <span>@propertyInfoAtLevel.PropertyValue</span> </td> </tr> } else if (propertyInfoAtLevel.PropertyValue != null && propertyInfoAtLevel.PropertyValue is PropertyInfoAtLevelNodeComponent) { var nestedLevel = (PropertyInfoAtLevelNodeComponent)propertyInfoAtLevel.PropertyValue; var collapseOrNotCssClass = Depth == 0 ? "collapse show" : "collapse"; var curDepth = Depth + 1; collapseOrNotCssClass += " depth" + Depth; var currentNestedDiv = "collapsingdiv_" + propertyInfoAtLevel.PropertyName; //must be a nested class property <tr> <td colspan="2"> <span>@propertyInfoAtLevel.PropertyName</span> <button id="@propertyInfoAtLevel.FullPropertyPath" type="button" @onclick="(e) => toggleExpandButton(e,propertyInfoAtLevel.FullPropertyPath)" class="fas btn btn-info fa-plus" data-toggle="collapse" data-target="#@currentNestedDiv"></button> <div id="@currentNestedDiv" class="@collapseOrNotCssClass"> <PropertyRowComponent PropertyInfoAtLevel="@nestedLevel" Depth="@curDepth" /> </div> </td> </tr> } } @code { }