Snomed Ct Codes - Validation using Verhoeff algorithm

In this article, an algorithm for validating Snomed Ct codes using a control digit will be presented. The source code in this article is available in my Github repo here :
🕸️https://github.com/toreaurstadboss/SnomedCtTools

Snomed Ct stands form Systemized Nomenclature Of MEDicine - Clinical Terms. It is an international standard of terminology used in Electronic Health Records (EHR), also known as patient journals. It has also national extensions in addition to its shared 370,000 items. Norway for example has 16,000 national (local) items.

To validate Snomed Ct codes we use the Verhoeff algorithm.

The Verhoeff algorithm calculates a checksum for error detection. Snomed Ct codes use the algorithm for the checksum, which is the last digit in its code. It was first published in 1969 by Dutch matematician Jacobus Verhoeff. It was also discovered much later by German matematician Peter Gumm in 1985, which then included a formal proof
in addition. The theory is based upon Group theory. The algorithm detects all single digit errors (one digit wrong) or format errors, one digit inserted or left out or transpositions (interchanging two adjacent digits). The research paper is here:

https://www.researchgate.net/publication/3084126_A_new_class_of_check-digit_methods_for_arbitrary_number_systems_Corresp
An easier overview of the algorithm is exaplained on the Wikipedia article of Verhoeff algorithm: 📖 https://en.wikipedia.org/wiki/Verhoeff_algorithm First off we consider a multiplication table for the algorithm, D. It is also called the Cayley table, more description on that in the following Wikipedia page:
📗 https://en.wikipedia.org/wiki/Cayley_table. The Cayley table of the Dihedral group D5 is used in the Verhoeff algorithm.

VerhoeffValidator.cs - Multiplication matrix - Cayley table D of Dihedral (5 sided polygon) Group D5 (P

  private static readonly int[,] D =
  {
      { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 },
      { 1, 2, 3, 4, 0, 6, 7, 8, 9, 5 },
      { 2, 3, 4, 0, 1, 7, 8, 9, 5, 6 },
      { 3, 4, 0, 1, 2, 8, 9, 5, 6, 7 },
      { 4, 0, 1, 2, 3, 9, 5, 6, 7, 8 },
      { 5, 9, 8, 7, 6, 0, 4, 3, 2, 1 },
      { 6, 5, 9, 8, 7, 1, 0, 4, 3, 2 },
      { 7, 6, 5, 9, 8, 2, 1, 0, 4, 3 },
      { 8, 7, 6, 5, 9, 3, 2, 1, 0, 4 },
      { 9, 8, 7, 6, 5, 4, 3, 2, 1, 0 }
  };

The permutations table P is applying permutations for the numbers 0-9 iteratively with the permutation (1 5 8 9 4 2 7 0)(3 6) We end up with an iterative mapping of permutations for each row

1 -> 5 -> 8 -> 9
^              |
|              v
0 <- 7 <- 2 <- 4

3 <-> 6

VerhoeffValidator.cs - Permutations matrix P

  private static readonly int[,] P =
  {
      { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 },
      { 1, 5, 7, 6, 2, 8, 3, 0, 9, 4 },
      { 5, 8, 0, 3, 7, 9, 6, 1, 4, 2 },
      { 8, 9, 1, 6, 0, 4, 3, 5, 2, 7 },
      { 9, 4, 5, 3, 1, 2, 6, 8, 7, 0 },
      { 4, 2, 8, 6, 5, 7, 3, 9, 0, 1 },
      { 2, 7, 9, 3, 8, 0, 6, 4, 1, 5 },
      { 7, 0, 4, 6, 9, 1, 3, 2, 5, 8 }
  };

The Verhoeff validation check then becomes the following method, where we perform an algorithm that does the following calculation for the checksum c :

  /// <summary>
    /// Checks whether a number has a valid Verhoeff check digit.
    /// </summary>
    /// <param name="number">The numeric identifier, including its check digit.</param>
    /// <returns><see langword="true" /> when the number passes the Verhoeff check.</returns>
    public static bool IsValidVerhoeff(string code)
    {
        int[] digits = code
            .Select(character => int.Parse(character.ToString()))
            .Reverse()
            .ToArray();
        int checksum = 0;

        for (int index = 0; index < digits.Length; index++)
        {
            checksum = D[checksum, P[index % 8, digits[index]]];
        }

        return checksum == 0;
    }

Snomed Ct validator

Snomed Ct codes validates in addition that the code is numerical (only digits) and has a length between 6 to 18 digits. Our validator will accept whitespace and normalizes the code be removing whitespace.

using System.Text.RegularExpressions;

namespace SnomedCtTools.Lib.Validation;


public static class SnomedCtValidator
{

    /// <summary>
    /// Checks if a Snomed Ct Code is valid.
    /// It should be :
    /// - 6 to 18 digits
    /// - All numerics (this method washes away whitespace anyways)
    /// - Passes the Verhoeff's dihedral check, which is checked by this 
    /// method internally.
    /// </summary>
    /// <param name="code"></param>
    /// <returns></returns>
    public static bool IsValidSnomedCt(string code)
    {
        if (string.IsNullOrWhiteSpace(code))
        {
            return false;
        }
        //normalize whitespace to allow flexibility concerning whitespace
        string normalizedCode = Regex.Replace(code, @"\s+", "");
        if (!normalizedCode.All(c => char.IsDigit(c)))
        {
            return false;
        }

        if (normalizedCode.Length < 6 || normalizedCode.Length > 18)
        {
            return false;
        }

        bool isValid = VerhoeffValidator.IsValidVerhoeff(normalizedCode);
        return isValid;
    }

}

Some unit tests validates well known Snomed Ct codes and also checks that erroneous codes are flagged as not valid code. This is done by looking at the last digit, which is the checksum and using the Verhoeff algorithm and checking we are between 6-18 digits in length as noted. And also, we normalize and remove whitespace to offer some flexibility.

using FluentAssertions;
using NUnit.Framework;
using SnomedCtTools.Lib.Validation;

namespace SnomedCtTools.Tests.Validation
{

    [TestFixture]
    public class SnomedCtValidatorTests
    {

        [Test]
        [TestCase("44054006", true, Description = "diabetes type 2")]
        [TestCase("440540006", false, Description = "not a snomed ct code")]
        [TestCase("67148009", true, Description = "Podagra")]
        [TestCase("671480091", false, Description = "not a snomed ct code")]
        public void SnomedCtValidationCheckForWellknownDiseases(string code, bool expected)
        {
            SnomedCtValidator.IsValidSnomedCt(code).Should().Be(expected);
        }

        [Test]
        [TestCase("24700007", true)]
        [TestCase("2470", false)]
        [TestCase("2470000000000000000000000000007", false)]
        public void SnomedCtValidationChecksLengthCorrect(string code, bool expected)
        {
            SnomedCtValidator.IsValidSnomedCt(code).Should().Be(expected);
        }

        [Test]
        [TestCase("24700007", true)]
        [TestCase(" 24700007", true)]
        [TestCase("24700007 ", true)]
        [TestCase("2470 0007  ", true)]
        [TestCase("2  470 00  07  ", true)]
        public void SnomedCtValidationAcceptsWhiteSpace(string code, bool expected)
        {
            SnomedCtValidator.IsValidSnomedCt(code).Should().Be(expected);
        }

        [Test]
        [TestCase("24700007", true)]
        [TestCase("k24700007", false)]
        [TestCase("24700007#", false)]
        [TestCase("2470@k0007kk", false)]
        [TestCase("2kk3k#@!470###00kk07ll", false)]
        public void SnomedCtValidationCheckIlleagalChars(string code, bool expected)
        {
            SnomedCtValidator.IsValidSnomedCt(code).Should().Be(expected);
        }

    }

}

Start up nginx Docker container and edit homepage

This article will be a quick tip how to get a web server up and running on your system, in this case using Docker image nginx to spin up a container. First off, install Docker. To install docker:

• The Docker website contains the container software you need to quickly spin up nginx. You need an account there, you can use for example a Google account to log in and create the account
• You probably want to use for just experimentation the Personal plan, which is free. Go to https://docs.docker.com/get-started/get-docker/
• Install Docker for Desktop for your OS - Linux, Windows or Mac

To get nginx up and running, do the following commands from the command line, after Docker is installed and Docker for Desktop has been started. This will make sure the Docker background system processes are running (Docker deaemon).

docker command (CLI)
        │
        ▼
Docker daemon (dockerd)
        │
        ├─ Creates containers
        ├─ Pulls images
        ├─ Creates networks
        ├─ Manages volumes
        └─ Starts/stops containers
        
 

Links to Nginx and Docker hub Nginx image

https://nginx.org

The docker hub image for nginx is here, it has been downloaded a billion times plus.

https://hub.docker.com/_/nginx

Getting nginx running on your system

First off we start the nginx image (it will be pulled the first time and downloaded, then cached later on). It is run as a daemon background service with -d. nginx will run at port 8080 to avoid conflicting with other services running at port 80 such as IIS web server. The nginx project's homepage is here:

docker run -d --name nginx_test -p 8080:80 nginx

Here, we name the container as nginx_test The system prompts back the container id We can also see the containers running image with name nginx like this :

docker ps --filter ancestor=nginx

CONTAINER ID   IMAGE     COMMAND                  CREATED          STATUS          PORTS                                     NAMES
a2cc9ed9b212   nginx     "/docker-entrypoint.…"   12 minutes ago   Up 12 minutes   0.0.0.0:8080->80/tcp, [::]:8080->80/tcp   nginx_test
ea3bf3541852   nginx     "/docker-entrypoint.…"   12 hours ago     Up 12 hours     80/tcp                                    container-b
505eec83cca5   nginx     "/docker-entrypoint.…"   12 hours ago     Up 12 hours     80/tcp   

In this case, I tested some nginx containers some hours ago and they are still running. We can stop containers by running docker stop <container-id> Our nginx_test container is now running, we can enter the container next.

docker exec -it nginx_test bash

The Linux OS that is running inside the nginx container has no editor installed, not even vim og vi (it is optimalized to use minimal disk space so it got the bare minimum to run the web server nginx itself). Lets install nano to edit files inside the container!

apt-get update
apt-get install nano

The homepage of nginx is installed in /usr/share/nginx/html Edit the index.html file by running nano . nano index.html

cd /usr/share/nginx/html
nano index.html

You can save the index.html file after editing in nano by entering Ctrl+O and write to the file.

Additional notes

This is just for testing out nginx. To stop all containers on your system for example, you can either exit out of Docker for desktop or run this command (Powershell, other OS-es got similar command):

docker stop $(docker ps -q)

This will output the container ids of what was stopped. Afterwards, you can check that no containers are running using this:

docker ps

You should see an empty list of containers, i.e. no containers running on your system.

 ~#@❯ docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

Just to note, if you are to run nginx on a server you most definately want to enable a drive for your container, for durability. But for now, this article was just a short article how to get a web server such as nginx up and running on your system and play around with Docker. Windows Users - Please note Docker takes much resources, so to stop Docker in Windows and if you run the default setup of using WSL2 containers, this command should stop Docker. You can start up Docker again by just running Docker Desktop again.

wsl --shutdown
taskkill /F /IM "Docker Desktop.exe"

Fire and forget - Try catch and the importance of awaiting async methods

Async void methods are sometimes used in fire and forget scenarios and in event handlers in application code that uses frameworks such as Windows Forms or WPF. Consider the following C# code :

async Task Main()
{
	try
	{
		Operations.TidyCache("");		
		Console.WriteLine("Completed Successfully");
	}
	catch (Exception ex)
	{
		Console.Error.WriteLine($"Critical error: {ex.Message}"); //this try-catch will not be entered due to lack of awaiting Operations.TidyCache
	}	                                                                                                                                                                                                                                                                                                                                                                                                                         
}

public static class Operations {
	
	
	public static async void TidyCache(string key){
		
		ArgumentException.ThrowIfNullOrEmpty(key);
		await Task.Run(async() =>
		{
			await System.Threading.Tasks.Task.Delay(500);
			Console.WriteLine("Finished running async task : waited half a second above");
		});
		
		
	}
}

Output from code above :
Completed Successfully ArgumentException (...) The value cannot be an empty string. (Parameter 'key')


The problem with the code shown above is that although we do an initial error of passing in an empty string as the parameter key to method TidyCache, our try-catch will not be actually catched. The reason for this is that our async method is called and entered and the ArgumentException is thrown inside the async state machin and therefore thrown outside the try/catch. The end result is that since we do not await the method either, the caller will not get the chance to catch the exception and the exception is thrown directly on the thread pool and can also in some applications lead to application crash. We can move the try catch inside the TidyCache method and let the async method return Task instead of void. The calling code uses a discard assignment.

First solution

async Task Main()
{
	try
	{
		_ =Operations.TidyCache("");
	
		Console.WriteLine("Completed Successfully");
	}
	catch (Exception ex)
	{
		Console.Error.WriteLine($"Critical error: {ex.Message}"); //this try-catch will not be entered due to lack of awaiting Operations.TidyCache
	}
}

public static class Operations
{

	public static async Task TidyCache(string key)
	{
		try
		{
			ArgumentException.ThrowIfNullOrEmpty(key);
			await Task.Delay(500);
			Console.WriteLine("Finished running async task");
		}
		catch (Exception ex)
		{
			Console.Error.WriteLine($"TidyCache failed: {ex.Message}");
		}
	}

}

Output from the code above :
TidyCache failed: The value cannot be an empty string. (Parameter 'key') Completed Successfully

Second solution

The first solution is the correct pattern for fire and forget pattern. The second solution shown below is the right way to handle exception for async methods by allowing the async task to flow its exception up to the caller and actually await the async method.

async Task Main()
{
	try
	{
		await Operations.TidyCache("");
		Console.WriteLine("Completed Successfully");
	}
	catch (Exception ex)
	{
		Console.Error.WriteLine($"Critical error: {ex.Message}"); //this try-catch will not be entered due to lack of awaiting Operations.TidyCache
	}	                                                                                                                                                                                                                                                                                                                                                                                                                                                   
}

public static class Operations {	
	
	public static async Task TidyCache(string key){
		
		ArgumentException.ThrowIfNullOrEmpty(key);
		await Task.Run(async() =>
		{
			await System.Threading.Tasks.Task.Delay(500);
			Console.WriteLine("Finished running async task : waited half a second above");
		});			
	}
	
}

Output from code above :
Critical error: The value cannot be an empty string. (Parameter 'key')


The second solution shows the behavior and execution flow most programmers will expect from the code. The exception inside the async method that is thrown is catched in the calling code. Finally some (block) quotes about async void methods:

“async void is where exceptions go to die.”

“async void is the ghost of a Task — it haunts your runtime.”

“If you think you need async void, you probably need therapy.”

“async void is like fire-and-forget — except you forget the fire.”

“Use async void when you want exceptions to bypass your try/catch like a VIP.”