Using IronPython to execute Python code from .NET

Let's look at some code showing how to execute Python code from .NET using IronPython! IronPython provides support for Python scripts to run inside .NET and utilizes the Dynamic Language Runtime - DLR. The DLR together allows the caller to get dynamic typing and dynamic method dispatch, which is central in the dynamic languages such as Python. IronPython was first released in 2004, some 20 years ago. It has continued to evolve slowly and provides seamless integration into .NET ecosystem for Python developers. In this article, I will present some simple code that shows how you can run Python code inside a .NET 8 console application. We will load up some tuples in an array in some simple Python code, using IronPython. Tuples in Python Tuples in Python are immutable (such as in C#) and are defined using parentheses and comma-separated. This is the same as in C#, but Python had tuple support over 20 years before C#. We will have to add one Nuget package, the IronPython package, in a net8.0 application.

HelloIronPythonDemo1.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="IronPython" Version="3.4.1" />
  </ItemGroup>

  <ItemGroup>
    <None Update="customers.py">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
  </ItemGroup>

</Project>

Consider the following array of tuples in Python :

customers.py

customers = [
	('Jenna', 42, 165),
	('Thor', 40, 174),
	('Christopher', 18, 170),
	('Liz', 16, 168),
 ]

Python code is very compact and you declare variables without specifying type such as in C#, Python uses a simple way of creating variables and while C# got support in C# 7 in 2017, Python has had support for tuples since its early days. In the Python 1.4 version, we find it documented here:
https://docs.python.org/release/1.4/tut/node37.html#SECTION00630000000000000000.
Bear in mind, this is way back in 1996, C# was over 20 years later with its tuple support. If you install IronPython, you get a terminal where you can enter Python code (plus more functionality with .NET) such as shown below, where tuples are created and tuples may be composed or 'packed' and also 'unpacked', which is called deconstructed in .NET tuples.

To execute code to retrieve this array of tuples, first create a ScriptEngine and then create a ScriptScope, which we will use to retrieve the Python-declared variable customers. We create a ScriptSource, where we use the ScriptEngine to load up either a string or a file. A dynamic variable will be used to get the array of tuples and we can loop through this array with a foreach loop and output its content.

Program.cs

 
 using IronPython.Hosting;
 using Microsoft.Scripting.Hosting;
 using static System.Console;   
 IronPythonDemo1.OutputSomeExternallyLoadedTuples();

 public class IronPythonDemo1
 {

    public static void OutputSomeExternallyLoadedTuples()
    {
        var engine = Python.CreateEngine();
        ScriptScope scope = engine.CreateScope();

        //ScriptSource source = engine.CreateScriptSourceFromString(tupleStatement);
        ScriptSource source = engine.CreateScriptSourceFromFile("customers.py");
        source.Execute(scope);
        dynamic customers = scope.GetVariable("customers");
        foreach (var customer in customers)
        {
            Console.WriteLine($"(Name = {StringExtensions.FixedLength(customer[0], 20)}, Age = {StringExtensions.FixedLength(customer[1].ToString(), 8)}, Height={StringExtensions.FixedLength(customer[2].ToString(), 8)})");
        }
    }

}
 
 
 

Documentation for named tuples are available here: https://docs.python.org/3/library/collections.html#collections.namedtuple Here is sample coding showing script that although it is more verbose, shows more readability of which field is which for a named tuple. In an ordinary tuple, you use indexes to retrieve the nth field (0-based). But with named tuples, you use a field name instead.

 
from collections import namedtuple

Customer = namedtuple('Customer', ['Name', 'Age', 'Height'])
 
customers2 =  [
    Customer(Name = 'Jenna', Age = 42, Height = 165),
    Customer(Name = 'Thor', Age = 38, Height = 174),
    Customer(Name = 'Christopher', Age = 42, Height = 170),
    Customer(Name = 'Liz', Age = 42, Height = 168),
 ]
   
for cust in customers2:
    print(f"{cust.Name} with a height of {cust.Height}(cm)")

This outputs:

 
Jenna with a height of 165(cm)
Thor with a height of 174(cm)
Christopher with a height of 170(cm)
Liz with a height of 168(cm)
When your tuple gets many fields, having this readability should reduce bugs. Also, if you add more fields to your tuple, you do not have to fix up indexes in your script. So code is a bit more verbose, but it is also more open for change and readable.

The FixedLength extension method is a simple method to output text to a fixed width.

 
 
 public static class StringExtensions
 {

    public static string FixedLength(this string input, int length, char paddingchar = ' ')
    {
        if (string.IsNullOrWhiteSpace(input))
        {
            return input;
        }
        if (input.Length > length)
            return input.Substring(0, length);
        else
            return input.PadRight(length, paddingchar);
    }

 } 
 

Creating a data table from IEnumerable of T and defining column order explicitly in C#

This article shows code how you can create a DataTable from a collection of T (IEnumerable<T>) and defining explicitly the column order. An extension method for this looks like the following:

public static class DataTableExtensions
{


	public static DataTable CreateOrderedDataTable<T>(this IEnumerable<T> data)
	{
		var dataTable = new DataTable();
		var orderedProps = typeof(T).GetProperties(BindingFlags.Instance | BindingFlags.Public)
		 .OrderBy(prop => GetColumnOrder(prop)).ToList();
		
		foreach (var prop in orderedProps){
			dataTable.Columns.Add(prop.Name, Nullable.GetUnderlyingType(prop.PropertyType) ?? prop.PropertyType);
		}
		
		if (data != null)
		{
			dataTable.BeginLoadData();
			var enumerator = data.GetEnumerator();
			while (enumerator.MoveNext()){
			   var item = enumerator.Current;
			   var rowValues = new List<object>();
			   foreach (var prop in orderedProps){
			    rowValues.Add(prop.GetValue(item, null));		   	
			   }
			   dataTable.Rows.Add(rowValues.ToArray());			 			
			}
			dataTable.AcceptChanges();
		}
		return dataTable;
	}

	static int GetColumnOrder(PropertyInfo prop)
	{
		var displayAttribute = prop.GetCustomAttributes(typeof(DisplayAttribute), false).FirstOrDefault() as DisplayAttribute;
		int orderKey = displayAttribute?.Order ?? prop.MetadataToken;		
		return orderKey;
	}
	
}

We order first by DisplayAttribute and the Order value, and fallback to property's MetadataToken. This is an integer value that also returns the order the property was declared, in case you want to order just by the way properties are defined. We get the enumerator here and fetch the row one by one. We could use a simple foreach loop here too. Note the use of BeginLoadData and AcceptChanges. Consider the two classes next. One class does not set any explicit order, the other class uses the Display attribute's Order value to define a custom order of columns for the DataTable.

public class Car
{

	public int Id { get; set; }

	public string Make { get; set; }

	public string Model { get; set; }

	public string Color { get; set; }
}


public class CarV2
{
	[Display(Order = 4)]
	public int Id { get; set; }
	
	[Display(Order = 3)]
	public string Make { get; set; }
	
	[Display(Order = 2)]
	public string Model { get; set; }

	[Display(Order = 14)]
	public bool IsElectric { get; set; }

	[Display(Order = -188865)]
	public string Color { get; set; }
	
}

Next, the following little program in Linqpad tests this extension method and displays the datatables resulting with column ordering set.

void Main()
{
	var cars = new List<Car>{
		new Car { Id = 1, Make = "Audi", Model = "A5", Color = "Blue" },
		new Car { Id = 2, Make = "Volvo", Model = "XC60", Color = "Silver" },
		new Car { Id = 3, Make = "Golf", Model = "GTI", Color = "White" },
		new Car { Id = 4, Make = "Audi", Model = "A5", Color = "Blue" },
	};
	var dataTable = cars.CreateOrderedDataTable();
	dataTable.Dump("Cars datatable, data type is: Car");
	
	var carV2s = new List<CarV2>{
		new CarV2 { Id = 1, Make = "Audi", Model = "A5", Color = "Blue" },
		new CarV2 { Id = 2, Make = "Volvo", Model = "XC60", Color = "Silver" },
		new CarV2 { Id = 3, Make = "Golf", Model = "GTI", Color = "White" },
		new CarV2 { Id = 4, Make = "Audi", Model = "A5", Color = "Blue" },
	};	
	var dataTableV2 = carV2s.CreateOrderedDataTable();
	dataTableV2.Dump("Carsv2 datatable, datatype is CarV2");

}

Generating repeated data into variable in SQL Server in T-SQL

Let's see how we can create repeated data into variable of SQL Server in T-SQL. Use the REPLICATE function to create repeated data like this:

DECLARE @myVariable NVARCHAR(MAX)
SET @myVariable = REPLICATE('.', 10)
PRINT @myVariable
PRINT len(@myVariable)

In case you want to set the variable to data which is longer than 8000 characters, you must convert the argument to NVARCHAR(MAX).

DECLARE @myVariable NVARCHAR(MAX)
SET @myVariable = REPLICATE(CONVERT(NVARCHAR(MAX),'.'), 1024*1024*2)
PRINT len(@myVariable)

Creating random content is also easy in T-SQL:

DECLARE @myVariable NVARCHAR(MAX)
SET @myVariable = REPLICATE(CONVERT(NVARCHAR(MAX),REPLACE(NEWID(),'-', '')), 4)
PRINT len(@myVariable)
PRINT @myVariable

NEWID() creates a new guid, and we strip away the '-' letter, giving 32 chars which we replicate above four times. Since we were below 8000 chars, we chould have skipped using convert to nvarchar(max).