Sunday, 8 May 2016

RSA algorithm demo in MonoDevelop and GtkSharp

This article will present a demo of using RSA in Monodevelop using GtkSharp UI framework. As you know, the Mono project offers an implementation of .NET framework, such as BCL, CLR, MSIL and so on - and also the classes in System.Security.Cryptography! So let us delve into the details of doing some RSA crypto! First off, the GUI looks like this:


In MonoDevelop we use the Stetic GUI Designer to build the GUI!





Cool! We can build apps that runs on Linux and Windows with Monodevelop! Now over to the code of this app!




using System;
using Gtk;
using System.Text;
using System.Security.Cryptography; 
using System.Security; 
using System.IO; 

public partial class MainWindow: Gtk.Window
{

 private RSACryptoServiceProvider _rsa;
 private RSAParameters _rsaPrivateKey;
 private RSAParameters _rsaPublicKey;
 private byte[] _cipherBytes; 
 private byte[] _decipherBytes; 
 private byte[] _plainTextBytes;

 public MainWindow () : base ("Pango")
 {
  Application.Init ();
  Build ();
  SetupControls ();
  Application.Run ();
 }

 private void SetupControls(){
  Gdk.Color color = new Gdk.Color (255, 30, 80);
     lblP.ModifyFont (Pango.FontDescription.FromString ("Purisa 10")); 
  //lblP.ModifyBg (StateType.Normal, new Gdk.Color (255, 80, 10));
 }

 protected void OnDeleteEvent (object sender, DeleteEventArgs a)
 {
  Application.Quit ();
  a.RetVal = true;
 }

 protected void btnRsaSetupClick (object sender, EventArgs e)
 {
  _rsa = new RSACryptoServiceProvider ();
 
  StringWriter writer = new StringWriter (); 
  string rsaSetupXml = _rsa.ToXmlString (true);
  writer.Write (rsaSetupXml); 
  //tbRsaSetup.Buffer.Text = writer.ToString ();
  writer.Close ();

  _rsaPrivateKey = _rsa.ExportParameters (true);
  _rsaPublicKey = _rsa.ExportParameters (false); 

  SetupControls ();
  DisplayRsaSetup (_rsaPrivateKey);
 } 

 private void DisplayRsaSetup (RSAParameters rsaParams){
  try {
   lblPValue.Text = Convert.ToBase64String (rsaParams.P);
   lblQValue.Text = Convert.ToBase64String (rsaParams.Q);
   lblModulusValue.Text = Convert.ToBase64String (rsaParams.Modulus);
   lblDValue.Text = Convert.ToBase64String(rsaParams.D);
   lblEValue.Text = Convert.ToBase64String (rsaParams.Exponent);

  } catch (Exception err) {
   Console.WriteLine (err.Message);
  }

 }

 protected void btnEncryptClicked (object sender, EventArgs e)
 {
  if (_rsa == null)
   return;
  _plainTextBytes = Encoding.UTF8.GetBytes (textViewPlainText.Buffer.Text);
  _cipherBytes = _rsa.Encrypt (_plainTextBytes, false);
  textviewEncrypted.Buffer.Text = Convert.ToBase64String(_cipherBytes);
 }

 protected void btnDecryptClicked (object sender, EventArgs e)
 {
  textviewDecrypted.Buffer.Text = string.Empty; 

  if (_rsa == null)
   return;
  if (_cipherBytes == null)
   return; 
  _decipherBytes = _rsa.Decrypt (_cipherBytes, false); 

  textviewDecrypted.Buffer.Text = Encoding.UTF8.GetString(_decipherBytes); 
 }

}





As you can see in the code, we instantiate a new RSACryptoServiceProvider instance. We use the Encrypt and Decrypt method, using the second argument set to false to not use the OAEP padding, that is the Optimal Assymetric Encryption Padding for compability. Setting false here for padding will use the PKCS# instead. PKCS stands for Public Key Cryptography Standards. I have tested also with the parameters set to true i OAEP, and it seems to work nice also with Monodevelop - so you could use both types of padding. Note that we use the ExportParameters methods of the RSACryptoServiceProvider to the the RSAParameters object. In assymetric encryption, we must guard our private key and expose our public key. This is a comprehensive demo of the RSA algorithm. We would use the ExportParameters method with the parameters set to false to not include the private key. To export the RSA parameters with more compability, you can export the parameters as XML. You can use the ToXmlString() method to export the XML as a string. You can either export the RSA parameters as a string or to a file, and you can then use the method FromXmlString() to import the RSA parameters.

 {
  _rsa = new RSACryptoServiceProvider ();
 
  StringWriter writer = new StringWriter (); 
  string rsaSetupXml = _rsa.ToXmlString (true);
  writer.Write (rsaSetupXml); 
  //tbRsaSetup.Buffer.Text = writer.ToString ();
  writer.Close ();

As you can see in the code above, you can use a StringWriter to write to a string, but you can also use a FileStream to write the contents out to a file. Using the ToXmlString - you will export the information needed for a public key by setting the argument of this method to FALSE. To include private key information, you would provide the value TRUE here. In the RSA algorithm the following is belonging to the "PUBLIC Domain":
  • Modulus
  • Public exponent E
The "PRIVATE Domain" contains the additional information:
  • Private exponent D
  • Prime P
  • Prime Q
Private domain will also reveal the values DQ, DP, InverseQ that is given by this extra information. The security of the RSA algorithm relies on the toughness of prime factorization of large prime numbers. RSA will use large numbers and the public key only contains the modulus (product) of the prime numbers and a public exponent E that the sender will use this information as a public key to encrypt the information. The receiver, which knows the private key can then decrypt the information with this extra information. So the key note here is to guard your private key and share your public key! And that you can do RSA encryption when making applications for Linux of course, with Monodevelop! The .NET Framework is already there for you to use and it is very updated. To work with this sample, a download link is shown below. Bunzip the file using the command: tar xjvf RsaDemo.tar.bz2 Monodevelop project with RsaDemo

tar xjvf Symmetric.tar.bz2 

Just so you know:
tar - Tape ARchiver
And the options:
x - extract
v - verbose output (lists all files as they are extracted)
j - deal with bzipped file
f - read from a file, rather than a tape device

"tar --help" will give you more options and info

After unpacking, just open the solution in MonoDevelop.

So .NET Developers - Start your engines - Start developing for Linux!

Friday, 6 May 2016

Symmetric crypto algorithms in C# with MonoDevelop and GTK-Sharp

Using MonoDevelop and GTK-Sharp (GTK#) offers a .NET developer to develop applications for other platforms such as applications in Linux and other OS-es. Let's look more at the very able MonoDeveloper IDE and GTK-Sharp. As a .NET developer who has written .NET applications for many years, MonoDevelop has fully matured into a very good platform to develop a multitiude of applications. GTK# resembles .NET Windows Forms in many ways. We will here use some symmetric crypto algorithms in .NET that is available with Mono framework. The demo provides Digital Encryption Standard (DES) and Triple-DES, plus the Advanced Encryption Standard (AES) - Rijndael. The GUI will look like this:

The GUI is designed with the GUI designer Stetic in Monodevelop, for developing GTK#-applications. We can choose the Mode of the cryptographic algorithm, default here is Cipher Block Chaining. We can also set the padding of the cryptographic algorithm. Note that not all combinations are legal. I have tested with Rijndael, Cipher Block Chaining and padding set to Zeros, which seems to be working ok. You can use the demo here to test out other combinations. You can also generate different Initialization Vectors and Keys to use with the algorithm.

The code to achieve the encryption and decryption is listed below:

using System;
using System.Security.Cryptography;
using Gtk;
using System.IO;

public partial class MainWindow: Gtk.Window
{

 private SymmetricAlgorithm _symmetricAlgorithm; 
 private byte[] _intializationVector; 
 private byte[] _key;
 private byte[] _cipherBytes;



 public MainWindow () : base (Gtk.WindowType.Toplevel)
 {
  Build ();
 }

 protected void OnDeleteEvent (object sender, DeleteEventArgs a)
 {
  Application.Quit ();
  a.RetVal = true;
 }

 protected void btnGenerateIVClick (object sender, EventArgs e)
 {
     _symmetricAlgorithm = CreateSymmetricAlgorithm (); 

  _symmetricAlgorithm.GenerateIV ();
  _intializationVector = _symmetricAlgorithm.IV;

  lbInitializationVector.Text = Convert.ToBase64String(_symmetricAlgorithm.IV);

//  MessageDialog msgBox = new MessageDialog (null, DialogFlags.Modal, 
//                      MessageType.Info, ButtonsType.Ok, "Why hello world!");
//  msgBox.Run ();
//  msgBox.Destroy ();
 }

 protected void btnEncrypt_Click (object sender, EventArgs e)
 {
  _symmetricAlgorithm = CreateSymmetricAlgorithm (); //ensure that we use the selected algorithm
  _cipherBytes = Encrypt(textviewPlainText.Buffer.Text);
  textviewCipher.Buffer.Text = Convert.ToBase64String(_cipherBytes); 
 }

 private byte[] Encrypt(string text){
  byte[] encrypted;
  ICryptoTransform encryptor = _symmetricAlgorithm.CreateEncryptor (_key, _intializationVector);
  using (MemoryStream msEncrypt = new MemoryStream ()) {
   using (CryptoStream csEncrypt = new CryptoStream (msEncrypt, encryptor, CryptoStreamMode.Write)) {
    using (StreamWriter swWriter = new StreamWriter (csEncrypt)) {
     swWriter.Write (text);
    }
    encrypted = msEncrypt.ToArray (); 
   }

  }
  return encrypted;
 }

 private string Decrypt(byte[] cipherBytes){
  try {
   ICryptoTransform decryptor = _symmetricAlgorithm.CreateDecryptor (_key, 
    _intializationVector);
   using (MemoryStream msEncrypt = new MemoryStream (cipherBytes)) {
    using (CryptoStream csEncrypt = new CryptoStream (msEncrypt, decryptor,
     CryptoStreamMode.Read)) {
     using (StreamReader sReader = new StreamReader (csEncrypt)) {
      string decrypted = sReader.ReadToEnd();
      return decrypted;
     }
    } 
   }
  } catch (Exception err) {
   Console.WriteLine (err.Message);
  }
  return string.Empty;
 }

 private SymmetricAlgorithm CreateSymmetricAlgorithm(){
  SymmetricAlgorithm sa = null;
  if (rbDES.Active)
   sa = DESCryptoServiceProvider.Create ();
  if (rbThreeDES.Active)
   sa = TripleDESCryptoServiceProvider.Create ();
  if (rbRijndael.Active)
   sa = RijndaelManaged.Create ();

  if (sa == null)
   sa = DESCryptoServiceProvider.Create (); 

  if (_intializationVector != null)
   sa.IV = _intializationVector;
  if (_key != null)
   sa.Key = _key;

  sa.Mode = GetCipherMode ();
  sa.Padding = GetPadding ();
  return sa;
 }

 private PaddingMode GetPadding(){
  if (rbPaddingNone.Active)
   return PaddingMode.None;
  if (rbPaddingZeros.Active)
   return PaddingMode.PKCS7;
  if (rbPaddingAnsiX923.Active)
   return PaddingMode.ANSIX923;
  if (rbPaddingISO1126.Active)
   return PaddingMode.ISO10126;
  return PaddingMode.Zeros;
 }

 private CipherMode GetCipherMode(){
  if (rbCBC.Active)
   return CipherMode.CBC;
  if (rbCFB.Active)
   return CipherMode.CFB;
  if (rbCTS.Active)
   return CipherMode.CTS;
  if (rbECB.Active)
   return CipherMode.ECB;
  if (rbOFB.Active)
   return CipherMode.OFB;

  return CipherMode.CBC;
 }

 protected void btnKeyClick (object sender, EventArgs e)
 {
  _symmetricAlgorithm = CreateSymmetricAlgorithm ();     

  _symmetricAlgorithm.GenerateKey ();  
  _key = _symmetricAlgorithm.Key;
  lblKey.Text = Convert.ToBase64String (_symmetricAlgorithm.Key);
 }

 protected void btnDecryptClick (object sender, EventArgs e)
 {
  _symmetricAlgorithm = CreateSymmetricAlgorithm (); //ensure that we use the selected algorithm
  string decrypted = Decrypt(Convert.FromBase64String(textviewCipher.Buffer.Text));
  textviewDecrypted.Buffer.Text = decrypted; 
 }
}











To open up this sample, I have uploaded the MonoDevelop project as a tar.bz2 file available for download here: Sample project Monodevelop in this article To unzip the tar bunzip2 file, just the following command:
tar xjvf Symmetric.tar.bz2 

Just so you know:
tar - Tape ARchiver
And the options:
x - extract
v - verbose output (lists all files as they are extracted)
j - deal with bzipped file
f - read from a file, rather than a tape device

"tar --help" will give you more options and info

After unpacking, just open the solution in MonoDevelop.





Wednesday, 4 May 2016

Creating a simple MD5 application using GTK# and Monodevelop

Let's look at building a simple application using GTK# and Monodevelop! I created this application using a Ubuntu 16.04 Xenial AMD64 Distribution of Linux running inside an Oracle VM VirtualBox on my Windows 10 Machine! First off, this article will show a very simple application written in Monodevelop IDE using GTK# to build a GUI. It resembles somewhat Windows Forms if you come from a Visual Studio background, such as I do. The application just takes some text input (plaintext) and computes a MD5 hash. Simple stuff. Defining the following form in MainWindow of the GTK# application: Moving over to the code bit, I define the following in MainWindow (Source of the form), which is the code behind:

using System;
using System.Linq;
using Gtk;
using System.Security.Cryptography;
using System.Text;


public partial class MainWindow: Gtk.Window
{
 public MainWindow () : base (Gtk.WindowType.Toplevel)
 {
  Build ();
  btnMd5.Clicked += OnBtnClick;
 }

 protected void OnDeleteEvent (object sender, DeleteEventArgs a)
 {
  Application.Quit ();
  a.RetVal = true;
 }

 protected void OnBtnClick (object sender, EventArgs args)
 {
  var md5 = MD5CryptoServiceProvider.Create ();
  byte[] plainTextBytes = Encoding.UTF8.GetBytes (tbPlainText.Buffer.Text); 
  byte[] hashBytes = md5.ComputeHash (plainTextBytes); 
  var sbuilder = new StringBuilder (); 

  sbuilder.Append(string.Join("",
  hashBytes.Select(x => x.ToString("x2")))); 
  tbHash.Buffer.Text = sbuilder.ToString();
 }

}



The code above instantiates a MD5CryptoServiceProvder instance, then computes a hash. We get a string representation of the MD5 hash using a StringBuilder and we use ToString("x2") - which assembles a hexidecimal string for us, which is the common way to represent a MD5 hash. A MD5 hash produces 128 bits = 16 bytes = 32 hexidecimal digits. A hexadecimal value can be 0-9 and A-F = 16 different values = one half byte.

We build up our GUI using the GUI designer inside Monodevelop. The GUI designer for GTK# in Monodevelop is called Stetic.
Stetic

Friday, 22 April 2016

Transfering a commit message from Git to Target Process

It is possible to transfer a commit message from Git to Target Process using the REST Api of TP. This is done using hooks in Git. The problem is that the hook is written using bash shell scripts (or perl) and unlike Mercurial - Git does not sport an obvious choice of tool to develop such hooks. I am going to present below a hook I wrote in bash shell to transmit your commit message to TP. I choose to do this in the post-commit hook. First off, go to the source repository you are working with and into the .git subfolder. Then go into the hooks folder. Now add a new file called: post-commit Ok, so now we add our hook:

#!/bin/sh
#
# Overfører kommentar fra Git til Target Process
# Bruk - Formater kommentaren som: 
# TP TASKID: Min innsjekkingskommentar her
#
# Dette vil overføre så kommentarer til TP 
# Merk: 
# Erstatt verdiene i scriptet som heter TP_LOGON og TP_PASSWORD
# med ditt pålogging til TP. Vil du ikke inkludere passordet ditt til TP kan du 
# ta bort passordet som argument til curl kommandoen 
#
# Merk at du må installere Cygwin først og installere curl og curl-lib. Nano anbefales som editor
# Cygwin - https://www.cygwin.com 
 
 
NAME=$(git branch | grep '*' | sed 's/* //') 
DESCRIPTION=$(git config branch."$NAME".description)
 
regex='TP ([0-9]+):*'
 
melding=$(git log -1 --pretty=format:%s)
 
echo "Viser melding her: "
echo $melding
 
tpnum=0
 
if [[ $melding =~ $regex ]]
then
 tpnum="${BASH_REMATCH[1]}"
        echo "TP number: $tpnum"
        curl -H "Content-Type: application/json" -X POST --data '{"Description":"'"$melding"'", "General": { "Id": "'"$tpnum"'"}}' https://someacme.tpondemand.com/api/v1/Comments?resultFormat=json -u TP_LOGON:TP_PASSWORD
 else 
  echo "Pusher ikke melding ut til Target Process. Bruk: Skriv TP TASKID: melding"
fi

Note that the curl command needs to be a one liner. To use this hook, just do some changes in your code and commit! git commit -m "TP 123: This is a checkin comment for the task with task Id 123 and will be shown in TP via a Git hook!" If you are working on a Windows system, you can download Cygwin (64-bits tested) and the libs curl and curl-lib. I like the Nano editor very much. Happy coding in Git and sharing your check in comments on TP! Pretty nifty to share progress with others!

Tuesday, 26 January 2016

Paged IQueryable ObjectContext EntityFramework

The following article displays how we can achieve retrieving data from EntityFramework using paged results with ObjectContext and sticking inside IQueryable<T> Let's review the extension mehod first:

 public static class EntityExtensions
    {

        public static IQueryable<TEntity> PagedResult<TEntity, TKey>(
            this IQueryable<TEntity> query, 
            Func<TEntity, TKey> sortingFunc, 
            int pageIndex = 1,
            int pageSize = 20)
        {
            var pagedResult = query.OrderBy(sortingFunc)
                .Skip(Math.Max(pageIndex - 0, 0) * pageSize)
                .Take(pageSize);
            return pagedResult.AsQueryable(); 
        }

}

And using AdventureWorks2008R2 database, here is some sample query that shows how we can use this query extension in Linq to Entities:

   using (var ctx = new AdventureWorks2008R2Entities())
   {
                var mountainStuff = from product in ctx.Products
                               where product.Name.Contains("Mountain")
                               select product;
                var firstMountainStuffPage = mountainStuff.PagedResult(p => p.Name, 1, 20);

                foreach (var item in firstMountainStuffPage)
                    Console.WriteLine(item.Name);
   }

            Console.WriteLine("Press any key to continue ..");
            Console.ReadKey();

Output

LL Mountain Frame - Black, 42
LL Mountain Frame - Black, 44
LL Mountain Frame - Black, 48
LL Mountain Frame - Black, 52
LL Mountain Frame - Silver, 40
LL Mountain Frame - Silver, 42
LL Mountain Frame - Silver, 44
LL Mountain Frame - Silver, 48
LL Mountain Frame - Silver, 52
LL Mountain Front Wheel
LL Mountain Handlebars
LL Mountain Pedal
LL Mountain Rear Wheel
LL Mountain Rim
LL Mountain Seat Assembly
LL Mountain Seat/Saddle
LL Mountain Tire
ML Mountain Frame - Black, 38
ML Mountain Frame - Black, 40
ML Mountain Frame - Black, 44
Press any key to continue ..

Conclusion

So there we are, we now have a query that we can reuse to get our paged result and we can pass in our sorting key. So now we can retrieve paged data from for example queries returning large result sets and only display a single page at a time, supporting quicker fetches from the server for clients, retrieving less data and support mobile clients better by getting data pagewise.

Thursday, 31 December 2015

Paged result with EntityFramework ObjectContext

Paged results are possible using Entity Framework. Sometimes it is desirable to create a utility method to support paged results in a simple way as an extension method. Let's look at how to achieve this. First we download the AdventureWorks 2008 R2 Sales database from: Download AdventureWorks 2008 R2 Sales After downloading the MDF file, restoring it should be easy. Open up SQL Management Studio and create a new query window. Next up, insert the following SQL:
CREATE DATABASE AdventureWorks2008R2  ON (FILENAME = '{drive}:{Folder}') 
FOR ATTACH_REBUILD_LOG
Note that {Drive} here can be c:\ and {Folder} here can be:
 
c:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA
We should have now our data and we can create a new Console Project in Visual Studio and choose adding an ADO Entity Data Object Model to our project, located under Data. Choose to generate from database and choose the Production.Product table in AdventureWorks2008 database. We can then add an EntityFramework 6.x EntityObject Generator. If you cannot locate this kind of item, download the Extension for Visual Studio from here: EF 6.x EntityObject Generator for C# This is necessary as Microsoft has defaulted to DbContext in EntityFramework 6. To support ObjectContext, we choose to use this extension. Finally we can code our Paged result extension method! Let's define the extension method next:

   public static List<TEntity> PagedResult<TEntity, TKey>(
            this ObjectContext ctx,
            Func<TEntity, TKey> sortingfunc,
            int pageIndex,
            int pageSize = 20)
            where TEntity : EntityObject
        {
            var result = ctx.CreateObjectSet<TEntity>().OrderBy(sortingfunc).Skip(Math.Max(pageIndex - 1, 0) * pageSize).Take(pageSize);
            return result.ToList();
        }

Note that our method did not require that much code. We allow to pass in our ordering member or column of our entity and we choose a default page size of 20 which can be adjusted. We specify the entity type (table) and we specify the page index. You can choose a page index of one to this method and that means the first page. This in fact is technically page index zero and we use the Max method here to protect inputting negative indexes. Let's test this extension method out!

 static void Main(string[] args)
        {

            using (var ctx = new AdventureWorks2008R2Entities())
            {
                var prods = ctx.PagedResult<Product, string>(x => x.Name, 2);


                foreach (var p in prods)
                    Console.WriteLine(p.Name);
            }

            Console.WriteLine("Press any key to continue ..");
            Console.ReadKey();
        }

And we get the desired result:
This may be readily inspected using the AdventureWorks2008 Sales database and select the content from the Production.Product table. So there we are, a reusable method to choose paged contents from a database using Entity Framework and ObjectContext. I prefer ObjectContext instead of DbContext because it provides more lowlevel functionality compared to DbContext. (Although DbContext is more convenient to use).

Wednesday, 16 December 2015

Create your very own generic GetById method using LINQ Expression Trees

Working with LINQ Expression Trees is sometimes somewhat a tough challenge for C#-programmers, since they usually do not work with Expression trees directly. But LINQ Expression Trees makes it possible to create highly dynamic functions and supports generics quite nicely. Let's create our very own generic GetById method using Linq Expression trees! First off, let's define a simple POCO and use the System.ComponentModel.DataAnnotations.KeyAttribute:

using System.ComponentModel.DataAnnotations;

namespace GetByIdLinqExpression
{
   
    public class ChoccieBar
    {

        [KeyAttribute]
        public int ItemStockId { get; set; }

        public string Title { get; set; }

        public int Qty { get; set; }

        public decimal ItemPrice { get; set; }


    }
}

Okay, let's then define our generic GetById method that will support LINQ expression trees:

using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Linq.Expressions;
using System;
using System.Collections.Generic;
using System.Linq; 

namespace GetByIdLinqExpression
{
   
    public static class LinqExtensions
    {

        public static TEntity GetById<TEntity>(this IEnumerable<TEntity> source, object primaryKey)
        {
            var itemFound = source.AsQueryable().SingleOrDefault<TEntity>(GetByKeyExpression<TEntity>(primaryKey));
            return itemFound;
        }

        private static Expression<Func<TEntity, bool>> GetByKeyExpression<TEntity>(object primaryKey)
        {
            var primaryKeyProperty = typeof(TEntity).GetProperties()
                .First(p => p.GetCustomAttributes(typeof(KeyAttribute), false).Count() > 0); 

            //Create entity => portion of lambda expression 
            ParameterExpression parameter = Expression.Parameter(typeof(TEntity), "element");

            //Create entity.Id portion of lambda expression 
            MemberExpression property = Expression.Property(parameter, primaryKeyProperty.Name);

            //Create 'id' portion of lambda expression 
            var equalsTo = Expression.Constant(primaryKey);

            //Create entity.Id == 'id' portion of lambda expression 
            var equality = Expression.Equal(property, equalsTo);

            //finally create the entire expression: entity => entity.Id = 'id' 
            Expression<Func<TEntity, bool>> retVal = Expression.Lambda<Func<TEntity, bool>>(equality,
                new[] { parameter });

            return retVal;
        }

    }

}

Note that in our customized extension method we will first take the IEnumerable collection and use the AsQueryable method, since we will use Linq Expression trees. Next we have our own private method that builds up the necessary lambda expression using parameterexpression, memberexpression, constantexpression and equal or equality expression before finally handing it over to a lambdaexpression. If you find it hard to follow, note that you will get nice tooltips as you debug the method. Learning how to assemble for example lambda expressions in Linq expression trees is a must. And yes, the code can quickly grow. The key point is that once you first got it right, you can unlock the complexity using simple calls such as GetById shown in the unit tests next. And of course other, more complex queries can be built up by simpler ones such as this. And let's then create some unit tests to test out this little baby:



    [TestClass]
    public class UnitTest1
    {
       
        [TestMethod]
        public void GetByIdDoesNotThrow()
        {
            var choccieBars = new List
            {
                new ChoccieBar { ItemStockId = 1, Qty = 5200, Title = "Marsbar", ItemPrice = 2.0M },
                new ChoccieBar { ItemStockId = 2, Qty = 5200, Title = "Milky Way", ItemPrice = 1.75M },
                new ChoccieBar { ItemStockId = 3, Qty = 5200, Title = "Bounty", ItemPrice = 2.25M },
                new ChoccieBar { ItemStockId = 4, Qty = 5200, Title = "Lion", ItemPrice = 1.75M }
            };

            var choccieFound = choccieBars.GetById(4);
            Assert.IsNotNull(choccieFound); 

        }

    }

And the test passes. Happy LINQ Expression coding! Remember that using Linq Expression trees there is a lot of things you can achieve in C#! Only creativity can stop you from what extension methods using linq expression trees can be made! And Linq Expression trees might seem difficult, but get started coding them and you soon will find out that they are fun to work with!

Tuesday, 15 December 2015

Generic base class supporting ObjectContext CRUD operations

In Entity Framework it is interesting to see if it is possible to create a base class that support CRUD operations with ObjectContext. I want to create a basic scenario where you can either insert, update (insert many) and delete rows into tables. The matching between entities and datacontracts will be matched upon the same kind of name of properties and data type. First off, let us define an interface for such a base class. Note that I have tested the code with Entity Framework 6.1.3.

using System.Collections.Generic;
using System.Data.Entity.Core.Objects.DataClasses;

namespace BaseClassObjectContext
{

    public interface IDefaultDbCrudOperation<TEntity, TDataContract>
        where TEntity : EntityObject
        where TDataContract : class
    {

        List<TDataContract> GetAll();

        TDataContract InsertOrUpdate(TDataContract dataContract);

        bool Delete(TDataContract entity);

        List<TDataContract> InsertOrUpdateMany(List<TDataContract> dataContracts);

    }

}


We will also need the help of Linq Expression trees, so here is a class QueryableExtensions that helps with that:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;

namespace BaseClassObjectContext
{


    /// <summary>
    /// Enables auto mapping features in Entity Framework
    /// </summary>
    /// <remarks>More information here: http://toreaurstad.blogspot.no/2015/02/automatic-mapping-for-deep-object.html </remarks>
    public static class QueryableExtensions
    {
        public static ProjectionExpression<TSource> Project<TSource>(this IQueryable<TSource> source)
        {
            return new ProjectionExpression<TSource>(source);
        }
    }

    public class ProjectionExpression<TSource>
    {
        private static readonly Dictionary<string, Expression> _expressionCache = new Dictionary<string, Expression>();

        private readonly IQueryable<TSource> _source;

        public ProjectionExpression(IQueryable<TSource> source)
        {
            _source = source;
        }

        public IQueryable<TDest> To<TDest>()
        {
            var queryExpression = GetCachedExpression<TDest>() ?? BuildExpression<TDest>();

            return _source.Select(queryExpression);
        }

        private static Expression<Func<TSource, TDest>> GetCachedExpression<TDest>()
        {
            var key = GetCacheKey<TDest>();

            return _expressionCache.ContainsKey(key) ? _expressionCache[key] as Expression<Func<TSource, TDest>> : null;
        }

        public static Expression<Func<TSource, TDest>> BuildExpression<TDest>()
        {
            var sourceProperties = typeof(TSource).GetProperties();
            var destinationProperties = typeof(TDest).GetProperties().Where(dest => dest.CanWrite);
            var parameterExpression = Expression.Parameter(typeof(TSource), "src");

            var bindings = destinationProperties
                                .Select(destinationProperty => BuildBinding(parameterExpression, destinationProperty, sourceProperties))
                                .Where(binding => binding != null);

            var expression = Expression.Lambda<Func<TSource, TDest>>(Expression.MemberInit(Expression.New(typeof(TDest)), bindings), parameterExpression);

            var key = GetCacheKey<TDest>();

            _expressionCache.Add(key, expression);

            return expression;
        }       

        private static MemberAssignment BuildBinding(Expression parameterExpression, MemberInfo destinationProperty, IEnumerable<PropertyInfo> sourceProperties)
        {
            var sourceProperty = sourceProperties.FirstOrDefault(src => src.Name == destinationProperty.Name);

            if (sourceProperty != null)
            {
                return Expression.Bind(destinationProperty, Expression.Property(parameterExpression, sourceProperty));
            }

            var propertyNameComponents = SplitCamelCase(destinationProperty.Name);

            if (propertyNameComponents.Length >= 2)
            {
                sourceProperty = sourceProperties.FirstOrDefault(src => src.Name == propertyNameComponents[0]);
                if (sourceProperty == null)
                    return null;

                var propertyPath = new List<PropertyInfo> { sourceProperty };
                TraversePropertyPath(propertyPath, propertyNameComponents, sourceProperty);

                if (propertyPath.Count != propertyNameComponents.Length)
                    return null; //must be able to identify the path 

                MemberExpression compoundExpression = null;

                for (int i = 0; i < propertyPath.Count; i++)
                {
                    compoundExpression = i == 0 ? Expression.Property(parameterExpression, propertyPath[0]) :
                        Expression.Property(compoundExpression, propertyPath[i]);
                }

                return compoundExpression != null ? Expression.Bind(destinationProperty, compoundExpression) : null;
            }

            return null;
        }

        private static List<PropertyInfo> TraversePropertyPath(List<PropertyInfo> propertyPath, string[] propertyNames,
            PropertyInfo currentPropertyInfo, int currentDepth = 1)
        {
            if (currentDepth >= propertyNames.Count() || currentPropertyInfo == null)
                return propertyPath; //do not go deeper into the object graph

            PropertyInfo subPropertyInfo = currentPropertyInfo.PropertyType.GetProperties().FirstOrDefault(src => src.Name == propertyNames[currentDepth]);
            if (subPropertyInfo == null)
                return null; //The property to look for was not found at a given depth 

            propertyPath.Add(subPropertyInfo);

            return TraversePropertyPath(propertyPath, propertyNames, subPropertyInfo, ++currentDepth);
        }

        private static string GetCacheKey<TDest>()
        {
            return string.Concat(typeof(TSource).FullName, typeof(TDest).FullName);
        }

        private static string[] SplitCamelCase(string input)
        {
            return Regex.Replace(input, "([A-Z])", " $1", RegexOptions.Compiled).Trim().Split(' ');
        }

    }

}


Next off we need a utility method for reflection:

using System;
using System.ComponentModel;
using System.Linq; 

namespace BaseClassObjectContext
{
    
    public static class ReflectionExtensions
    {


        public static bool HasAttribute<TAttribute>(this PropertyDescriptor pr)
        {
            return pr.Attributes.OfType<TAttribute>().Any(); 
        }

    }
}


The base class for handling all this is then is the following:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
using System.Data.Entity.Core;
using System.Data.Entity.Core.Mapping;
using System.Data.Entity.Core.Metadata.Edm;
using System.Data.Entity.Core.Objects;
using System.Data.Entity.Core.Objects.DataClasses;
using System.Data.Entity.Infrastructure;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;

namespace BaseClassObjectContext
{


    public class BaseDataManager<TEntity, TDataContract> : IDefaultDbCrudOperation<TEntity, TDataContract>
        where TEntity : EntityObject
        where TDataContract : class
    {

        public bool Delete(TDataContract dataContract)
        {
            using (var ctx = new BooksEntities())
            {
                var primaryKey = GetPrimaryKey(dataContract);
                var entityFound = FindEntityByKey(primaryKey, ctx);
                if (entityFound == null)
                    return false;
                ctx.CreateObjectSet<TEntity>().DeleteObject(entityFound);
                ctx.SaveChanges();
                return true;
            }
        }

        private TEntity FindEntityByKey(object primaryKey, ObjectContext ctx)
        {
            return ctx.CreateObjectSet<TEntity>().SingleOrDefault(GetEqualityExpression(primaryKey));
        }

        /// <summary>
        /// Builds up an equality expression using LINQ Expression trees
        /// </summary>
        /// <param name="primaryKey"></param>
        /// <remarks>Source: http://dotnetspeak.com/2013/09/use-reflection-and-expression-to-find-an-entity-by-primary-key </remarks>
        private static Expression<Func<TEntity, bool>> GetEqualityExpression(object primaryKey)
        {
            var primaryKeyProperty = typeof(TEntity).GetProperties()
                .First(p => p.GetCustomAttributes(typeof(EdmScalarPropertyAttribute), true)
                .Any(pc => ((EdmScalarPropertyAttribute)pc).EntityKeyProperty));

            //Create entity => portion of lambda expression 
            ParameterExpression parameter = Expression.Parameter(typeof(TEntity), "entity");

            //Create entity.Id portion of lambda expression 
            MemberExpression property = Expression.Property(parameter, primaryKeyProperty.Name);

            //Create 'id' portion of lambda expression 
            var equalsTo = Expression.Constant(primaryKey);

            //Create entity.Id == 'id' portion of lambda expression 
            var equality = Expression.Equal(property, equalsTo);

            //finally create the entire expression: entity => entity.Id = 'id' 
            Expression<Func<TEntity, bool>> retVal = Expression.Lambda<Func<TEntity, bool>>(equality,
                new[] { parameter });

            return retVal;
        }

        private TEntity FindEntityByKeys(object[] primaryKeys, ObjectContext ctx)
        {
            foreach (var keyValues in ctx.CreateObjectSet<TEntity>().Select(t =>
                new { Key = t.EntityKey, Values = t.EntityKey.EntityKeyValues }))
            {
                if (keyValues.Values.ToArray().SequenceEqual(primaryKeys))
                    return ctx.GetObjectByKey(keyValues.Key) as TEntity;
            }
            return null;
        }

        public List<TDataContract> GetAll()
        {
            using (var ctx = new BooksEntities())
            {
                return ctx.CreateObjectSet<TEntity>().Project().To<TDataContract>().ToList();
            }
        }

        public TDataContract InsertOrUpdate(TDataContract dataContract)
        {
            using (var ctx = new BooksEntities())
            {
                var primaryKey = GetPrimaryKey(dataContract);
                var entityToInsertOrUpdate = FindEntityByKey(primaryKey, ctx);
                bool isNew = entityToInsertOrUpdate == null;

                entityToInsertOrUpdate = Queryable.AsQueryable(new[] { dataContract }).Project().To<TEntity>().First();

                if (isNew)
                    ctx.CreateObjectSet<TEntity>().AddObject(entityToInsertOrUpdate);
                else
                {
                    var existingEntity = FindEntityByKey(primaryKey, ctx);
                    ctx.CreateObjectSet<TEntity>().Detach(existingEntity);
                    ctx.CreateObjectSet<TEntity>().Attach(entityToInsertOrUpdate);
                    ctx.ObjectStateManager.ChangeObjectState(entityToInsertOrUpdate, System.Data.Entity.EntityState.Modified);
                }

                ctx.SaveChanges();

                return Queryable.AsQueryable(new[] { entityToInsertOrUpdate }).Project().To<TDataContract>().First();
            }
        }  


        public List<TDataContract> InsertOrUpdateMany(List<TDataContract> dataContracts)
        {
            if (dataContracts == null)
                throw new ArgumentNullException("An empty list was provided!");
            var changesMade = new List<TDataContract>();
            foreach (var dc in dataContracts)
                changesMade.Add(InsertOrUpdate(dc)); //Simple logic in this case 
            return changesMade;
        }

        private object GetPrimaryKey(TDataContract entity)
        {
            PropertyDescriptor pDesc = TypeDescriptor.GetProperties(entity)
                .Cast<PropertyDescriptor>().FirstOrDefault(p => p.HasAttribute<KeyAttribute>());
            if (pDesc == null)
                throw new InvalidOperationException("Provided datacontract must have one column with Key attribute!");

            var primaryKey = pDesc.GetValue(entity);
            return primaryKey;
        }

        private string GetPrimaryKeyName(TDataContract entity)
        {

            var primaryKeyProperty = typeof(TEntity).GetProperties()
              .First(p => p.GetCustomAttributes(typeof(EdmScalarPropertyAttribute), true)
              .Any(pc => ((EdmScalarPropertyAttribute)pc).EntityKeyProperty));
            return primaryKeyProperty.Name;
        }

        private static MetadataWorkspace _metaDataWorkSpace;

        public static string GetTableName(Type type, ObjectContext context)
        {
            if (_metaDataWorkSpace == null)
                _metaDataWorkSpace = context.MetadataWorkspace;

            // Get the part of the model that contains info about the actual CLR types
            var objectItemCollection = ((ObjectItemCollection)_metaDataWorkSpace.GetItemCollection(DataSpace.OSpace));

            // Get the entity type from the model that maps to the CLR type
            var entityType = _metaDataWorkSpace
                    .GetItems<EntityType>(DataSpace.OSpace)
                    .Single(e => objectItemCollection.GetClrType(e) == type);

            // Get the entity set that uses this entity type
            var entitySet = _metaDataWorkSpace
                .GetItems<EntityContainer>(DataSpace.CSpace)
                .Single()
                .EntitySets
                .Single(s => s.ElementType.Name == entityType.Name);

            // Find the mapping between conceptual and storage model for this entity set
            var mapping = _metaDataWorkSpace.GetItems<EntityContainerMapping>(DataSpace.CSSpace)
                    .Single()
                    .EntitySetMappings
                    .Single(s => s.EntitySet == entitySet);

            // Find the storage entity set (table) that the entity is mapped
            var table = mapping
                .EntityTypeMappings.Single()
                .Fragments.Single()
                .StoreEntitySet;

            // Return the table name from the storage entity set
            return (string)table.MetadataProperties["Table"].Value ?? table.Name;
        }
    }

}


The following to compact DAL-layer managers can then be defined in a sample database:

namespace BaseClassObjectContext
{

    public class AuthorManager : BaseDataManager<Author, AuthorDataContract>
    {

    }

}

namespace BaseClassObjectContext
{
    
    public class BookManager : BaseDataManager<Book, BookDataContract>
    {

    }

}


See how little code we need to write to work against a table with Entity Framework now and support easy methods such as GetAll, InsertOrUpdate, InsertOrUpdateMany and Delete? This may seem overly complex, but by developing a sturdy generic code that can handle different mapping scenarios, it would be possible to end up with a Data Access Layer (DAL) that is more easy to maintain than a regular one with the same kind of tedious mapping that you end up with the default path chosen. I have also created some default integration tests:

using System;
using NUnit.Framework;

namespace BaseClassObjectContext.Test
{
  
    [TestFixture]
    public class BookAuthorCrudTests
    {

        [Test]
        public void InsertAuthorAndBookAndDeleteAfterwardsTest()
        {

            var authorManager = new AuthorManager();
            var savedAuthor = authorManager.InsertOrUpdate(new AuthorDataContract
            {
                Name = "Anne B. Ragde",
                Age = 58
            });
            var bookManager = new BookManager();
            var savedBook = bookManager.InsertOrUpdate(new BookDataContract
            {
                PageCount = 302,
                Title = "Eremittkrepsene",
                ISBN = " 9788252560930",
                AuthorId = savedAuthor.AuthorId
            });
            Assert.IsTrue(savedAuthor.AuthorId > 0);
            Assert.IsTrue(savedBook.BookId > 0);

            var authors = authorManager.GetAll();
            var books = authorManager.GetAll();

            CollectionAssert.IsNotEmpty(authors);
            CollectionAssert.IsNotEmpty(books);

            bool bookDeleted = bookManager.Delete(savedBook);
            bool authorDeleted = authorManager.Delete(savedAuthor);

            Assert.IsTrue(authorDeleted);
            Assert.IsTrue(bookDeleted);
        }

        [Test]
        public void InsertAuthorAndBookAndGetAllTest()
        {
            var authorManager = new AuthorManager();
            var savedAuthor = authorManager.InsertOrUpdate(new AuthorDataContract
            {
                Name = "Anne B. Ragde",
                Age = 58              
            });
            var bookManager = new BookManager();
            var savedBook = bookManager.InsertOrUpdate(new BookDataContract
            {
                PageCount = 313,
                Title = "Berlinerpopplene",
                ISBN = " 9788249509584",
                AuthorId = savedAuthor.AuthorId
            });
            Assert.IsTrue(savedAuthor.AuthorId > 0); 
            Assert.IsTrue(savedBook.BookId > 0);

            var authors = authorManager.GetAll();
            var books = authorManager.GetAll();

            CollectionAssert.IsNotEmpty(authors);
            CollectionAssert.IsNotEmpty(books);

            savedBook.Title = "Berlinerpoplene";
            var savedBookAgain = bookManager.InsertOrUpdate(savedBook);
            Assert.AreEqual(savedBook.Title, savedBookAgain.Title); 
        }



    }
}


Note that the code works, but needs further refinement and cleanup to support more scenarios. What would be nice would be to device a way to handle navigation properties, lazy loading and lists within the entities. I have not tested this. Some support for navigation properties should be present with the QueryableExtensions class.

Download sample Visual Studio Solution:

[GenericObjectContextCrudSampleSolution.zip | 14.6 MB | (.Zip) file]
The sample database contains two simple tables: Authors and Books, where a Book has an Author. To test out all the code above, you can download a Zip file below with the source code. I have used Visual Studio 2013. Enjoy and happy Entity Framework coding!