Monday 23 July 2018

Using MSMQ with WCF and message security

This article will present how you can get started using MSMQ as the communication protocol with WCF and apply message level security. MSMQ has got some strengths compared to HTTP requests and other communication protocols:
  • Requests are default durable, that is if the server is down, the client can send the messages to the server later when it is back again
  • There are no responses as the protocol is one-way and sometimes avoiding a response is a gain
  • All requests are queued and can be made ordered. Using transactional MSMQ queues allows requests only be sent once
  • It can utilize features such as Journal, system queues such as poison messages and dead letters and integrates well with Biztalk
  • It is the most resilient and sturdy communication protocol around and works well also in server-server scenarios and for clients
It also has some weaknesses:
  • There are no responses, so it requires to inspect the queue(s) if something went wrong
  • If there are a lot of messages, chances are that MSMQ queues will go full - it is not the quickest protocol
  • Clients must also have MSMQ installed, not default set up in Windows
  • There is a learning curve for developers and users as MSMQ is less known protocol compared to HTTP, HTTPS and TCP
I have created a sample to get you started with WCF and MSMQ. It will support Message-level security with self signed certificates for client and server. First clone this repo with Git:
git clone https://toreaurstad@bitbucket.org/toreaurstad/demonetmsmqwcfgit.git

Open up the solution in Visual Studio (2017 or 2015). First off, the solution needs to set up self signed certificates for client and server on your developer PC. Run the Script in the Scripts folder in the Host project, the Powershell script is: CreateCertificatesMsmqDemo.ps1 Run it as admin, as it will use Powershell to generate a new certificate, copy over Openssl.exe to c:\openssl folder (if you already have c:\openssl populated, you might want to change this) and convert the certificate to have a RSA format instead of CNG. There are more setup to do, such as selecting your web site in Internet Information Server admin (inetmgr) and setting up enabled protocols to http, net.msmq for the web site for this solution. You need to install MSMQ as a feature in Windows with required subfeatures. Also after the certificates are installed, right click them (in MMC you select Local Computer and Personal certificate store) and select Manage private keys. Now adjust security settings here so that your App Pool user can access the certificates for MSMQ.

Here is where you in Inetmgr (IIS Admin) set up enabled protocols to MSMQ for the web site of this article:
Moving on to the solution itself, I will describe the implementation details. Note that the creation of the MSMQ queue is helped programatically, as the queue cannot be created automatically by WCF. I created a ServiceHostFactory class to create the MSMQ queue that is used in the communication between the client and server.
using System;
using System.Configuration;
using System.Messaging;
using System.ServiceModel;
using System.ServiceModel.Activation;

namespace WcfDemoNetMsmqBinding.Host
{
    public class MessageQueueFactory : ServiceHostFactory
    {

        public override ServiceHostBase CreateServiceHost(string constructorString, Uri[] baseAddresses)
        {
            CreateMsmqIfMissing();
            return base.CreateServiceHost(constructorString, baseAddresses);
        }

        public static void CreateMsmqIfMissing()
        {
            string queueName = string.Format(@".\private$\{0}", MessageQueueName);
            if (!MessageQueue.Exists(queueName))
            {
                MessageQueue createdMessageQueue = MessageQueue.Create(queueName, true);
                string usernName = System.Security.Principal.WindowsIdentity.GetCurrent().Name;
                createdMessageQueue.SetPermissions(usernName, MessageQueueAccessRights.FullControl);

            }
        }

        public static string MessageQueueName
        {
            get
            {
                return ConfigurationManager.AppSettings["QueueName"];

            }
        }

    }
}
This class will create the MessageQueue if it is missing. The QueueName is an appsetting in web config. The .svc file for the Msmq service contains a reference to the factory.

<%@ ServiceHost Language="C#" Debug="true" Service="WcfDemoNetMsmqBinding.Host.MessageQueueService" Factory="WcfDemoNetMsmqBinding.Host.MessageQueueFactory" %>

Now, the setup of the wcf service is done declaratively in web.config (it could be done in code, but I chose to use web.config for most of this sample for defining the MSMQ service). This is the web.config that define the MSMQ service, as you can see it is not very extensive to get started with MSMQ and WCF:

<?xml version="1.0"?>
<configuration>

  <appSettings>
    <add key="aspnet:UseTaskFriendlySynchronizationContext" value="true" />
    <add key="QueueName" value="DemoQueue3" />
  </appSettings>
  <system.web>
    <compilation debug="true" targetFramework="4.6.1" />
    <httpRuntime targetFramework="4.6.1"/>
  </system.web>
  
  <system.serviceModel>

    <services>
      <service name="WcfDemoNetMsmqBinding.Host.MessageQueueService" behaviorConfiguration="NetMsmqBehavior">
        <endpoint contract="WcfDemoNetMsmqBinding.Host.IMessageQueueService" name="NetMsmqEndpoint" 
                  binding="netMsmqBinding" bindingConfiguration="NetMsmq" address="net.msmq://localhost/private/DemoQueue3" />
      </service>    
    </services>

    <bindings>
      <netMsmqBinding>
        <binding name="NetMsmq" durable="true" exactlyOnce="true" receiveErrorHandling="Move" useActiveDirectory="False" queueTransferProtocol="Native">
          <security mode="Message">
            <message clientCredentialType="Certificate"/>
          </security>
        </binding>
      </netMsmqBinding>
    </bindings>
    
      
    <behaviors>
      <serviceBehaviors>
        <behavior name="NetMsmqBehavior">
          <!-- To avoid disclosing metadata information, set the values below to false before deployment -->
          <serviceMetadata httpGetEnabled="true" />
          <!-- To receive exception details in faults for debugging purposes, set the value below to true.  Set to false before deployment to avoid disclosing exception information -->
          <serviceDebug includeExceptionDetailInFaults="true"/>
        <serviceCredentials>
          <serviceCertificate findValue="MSMQWcfDemoserver" storeLocation="LocalMachine" storeName="My" x509FindType="FindBySubjectName" />
            <clientCertificate>
              <certificate findValue="MSMQWcfDemoClient" storeLocation="LocalMachine" storeName="My" x509FindType="FindBySubjectName" />
              <authentication certificateValidationMode="PeerTrust" />
            </clientCertificate>
        </serviceCredentials>
        
        </behavior>
      </serviceBehaviors>
    </behaviors>

  </system.serviceModel>
  <system.webServer>
    <modules runAllManagedModulesForAllRequests="true"/>
    <!--
        To browse web app root directory during debugging, set the value below to true.
        Set to false before deployment to avoid disclosing web app folder information.
      -->
    <directoryBrowse enabled="true"/>
  </system.webServer>

</configuration>

Note that the two certificates that were generated in this sample is set up in serviceCredentials element. We define the serviceCertificae and clientCertificate here. The client takes note to point to the same queue, net.msmq://localhost/private/DemoQueue3 Note that this example is tested with the client and server on same machine. The client could have a local queue in case the server was on a different machine to support durability in the scenario of many computers The client sets up the corresponding certificates in the app.config:
 
 <?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <startup> 
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6.1" />
    </startup>
    <system.serviceModel>
        <bindings>
            <netMsmqBinding>
                <binding name="NetMsmqEndpoint">
                    <security mode="Message">
                        <message clientCredentialType="Certificate" algorithmSuite="Default" />
                    </security>
                </binding>
            </netMsmqBinding>
        </bindings>
      <behaviors>
        <endpointBehaviors>
          <behavior name="MsmqEndpointBehavior">
            <clientCredentials>
              <clientCertificate storeName="My" storeLocation="LocalMachine" x509FindType="FindBySubjectName"
                                  findValue="MSMQWcfDemoClient" />
              <serviceCertificate>
                <defaultCertificate storeLocation="LocalMachine" storeName="My" x509FindType="FindBySubjectName" findValue="MSMQWcfDemoserver"/>
              </serviceCertificate>
            
            </clientCredentials>
            
          </behavior>
        </endpointBehaviors>
      </behaviors>
        <client>
            <endpoint address="net.msmq://localhost/private/DemoQueue3" binding="netMsmqBinding"
                bindingConfiguration="NetMsmqEndpoint" behaviorConfiguration="MsmqEndpointBehavior" contract="MessageQueueServiceProxy.IMessageQueueService"
                name="NetMsmqEndpoint">
              <identity>
                <dns value="MSMQWcfDemoServer"/>
              </identity>

            </endpoint>
        </client>
    </system.serviceModel>
  <system.web>
    <compilation debug="true" />
  </system.web>
</configuration>

All in all, we end up with a sample where you can communicate between a client and a service using WCF with durable and ordered requests using NetMsmqBinding. As I have presented in earlier articles, the communication is actually in the form of WCF message inside every MSMQ message. The WCF message itself is protected, as this picture shows.

(it is protected using the self signed certificate we generated)
The Powershell script InspectMessageQueueWcfContent.ps1 is included, if you want to inspect the WCF messages themselves of the queue themselves. Note that if you enable Journal on the MSMQ queue this solution creates, you can see the MSMQ messages after they have been received and consumed, using compmgmt.msc Or an alternative is to use QueueExplorer instead, available as a trial here: Queue Explorer
This tool can view the WCF message inside the MSMQ message as in my Powershell script, but also display syntax coloring and other functionality for quickly navigation of your MSMQ queues. Are MSMQ in WCF an alternative for ordinary scenarios using HTTP, TCP or Federated bindings? This article was meant at least to give a demo of the capability WCF gives developers to utilize MSMQ as the communication protocol between client and server. Hope you found this article interesting.

Sunday 22 July 2018

Self-signed certificates in .NET with RsaCryptoServiceProvider

This article is written after experiencing how difficult it is today to create self-signed certificates that works with .NET and specificially WCF. If you use the default method in Windows, using the cmdlet New-SelfSignedCertificate in Powershell, chances are high that you will stumble upon this error: System.Security.Cryptography.CryptographicException: Invalid provider type specified. A few years back, creating certificates was way easier using makecert.exe Then Windows required the certificates to have length of minimum 1024. After some years passed, makecert.exe was deprecated. Microsoft now offers .NET developers to create a self signed certificate the cmdlet New-SelfSignedCertificate to use in development and test environments. Actually it turns out that you still must adjust the private part of certificate to use the RSACryptoServiceProvider by using OpenSSL to achieve this and then import to the certificate, sadly Microsoft gives developers tools for self-signed certificate generation that needs to use third-part libraries such as SSLR to work properly with .NET, at least this is the case with WCF.. Here is the Powershell script I ended up with:
 Write-Host "Generating a self signed certificate for client in MSMQ WCF demo"

$clientCertFile = "C:\temp\MSMQWcfDemoClient.pfx" 

$clientCertFileRsaFormatted = "C:\temp\MSMQWcfDemoClient.RSAConverted.pfx"

$cert = New-SelfSignedCertificate -Subject "CN=MSMQWcfDemoClient" -certstorelocation cert:\localmachine\my `
-NotAfter (Get-Date).AddYears(3) -KeyLength 2048 -KeySpec KeyExchange
$pwd = ConvertTo-SecureString -String ‘bongo’ -Force -AsPlainText 
$path = 'cert:\localMachine\my\' + $cert.thumbprint 
Export-PfxCertificate -cert $path -FilePath $clientCertFile -Password $pwd

$clientCertPasswordSec = ConvertTo-SecureString "bongo" -AsPlainText -Force


Write-Host "Generating a self signed certificate for server in MSMQ WCF demo"

$serverCertFile = "C:\temp\MSMQWcfDemoServer.pfx" 

$serverCertFileRsaFormatted =  "C:\temp\MSMQWcfDemoServer.RSAConverted.pfx"

$certServer = New-SelfSignedCertificate -Subject "CN=MSMQWcfDemoserver" -certstorelocation cert:\localmachine\my `
  -NotAfter (Get-Date).AddYears(3) -KeyExportPolicy Exportable -KeyLength 2048  -KeySpec KeyExchange
$pwdServer = ConvertTo-SecureString -String ‘kongo’ -Force -AsPlainText
$pathServer = 'cert:\localMachine\my\' + $certServer.thumbprint 
Export-PfxCertificate -cert $pathServer -FilePath $serverCertFile -Password $pwdServer

$serverCertPasswordSec = ConvertTo-SecureString "kongo" -AsPlainText -Force

$command = @'
cmd.exe /c c:\temp\rsaconvert.bat
'@

Write-Host "Starting bat file to convert from CNG to RSA format.." 

Invoke-Expression -Command:$command 

Write-Host "Importing RSA formatted certificates.."

Import-PfxCertificate -FilePath $clientCertFileRsaFormatted -CertStoreLocation Cert:\LocalMachine\My -Password $clientCertPasswordSec
Import-PfxCertificate -FilePath $clientCertFileRsaFormatted -CertStoreLocation Cert:\LocalMachine\root -Password $clientCertPasswordSec



Import-PfxCertificate -FilePath $serverCertFileRsaFormatted -CertStoreLocation Cert:\LocalMachine\My -Password $serverCertPasswordSec
Import-PfxCertificate -FilePath $serverCertFileRsaFormatted -CertStoreLocation Cert:\LocalMachine\root -Password $serverCertPasswordSec

The powershell script uses a bat file that calls Openssl to convert the pfx certificate to use the RSACryptoServiceProvider. This is how the bat file looks like:
echo Generating RSA format certificates for server using OpenSSL..

c:\openssl\openssl.exe pkcs12 -in "C:\temp\MSMQWcfDemoserver.pfx" -nokeys -out "C:\temp\MSMQWcfDemoserver.cer" -passin "pass:kongo"
c:\openssl\OpenSSL.exe pkcs12 -in "C:\temp\MSMQWcfDemoserver.pfx" -nocerts -out "C:\temp\MSMQWcfDemoserver.pem" -passin "pass:kongo" -passout "pass:kongo"
c:\openssl\OpenSSL.exe rsa -inform PEM -in "C:\temp\MSMQWcfDemoserver.pem" -out "C:\temp\MSMQWcfDemoserver.rsa" -passin "pass:kongo" -passout "pass:kongo"
c:\openssl\openssl.exe pkcs12 -export -in  "C:\temp\MSMQWcfDemoserver.cer" -inkey "C:\temp\MSMQWcfDemoserver.rsa" -out "C:\temp\MSMQWcfDemoserver.RSAConverted.pfx" -passin "pass:kongo" -passout "pass:kongo"


echo Generating RSA format certificates for client using OpenSSL..

c:\openssl\openssl.exe pkcs12 -in "C:\temp\MSMQWcfDemoclient.pfx" -nokeys -out "C:\temp\MSMQWcfDemoclient.cer" -passin "pass:bongo"
c:\openssl\OpenSSL.exe pkcs12 -in "C:\temp\MSMQWcfDemoclient.pfx" -nocerts -out "C:\temp\MSMQWcfDemoclient.pem" -passin "pass:bongo" -passout "pass:bongo"
c:\openssl\OpenSSL.exe rsa -inform PEM -in "C:\temp\MSMQWcfDemoclient.pem" -out "C:\temp\MSMQWcfDemoclient.rsa" -passin "pass:bongo" -passout "pass:bongo"
c:\openssl\openssl.exe pkcs12 -export -in  "C:\temp\MSMQWcfDemoclient.cer" -inkey "C:\temp\MSMQWcfDemoclient.rsa" -out "C:\temp\MSMQWcfDemoclient.RSAConverted.pfx" -passin "pass:bongo" -passout "pass:bongo"


The bat file uses OpenSSL to extract (dismantle) the public part of the certificate into a .cer file (this part could have been done with MMC). The next step is to generate a from our existing pfx file to export to a pem file and then a rsa file with OpenSSL. We then meld the .cer file and .rsa file into a converted .pfx file. This file is with the Powershell script automatically imported into the certificate store Personal (My) of certificate location Local Computer. The Powershell script also Import-PfxCertificate to Trusted Root Certification Authorities. Anyways, my goal was to give you a demo about .NET, WCF and NetMsmqBinding using message security, but I first had to get over this hurdle to be able to have some protection in WCF with certificate and had no idea that Microsoft had given developers so cumbersome tools to generate a self signed certificate to actually work with WCF. Now my MSMQ message queue is filled with encrypted and protection MSMQ messages (containing WCF message to be consumed)! :)
Note: the MSMQ queue was inspected using this Powershell script:

#
# InspectMessageQueueWcfContent.ps1
#


#
# InspectMessageQueue.ps1
#

[System.Reflection.Assembly]::LoadWithPartialName("System.Messaging") | Out-Null
[System.Reflection.Assembly]::LoadWithPartialName("System.Xml") | Out-Null
[System.Reflection.Assembly]::LoadWithPartialName("System.ServiceModel") | Out-Null
[System.Reflection.Assembly]::LoadWithPartialName("System.IO") | Out-Null


$queuePath = ".\private$\demoqueue3"

Write-Host "Powershell MSMQ queue WCF inspector v0.1. Inspecting queue contents of the queue: $queuePath"
Write-Host ""


Run-MainDemoIterateMsmq $queuePath



Function Get-XmlFromWcfMessage([System.Messaging.Message] $msg) {
   
    $doc = New-Object System.Xml.XmlDocument;
    $messageLength = [int] $msg.BodyStream.Length


    $buffer = New-Object byte[] $messageLength

    
    $msg.BodyStream.Read($buffer, 0, $messageLength)

    $envelopeStart = Find-SoapEnvelopeStart($buffer)

    $envelopeStart = $envelopeStart - 0

    $envelopeLength = $($buffer.Length - $envelopeStart)
    #Write-Host $envelopeStart


    $stream = New-Object System.IO.MemoryStream($buffer, $envelopeStart, $envelopeLength)

    $elm = New-Object System.ServiceModel.Channels.BinaryMessageEncodingBindingElement
    $elm.ReaderQuotas.MaxStringContentLength = 10000000
    $elm.ReaderQuotas.MaxBytesPerRead = 10000000


    $msg1 = $elm.CreateMessageEncoderFactory().Encoder.ReadMessage($stream, 10000000);

    $doc.Load($msg1.GetReaderAtBodyContents());

    $msg.BodyStream.Position = 0;



    return $doc;
}

Function Find-SoapEnvelopeStart([byte[]] $stream)
{
    $i = 0;
    $j = 0;
    $prevByte = $stream[$i];
    $curByte = [byte]$j;
    for ($i = 0; $i -lt $stream.Length; $i++)
    {
        $curByte = $stream[$i];
        if ($curByte -eq [byte] 0x02 -and $prevByte -eq [byte] 0x56) {
            break;
        }
        $prevByte = $curByte;
    }
    return $i - 1;
}


Function Run-MainDemoIterateMsmq([string] $queuePath) {

$queue = New-Object System.Messaging.MessageQueue $queuePath

foreach ($message in $queue.GetAllMessages()){
  $xmlDoc = Get-XmlFromWcfMessage $message 
  Write-Host $xmlDoc.OuterXml
 } 


}





Saturday 21 July 2018

Reading WCF messages in MSMQ queues in Powershell

The last article looked at displaying MSMQ contents for a MSMQ queue used with NetMsmqBinding using Powershell. The message queues contained characters that were unreadable. That is because the message queue items contains actually ready to consume WCF messages. This article will present a new Powershell script where the MSMQ body contents is finally readable using Powershell and .NET WCF classes in System.ServiceModel. Here is how the contents looks using regular methods to extract the MSMQ Message queue items with garbled contents. This is as noted due to the MSMQ queue items each containing a corresponding WCF message.
Let's fix this up by using System.ServiceModel classes!
[System.Reflection.Assembly]::LoadWithPartialName("System.Messaging") | Out-Null
[System.Reflection.Assembly]::LoadWithPartialName("System.Xml") | Out-Null
[System.Reflection.Assembly]::LoadWithPartialName("System.ServiceModel") | Out-Null
[System.Reflection.Assembly]::LoadWithPartialName("System.IO") | Out-Null


$queuePath = ".\private$\demoqueue4"

Write-Host "Powershell MSMQ queue WCF inspector v0.1. Inspecting queue contents of the queue: $queuePath"
Write-Host ""


Run-MainDemoIterateMsmq $queuePath



Function Get-XmlFromWcfMessage([System.Messaging.Message] $msg) {
   
    $doc = New-Object System.Xml.XmlDocument;
    $messageLength = [int] $msg.BodyStream.Length


    $buffer = New-Object byte[] $messageLength

    
    $msg.BodyStream.Read($buffer, 0, $messageLength)

    $envelopeStart = Find-SoapEnvelopeStart($buffer);

    $envelopeLength = $($buffer.Length - $envelopeStart)
    #Write-Host $envelopeStart


    $stream = New-Object System.IO.MemoryStream($buffer, $envelopeStart, $envelopeLength)

    $elm = New-Object System.ServiceModel.Channels.BinaryMessageEncodingBindingElement
    $elm.ReaderQuotas.MaxStringContentLength = 10000000
    $elm.ReaderQuotas.MaxBytesPerRead = 10000000


    $msg1 = $elm.CreateMessageEncoderFactory().Encoder.ReadMessage($stream, 10000000);

    $doc.Load($msg1.GetReaderAtBodyContents());

    $msg.BodyStream.Position = 0;



    return $doc;
}

Function Find-SoapEnvelopeStart([byte[]] $stream)
{
    $i = 0;
    $j = 0;
    $prevByte = $stream[$i];
    $curByte = [byte]$j;
    for ($i = 0; $i -lt $stream.Length; $i++)
    {
        $curByte = $stream[$i];
        if ($curByte -eq [byte] 0x02 -and $prevByte -eq [byte] 0x56) {
            break;
        }
        $prevByte = $curByte;
    }
    return $i - 1;
}


Function Run-MainDemoIterateMsmq([string] $queuePath) {

$queue = New-Object System.Messaging.MessageQueue $queuePath

foreach ($message in $queue.GetAllMessages()){
  $xmlDoc = Get-XmlFromWcfMessage $message 
  Write-Host $xmlDoc.OuterXml
 } 


}
The soap envelope is found looking after the escape sequence "V" followed with the special ANSI character 0x02 (STX = Start of Text). From this point on, the rest of the WCF Message is our SOAP body! We then get readable output!
Finally, the WCF messages inside the MSMQ queue that is filled up using NetMsmqBinding is readable! In my next article I will present a demo solution of how to use NetMsmqBinding in WCF! Until then, you can clone the solution already from here: git clone git@bitbucket.org:toreaurstad/demonetmsmqwcfgit.git