Simple Base64 encoder in C#

Just read a bit about Base64 encoding and decided to try out writing my own in Linqpad for strings! The way you Base64 encode is to treat each char in the input string as a byte value with groups of six bytes (this byte value is zero padded left) and then mapping the 2^6 values into a Base64 table and outputting the corresponding values into a resulting string. Note that you also pad the Base64 string with '=' char to ensure the entire bit length is divisible with three. That is why Base64 strings in .NET always have 0,1 or 2 '=' chars at the end. The characters used in the Base64 encoding is in .NET the chars [A-z] and [0-9] plus + and slash /.

void Main()
{
 string wordToBase64Encode = "Hello world from base64 encoder. This is a sample input string, does it work?"; 
 string wordBase64EncodedAccordingToNet = Convert.ToBase64String(Encoding.ASCII.GetBytes(wordToBase64Encode)).Dump();
 string wordBase64EncodedAccordingToCustomBase64 = wordToBase64Encode.ToBase64String().Dump();
 (("Are the two strings equal?: ") + string.Equals(wordBase64EncodedAccordingToNet, wordBase64EncodedAccordingToCustomBase64)).Dump();
 
}

public static class LowLevelBase64Extensions {

    private static readonly char[] _base64Table =
 {
  'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 
  'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z',
  'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 
  'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z',
  '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '+', '/',
 };

 public static string ToBase64String(this string toBeEncoded){
     byte[] toBeEncodedByteArray = Encoding.ASCII.GetBytes(toBeEncoded);
  string toBeEncodedBinaryString = 
  string.Join("", 
   toBeEncodedByteArray.Select(b => (Convert.ToString(b, 2).PadLeft(8, '0'))));
   
  //toBeEncodedBinaryString.Length.Dump();
   
  int padCharacters = toBeEncodedBinaryString.Length % 3;
  //toBeEncoded.Dump();
       
  StringWriter sw = new StringWriter();

  for (var i = 0; i < toBeEncodedBinaryString.Length; i = i + 6)
  {
   string encodingToMap =
    toBeEncodedBinaryString.Substring(i, Math.Min(toBeEncodedBinaryString.Length - i, 6))
    .PadRight(6, '0');
   //encodingToMap.Dump();
   byte encodingToMapByte = Convert.ToByte(encodingToMap, 2);
   char encodingMapped = _base64Table[(int)encodingToMapByte];
   sw.Write(encodingMapped);
  }
  
  sw.Write(new String('=', padCharacters));
  
  //Check 24 bits / 3 byte padding 
  return sw.ToString();
 }
}

Note - the casting to int of the encodingToMapByte also works if you cast the byte representation to short or byte. I have compared this custom Base64Encoder with .NET's own Convert.ToBase64String(), and they give equal strings in the use cases I have tried out. Make note that this is just playing around, .NET's own method is optimized for speed. This is more to show what happens with the input string to generate a Base64 encoded string. You can use this code as a basis for custom encoding shemes like Base32. I have used Encoding.ASCII.GetBytes to look up the ASCII code. So this implementation primarily supports input strings in languages using ASCII and Extended ASCII characters, at least that is what I have tested it with. The output in Linqpad shows the following corresponding results:

Consuming video from WCF as a Base64 encoded strings

This article follows up our quest of loading up video byte arrays from WCF efficiently, but now the byte array is converted into a Base64 encoded string. You can start by cloning the repository I have prepared from here:
git clone https://toreaurstad@bitbucket.org/toreaurstad/wcfaudiostreamdemo.git git fetch && git checkout VideBase64String_05082018
Base64 encoded strings take six bits from the byte array and designates is as a char where the char can be one of 64 characters, in MIME implementation it is [A-z][0-9] and + and /, which is 64 different characters that the six bytes are encoded into. This means that 24 bits can be represented as three Base64 encodeded characters, that is 3 bytes in our byte array can be represented as FOUR base64 encoded chars. This 3:4 ratio is the rationale behind the MTOM optimization in WCF (to be discussed later). However, recap from the previous article where we downloaded a video sizing 4 MB on disk and Fiddler reporting it to be about 3 times larger. This is bloated data that I want to explore if we can fix up a bit. The bloated data is because I have managed not to truly enforce the XMLHttpRequest to send the data through the WCF REST binding (webHttpBinding) as true binary data - binary data is still sent as a string object to Javascript. This is sad, and I want to try to speed this up and avoid bloated data. To get this size down, we can start by using a Base64 encoded string instead in our XmlHttpRequest. First off, we define a service contract operation to return our string with Base64 data:

        [OperationContract]
        [WebGet(UriTemplate = "mediabytes/{videofile}")]
        string GetVideoAsBase64(string videofile);

Now our Service Implementation looks like this:

using System;
using System.IO;
using System.ServiceModel.Activation;
using System.ServiceModel.Web;
using System.Web;
using WcfStreamAudioDemo.Common;

namespace WcfStreamAudioDemo.Host
{

    [AspNetCompatibilityRequirements (RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
    public class VideoService : IVideoServiceContract
    {

        public Stream GetVideo(string videofile)
        {
            return GetVideoStream(videofile, ".mp4");
        }

        public string GetVideoAsBase64(string videofile)
        {
            Stream videoStream = GetVideoStream(videofile, ".mp4");

            byte[] buffer = new byte[32*1024];
            int count;
            var memoryStream = new MemoryStream();

            while ((count = videoStream.Read(buffer, 0, buffer.Length)) > 0)
            {
                memoryStream.Write(buffer, 0, count);
            }

            return Convert.ToBase64String(memoryStream.ToArray()); 
        }

        private Stream GetVideoStream(string videofile, string extension)
        {
            string mediaFolder = HttpContext.Current.Server.MapPath("~/media");
            string videoFileFullPath = Path.Combine(mediaFolder, videofile) + extension;
            return File.OpenRead(videoFileFullPath);
        }

    }
}

The client side code will now retrieve a Base64Encoded string. The following client side scripts loads up our video:

 <script type="text/javascript">

        window.onload = init;
      
        var source; //Video buffer 

        function init() {
            loadByteArray('http://myserver/WcfStreamAudioDemo.Host/VideoService.svc/mediabytes/sintel_trailer-480p');
        }

        function base64ToArrayBuffer(base64) {
            var binaryString =  window.atob(base64);
            var len = binaryString.length;
            var bytes = new Uint8Array( len );
            for (var i = 0; i < len; i++)        {
                bytes[i] = binaryString.charCodeAt(i);
            }
            return bytes.buffer;
        }

        function loadByteArray(url) {
            
            var request = new XMLHttpRequest();

            //request.overrideMimeType('text\/plain; charset=x-user-defined');
            //request.responseType = 'blob';

            request.open('GET', url, true);
            //request.setRequestHeader("Content-Type", "video/mp4");


            request.onload = function() {
                //console.log(request.response);
                debugger;

                var responseData = request.response;
                if (responseData === undefined || responseData === null)
                    return;

                responseData = responseData.slice(0, -9);
                responseData = responseData.substring(68);

                var videoByteArrayView = base64ToArrayBuffer(responseData);
            
                var blob = new Blob([videoByteArrayView], { type: "text/plain;charset=utf-8" });

                var blobUrl = URL.createObjectURL(blob);

                var videoCtrlFedByByteArray = document.getElementById("videoCtrlFedByByteArray");

                videoCtrlFedByByteArray.setAttribute("src", blobUrl);


            } //request.onload 

            request.send();

        }

    </script>

There is still a lot of manual Js scripting here to prepare the video here, I will try to explain. We have now managed to drastically reduce the bloated data of our original 4.4 MB video down from 15+ MB to just 5.8 MB using Base64 encoded string. The retrieved Base64 encoded string is first decoded with the Js atob() function ("ASCII to BINARY"). A Uint8Array is initialized with the length of this decoded binary string and the binary string is iterated by using the charCodeAt method, returning the ArrayBuffer inside the constructed Uint8Array. Note that I got the Base64 encoded string as a XML first, which is why I do some chopping off the string using slice and substring. Anyways, after getting an ArrayBuffer from the Uint8Array's buffer property, we can construct a Blob object (Binary large object) and create an object url and then set the "src" attribute of our HTML5 Video control to this blobUrl. This is still not very elegant, I have seen examples using "arraybuffer" as responseType of the XmlHttpRequest object in Js to retrieve binary data, but I got garbled data back trying to use it with WCF REST. So in this article you have seen a way to send binary data from a WCF method by using a Base64 encoded string as an intermediary between the Server and the client. We got the file size of the request according to Fiddler inspection down to much less than the bloated binary array transfer that actually sent the byte array as a text string back to the client.

Wcf byte array + HTML 5 Video + Custombinding

Again, this article looks at different ways to load byte array from WCF representing video data to web clients, this approach will return a byte array and use a CustomBinding in WCF. You can start by cloning the repository I have prepared from here:
git clone https://toreaurstad@bitbucket.org/toreaurstad/wcfaudiostreamdemo.git git fetch && git checkout VideBase64String_05082018
The following method is added to our WCF service contract, note that now we leave the webHttpBinding and use a CustomBinding.

     [OperationContract]
     byte[] GetVideoBytes(string videofile);

There is no [WebGet] attribute this time, as noted we will not use WCF REST but SOAP instead. The web.config of the host website for the WCF services exposes this CustomBinding:

<system.serviceModel>

    <services>
      
      <service name="WcfStreamAudioDemo.Host.AudioService">
        <endpoint behaviorConfiguration="RestBehaviorConfig" binding="webHttpBinding" bindingConfiguration="HttpStreaming" contract="WcfStreamAudioDemo.Common.IAudioServiceContract" />
      </service>
      
      <service name="WcfStreamAudioDemo.Host.VideoService">
        <endpoint behaviorConfiguration="RestBehaviorConfig" binding="webHttpBinding" bindingConfiguration="HttpStreaming" contract="WcfStreamAudioDemo.Common.IVideoServiceContract" />
        <endpoint address="custom" binding="customBinding" bindingConfiguration="CustomBinding" contract="WcfStreamAudioDemo.Common.IVideoServiceContract" />
      </service> 

    </services>
    

    <bindings>
      
     <customBinding>
        <binding name="CustomBinding">
          <binaryMessageEncoding>
            <readerQuotas maxArrayLength="100000000" maxStringContentLength="100000000"/>
          </binaryMessageEncoding>
          <httpTransport />
        </binding>
      </customBinding>

      <webHttpBinding>
        <binding name="HttpStreaming" transferMode="Streamed" maxReceivedMessageSize="1000000000" maxBufferPoolSize="100000000">
          <readerQuotas maxArrayLength="100000000" maxStringContentLength="100000000"/>
        </binding>
      </webHttpBinding>
    </bindings>
    
    <behaviors>

      <serviceBehaviors>
        <behavior>
          <!-- To avoid disclosing metadata information, set the values below to false before deployment -->
          <serviceMetadata httpGetEnabled="true" httpsGetEnabled="true"/>
          <!-- To receive exception details in faults for debugging purposes, set the value below to true.  Set to false before deployment to avoid disclosing exception information -->
          <serviceDebug includeExceptionDetailInFaults="true"/>
        </behavior>
      </serviceBehaviors>
     
      <endpointBehaviors>
        <behavior name="RestBehaviorConfig">
          <webHttp />
        </behavior>
      </endpointBehaviors>

    </behaviors>
    <protocolMapping>
        <add binding="basicHttpsBinding" scheme="https" />
    </protocolMapping>    
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
  </system.serviceModel>

Note that the custom binding allows us to set up a binaryMessageEncoding. The next step is to go to the client project and add a service reference to the host project containing the WCF service. The app.config file is then updated, relevant parts shown here:

 <system.webServer>
    <directoryBrowse enabled="true" />
    <security>
      <requestFiltering>
        <requestLimits maxAllowedContentLength="1073741824" />
      </requestFiltering>
    </security>
        <staticContent>
            <mimeMap fileExtension=".mp4" mimeType="video/mp4" />
        </staticContent>
  </system.webServer>


  <system.serviceModel>
    <bindings>
      
      <customBinding>
        <binding name="CustomBinding">
          <binaryMessageEncoding>
            <readerQuotas maxArrayLength="100000000" maxStringContentLength="100000000" />
          </binaryMessageEncoding>
          <httpTransport maxReceivedMessageSize="100000000" />
        </binding>
      </customBinding>

    </bindings>
    <client>
      <endpoint address="http://he139920.helsemn.no/WcfStreamAudioDemo.Host/VideoService.svc/custom" contract="VideoService.IVideoServiceContract" binding="customBinding" bindingConfiguration="CustomBinding" />
    </client>
    
  </system.serviceModel>

Note the changes of setting up a MIME map for .mp4 files and setting up requestlimits. I will explain this soon. The client script now actually consists both of a server side script run in ASP.NET that sets up the video control in two ways. One way is to construct a HTML5 Data Url. This clearly was demanding for the browser to cope with and is not recommended. The video is actually embed on the page as a base64 encoded string! For a small video such as our video, it is actually possible still. It is of course fascinating that we can embed an entire video on our ASPX web page just as a Base64 encoded string, like it or not. The way to do this anyways is like this:

<script runat="server">

    private void btnInvokeWcfService_OnClick(object sender, EventArgs e)
    {

        using (var proxy = new VideoServiceContractClient())
        {
            byte[] payload = proxy.GetVideoBytes("sintel_trailer-480p");

            SetVideoSourceToHtmlDataUri(payload);

        }
    }


    private void SetVideoSourceToHtmlDataUri(byte[] payload)
    {
        //Set a base64 Html data uri

        videoCtrlFedByByteArrayThroughProxy.Attributes["type"] = "video/mp4";

        string base64String = Convert.ToBase64String(payload);

        videoCtrlFedByByteArrayThroughProxy.Attributes["src"] = "data:video/mp4;base64," + base64String;
    }

</script>

The following image shows how this actually works! Of course, this gives rendering and performance issues, as the web page now got very large content - the entire video is embedded into the page! Another way is to write the byte array to a temporary file on the server, and set the src attribute to this temporary file.

<script runat="server">

    private void btnInvokeWcfService_OnClick(object sender, EventArgs e)
    {

        using (var proxy = new VideoServiceContractClient())
        {
            byte[] payload = proxy.GetVideoBytes("sintel_trailer-480p");

            SetVideoSourceToTempFile(payload);

        }
    }

    private void SetVideoSourceToTempFile(byte[] payload)
    {
        //write to a temp file 
        string tempfile = Path.GetRandomFileName() + ".mp4";
        string tempfilePathForWebServer = HttpContext.Current.Server.MapPath("media/") + tempfile;
        File.WriteAllBytes(tempfilePathForWebServer, payload);

        videoCtrlFedByByteArrayThroughProxy.Attributes["src"] = "media/" + tempfile;
    }

</script>

Note that in these two samples, I have adjusted the HTML5 video control to be accessible to ASP.NET like this:

   <asp:Button runat="server" ID="btnInvokeWcfService" Text="Load Video" OnClick="btnInvokeWcfService_OnClick"/>
        
        <video id="videoCtrlFedByByteArrayThroughProxy" type="video/mp4"  runat="server"  controls width="320" height="240">         
            <p>Your browser doesn't support HTML5 video. Here is a <a href="http://wcfaudiodemohost.azurewebsites.net/VideoService.svc/mediabytes/sintel_trailer-480p">link to the video</a> instead.</p> 
        </video>

This method of retrieving video from WCF is the quickest way, it only fetches the original byte array data and it loads quickly. An adjusted version would not write to a file directly accessible on the web server, but use for example IsolatedStorage instead. I will look into that in the future. Hope you found this article interesting. CustomBindings in WCF gives you a lot of flexibility!