Showing posts with label web. Show all posts
Showing posts with label web. Show all posts

Sunday, 5 August 2018

Consuming video from WCF as a Base64 encoded strings

This article follows up our quest of loading up video byte arrays from WCF efficiently, but now the byte array is converted into a Base64 encoded string. You can start by cloning the repository I have prepared from here:
git clone https://toreaurstad@bitbucket.org/toreaurstad/wcfaudiostreamdemo.git git fetch && git checkout VideBase64String_05082018
Base64 encoded strings take six bits from the byte array and designates is as a char where the char can be one of 64 characters, in MIME implementation it is [A-z][0-9] and + and /, which is 64 different characters that the six bytes are encoded into. This means that 24 bits can be represented as three Base64 encodeded characters, that is 3 bytes in our byte array can be represented as FOUR base64 encoded chars. This 3:4 ratio is the rationale behind the MTOM optimization in WCF (to be discussed later). However, recap from the previous article where we downloaded a video sizing 4 MB on disk and Fiddler reporting it to be about 3 times larger. This is bloated data that I want to explore if we can fix up a bit. The bloated data is because I have managed not to truly enforce the XMLHttpRequest to send the data through the WCF REST binding (webHttpBinding) as true binary data - binary data is still sent as a string object to Javascript. This is sad, and I want to try to speed this up and avoid bloated data. To get this size down, we can start by using a Base64 encoded string instead in our XmlHttpRequest. First off, we define a service contract operation to return our string with Base64 data:
        [OperationContract]
        [WebGet(UriTemplate = "mediabytes/{videofile}")]
        string GetVideoAsBase64(string videofile);
Now our Service Implementation looks like this:

using System;
using System.IO;
using System.ServiceModel.Activation;
using System.ServiceModel.Web;
using System.Web;
using WcfStreamAudioDemo.Common;

namespace WcfStreamAudioDemo.Host
{

    [AspNetCompatibilityRequirements (RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
    public class VideoService : IVideoServiceContract
    {

        public Stream GetVideo(string videofile)
        {
            return GetVideoStream(videofile, ".mp4");
        }

        public string GetVideoAsBase64(string videofile)
        {
            Stream videoStream = GetVideoStream(videofile, ".mp4");

            byte[] buffer = new byte[32*1024];
            int count;
            var memoryStream = new MemoryStream();

            while ((count = videoStream.Read(buffer, 0, buffer.Length)) > 0)
            {
                memoryStream.Write(buffer, 0, count);
            }

            return Convert.ToBase64String(memoryStream.ToArray()); 
        }

        private Stream GetVideoStream(string videofile, string extension)
        {
            string mediaFolder = HttpContext.Current.Server.MapPath("~/media");
            string videoFileFullPath = Path.Combine(mediaFolder, videofile) + extension;
            return File.OpenRead(videoFileFullPath);
        }

    }
}


The client side code will now retrieve a Base64Encoded string. The following client side scripts loads up our video:

 <script type="text/javascript">

        window.onload = init;
      
        var source; //Video buffer 

        function init() {
            loadByteArray('http://myserver/WcfStreamAudioDemo.Host/VideoService.svc/mediabytes/sintel_trailer-480p');
        }

        function base64ToArrayBuffer(base64) {
            var binaryString =  window.atob(base64);
            var len = binaryString.length;
            var bytes = new Uint8Array( len );
            for (var i = 0; i < len; i++)        {
                bytes[i] = binaryString.charCodeAt(i);
            }
            return bytes.buffer;
        }

        function loadByteArray(url) {
            
            var request = new XMLHttpRequest();

            //request.overrideMimeType('text\/plain; charset=x-user-defined');
            //request.responseType = 'blob';

            request.open('GET', url, true);
            //request.setRequestHeader("Content-Type", "video/mp4");


            request.onload = function() {
                //console.log(request.response);
                debugger;

                var responseData = request.response;
                if (responseData === undefined || responseData === null)
                    return;

                responseData = responseData.slice(0, -9);
                responseData = responseData.substring(68);

                var videoByteArrayView = base64ToArrayBuffer(responseData);
            
                var blob = new Blob([videoByteArrayView], { type: "text/plain;charset=utf-8" });

                var blobUrl = URL.createObjectURL(blob);

                var videoCtrlFedByByteArray = document.getElementById("videoCtrlFedByByteArray");

                videoCtrlFedByByteArray.setAttribute("src", blobUrl);


            } //request.onload 

            request.send();

        }

    </script>

There is still a lot of manual Js scripting here to prepare the video here, I will try to explain. We have now managed to drastically reduce the bloated data of our original 4.4 MB video down from 15+ MB to just 5.8 MB using Base64 encoded string. The retrieved Base64 encoded string is first decoded with the Js atob() function ("ASCII to BINARY"). A Uint8Array is initialized with the length of this decoded binary string and the binary string is iterated by using the charCodeAt method, returning the ArrayBuffer inside the constructed Uint8Array. Note that I got the Base64 encoded string as a XML first, which is why I do some chopping off the string using slice and substring. Anyways, after getting an ArrayBuffer from the Uint8Array's buffer property, we can construct a Blob object (Binary large object) and create an object url and then set the "src" attribute of our HTML5 Video control to this blobUrl. This is still not very elegant, I have seen examples using "arraybuffer" as responseType of the XmlHttpRequest object in Js to retrieve binary data, but I got garbled data back trying to use it with WCF REST. So in this article you have seen a way to send binary data from a WCF method by using a Base64 encoded string as an intermediary between the Server and the client. We got the file size of the request according to Fiddler inspection down to much less than the bloated binary array transfer that actually sent the byte array as a text string back to the client.

Wcf byte array + HTML 5 Video + Custombinding

Again, this article looks at different ways to load byte array from WCF representing video data to web clients, this approach will return a byte array and use a CustomBinding in WCF. You can start by cloning the repository I have prepared from here:
git clone https://toreaurstad@bitbucket.org/toreaurstad/wcfaudiostreamdemo.git git fetch && git checkout VideBase64String_05082018
The following method is added to our WCF service contract, note that now we leave the webHttpBinding and use a CustomBinding.

     [OperationContract]
     byte[] GetVideoBytes(string videofile);

There is no [WebGet] attribute this time, as noted we will not use WCF REST but SOAP instead. The web.config of the host website for the WCF services exposes this CustomBinding:

<system.serviceModel>

    <services>
      
      <service name="WcfStreamAudioDemo.Host.AudioService">
        <endpoint behaviorConfiguration="RestBehaviorConfig" binding="webHttpBinding" bindingConfiguration="HttpStreaming" contract="WcfStreamAudioDemo.Common.IAudioServiceContract" />
      </service>
      
      <service name="WcfStreamAudioDemo.Host.VideoService">
        <endpoint behaviorConfiguration="RestBehaviorConfig" binding="webHttpBinding" bindingConfiguration="HttpStreaming" contract="WcfStreamAudioDemo.Common.IVideoServiceContract" />
        <endpoint address="custom" binding="customBinding" bindingConfiguration="CustomBinding" contract="WcfStreamAudioDemo.Common.IVideoServiceContract" />
      </service> 

    </services>
    

    <bindings>
      
     <customBinding>
        <binding name="CustomBinding">
          <binaryMessageEncoding>
            <readerQuotas maxArrayLength="100000000" maxStringContentLength="100000000"/>
          </binaryMessageEncoding>
          <httpTransport />
        </binding>
      </customBinding>

      <webHttpBinding>
        <binding name="HttpStreaming" transferMode="Streamed" maxReceivedMessageSize="1000000000" maxBufferPoolSize="100000000">
          <readerQuotas maxArrayLength="100000000" maxStringContentLength="100000000"/>
        </binding>
      </webHttpBinding>
    </bindings>
    
    <behaviors>

      <serviceBehaviors>
        <behavior>
          <!-- To avoid disclosing metadata information, set the values below to false before deployment -->
          <serviceMetadata httpGetEnabled="true" httpsGetEnabled="true"/>
          <!-- To receive exception details in faults for debugging purposes, set the value below to true.  Set to false before deployment to avoid disclosing exception information -->
          <serviceDebug includeExceptionDetailInFaults="true"/>
        </behavior>
      </serviceBehaviors>
     
      <endpointBehaviors>
        <behavior name="RestBehaviorConfig">
          <webHttp />
        </behavior>
      </endpointBehaviors>

    </behaviors>
    <protocolMapping>
        <add binding="basicHttpsBinding" scheme="https" />
    </protocolMapping>    
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
  </system.serviceModel>

Note that the custom binding allows us to set up a binaryMessageEncoding. The next step is to go to the client project and add a service reference to the host project containing the WCF service. The app.config file is then updated, relevant parts shown here:
 <system.webServer>
    <directoryBrowse enabled="true" />
    <security>
      <requestFiltering>
        <requestLimits maxAllowedContentLength="1073741824" />
      </requestFiltering>
    </security>
        <staticContent>
            <mimeMap fileExtension=".mp4" mimeType="video/mp4" />
        </staticContent>
  </system.webServer>


  <system.serviceModel>
    <bindings>
      
      <customBinding>
        <binding name="CustomBinding">
          <binaryMessageEncoding>
            <readerQuotas maxArrayLength="100000000" maxStringContentLength="100000000" />
          </binaryMessageEncoding>
          <httpTransport maxReceivedMessageSize="100000000" />
        </binding>
      </customBinding>

    </bindings>
    <client>
      <endpoint address="http://he139920.helsemn.no/WcfStreamAudioDemo.Host/VideoService.svc/custom" contract="VideoService.IVideoServiceContract" binding="customBinding" bindingConfiguration="CustomBinding" />
    </client>
    
  </system.serviceModel>
Note the changes of setting up a MIME map for .mp4 files and setting up requestlimits. I will explain this soon. The client script now actually consists both of a server side script run in ASP.NET that sets up the video control in two ways. One way is to construct a HTML5 Data Url. This clearly was demanding for the browser to cope with and is not recommended. The video is actually embed on the page as a base64 encoded string! For a small video such as our video, it is actually possible still. It is of course fascinating that we can embed an entire video on our ASPX web page just as a Base64 encoded string, like it or not. The way to do this anyways is like this:

<script runat="server">

    private void btnInvokeWcfService_OnClick(object sender, EventArgs e)
    {

        using (var proxy = new VideoServiceContractClient())
        {
            byte[] payload = proxy.GetVideoBytes("sintel_trailer-480p");

            SetVideoSourceToHtmlDataUri(payload);

        }
    }


    private void SetVideoSourceToHtmlDataUri(byte[] payload)
    {
        //Set a base64 Html data uri

        videoCtrlFedByByteArrayThroughProxy.Attributes["type"] = "video/mp4";

        string base64String = Convert.ToBase64String(payload);

        videoCtrlFedByByteArrayThroughProxy.Attributes["src"] = "data:video/mp4;base64," + base64String;
    }

</script>

The following image shows how this actually works! Of course, this gives rendering and performance issues, as the web page now got very large content - the entire video is embedded into the page! Another way is to write the byte array to a temporary file on the server, and set the src attribute to this temporary file.

<script runat="server">

    private void btnInvokeWcfService_OnClick(object sender, EventArgs e)
    {

        using (var proxy = new VideoServiceContractClient())
        {
            byte[] payload = proxy.GetVideoBytes("sintel_trailer-480p");

            SetVideoSourceToTempFile(payload);

        }
    }

    private void SetVideoSourceToTempFile(byte[] payload)
    {
        //write to a temp file 
        string tempfile = Path.GetRandomFileName() + ".mp4";
        string tempfilePathForWebServer = HttpContext.Current.Server.MapPath("media/") + tempfile;
        File.WriteAllBytes(tempfilePathForWebServer, payload);

        videoCtrlFedByByteArrayThroughProxy.Attributes["src"] = "media/" + tempfile;
    }

</script>

Note that in these two samples, I have adjusted the HTML5 video control to be accessible to ASP.NET like this:

   <asp:Button runat="server" ID="btnInvokeWcfService" Text="Load Video" OnClick="btnInvokeWcfService_OnClick"/>
        
        <video id="videoCtrlFedByByteArrayThroughProxy" type="video/mp4"  runat="server"  controls width="320" height="240">         
            <p>Your browser doesn't support HTML5 video. Here is a <a href="http://wcfaudiodemohost.azurewebsites.net/VideoService.svc/mediabytes/sintel_trailer-480p">link to the video</a> instead.</p> 
        </video>


This method of retrieving video from WCF is the quickest way, it only fetches the original byte array data and it loads quickly. An adjusted version would not write to a file directly accessible on the web server, but use for example IsolatedStorage instead. I will look into that in the future. Hope you found this article interesting. CustomBindings in WCF gives you a lot of flexibility!

Thursday, 2 August 2018

Streaming audio with WCF

This article will present a way to stream audio with WCF. In my example, I will play a Waveform Audio File Format file (.wav), but my sample should support any well-known audio formats. Note that WCF is not automatically the best streaming service to provide to your clients. Other protocols such as RTSP have better handling of graceful degradation, random seek and so on. This article will only show how you can provide a simple one-direction playback of audio files by using WCF. You can start by cloning the repository I have made public from here:

git@bitbucket.org:toreaurstad/wcfaudiostreamdemo.git

First off, we need to specify a content type of the method that returns a Stream with the audio. This can be achieved by implementing a class that inherits from System.Attribute and implementing IOperationBehavior. The class below sets a content type in its constructor and sets a Formatter of the DispatcherOperation in ApplyDispatchBehavior to the class ContentTypeMessageFormatter.


using System;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;
using System.ServiceModel.Dispatcher;


namespace WcfStreamAudioDemo.Common
{
    public class ContentTypeAttribute : Attribute, IOperationBehavior
    {
        public ContentTypeAttribute(string contentType)
        {
            ContentType = contentType;
        }

        public string ContentType { get; } //readonly auto property

        public void AddBindingParameters(OperationDescription operationDescription, BindingParameterCollection bindingParameters)
        {
          
        }

        public void ApplyClientBehavior(OperationDescription operationDescription, ClientOperation clientOperation)
        {
            
        }

        public void ApplyDispatchBehavior(OperationDescription operationDescription, DispatchOperation dispatchOperation)
        {
            // ReSharper disable once ArrangeThisQualifier
            dispatchOperation.Formatter = new ContentTypeMessageFormatter(dispatchOperation.Formatter, this.ContentType);
        }

        public void Validate(OperationDescription operationDescription)
        {
            
        }

      }
}

The ContentTypeMessageFormatter class is shown below:

using System.ServiceModel.Channels;
using System.ServiceModel.Dispatcher;
using System.ServiceModel.Web;
namespace WcfStreamAudioDemo.Common
{
    public class ContentTypeMessageFormatter : IDispatchMessageFormatter
    {

        private readonly IDispatchMessageFormatter _formatter;
        private readonly string _contentType;

        public ContentTypeMessageFormatter(IDispatchMessageFormatter formatter, string contentType)
        {
            _formatter = formatter;
            _contentType = contentType; 
        }

        public void DeserializeRequest(Message message, object[] parameters)
        {
            _formatter.DeserializeRequest(message, parameters);
        }

        public Message SerializeReply(MessageVersion messageVersion, object[] parameters, object result)
        {
            if (!string.IsNullOrEmpty(_contentType))
                WebOperationContext.Current.OutgoingResponse.ContentType = _contentType;
            return _formatter.SerializeReply(messageVersion, parameters, result);
        }

    }
}

Next step is to define a Service Contract in WCF to test this out:

using System.IO;
using System.ServiceModel;
using System.ServiceModel.Web;

namespace WcfStreamAudioDemo.Common
{

    [ServiceContract]
    public interface IAudioServiceContract
    {

        [OperationContract]
        [ContentType("audio/wav")]
        [WebGet(UriTemplate = "media/{trackName}")]
        Stream GetAudio(string trackName);
    }
}

The configuration of the WCF service in web.config sets up streaming by using transferMode set to Streamed on a webHttpBinding.

  <system.serviceModel>

    <services>
      
      <service name="WcfStreamAudioDemo.Host.AudioService">
        <endpoint behaviorConfiguration="RestBehaviorConfig" binding="webHttpBinding" bindingConfiguration="HttpStreaming" contract="WcfStreamAudioDemo.Common.IAudioServiceContract" />
      </service>
    </services>

    <bindings>
      <webHttpBinding>
        <binding name="HttpStreaming" transferMode="Streamed" maxReceivedMessageSize="1000000000" />
      </webHttpBinding>
    </bindings>
    
    <behaviors>

      <serviceBehaviors>
        <behavior>
          <!-- To avoid disclosing metadata information, set the values below to false before deployment -->
          <serviceMetadata httpGetEnabled="true" httpsGetEnabled="true"/>
          <!-- To receive exception details in faults for debugging purposes, set the value below to true.  Set to false before deployment to avoid disclosing exception information -->
          <serviceDebug includeExceptionDetailInFaults="false"/>
        </behavior>
      </serviceBehaviors>
     
      <endpointBehaviors>
        <behavior name="RestBehaviorConfig">
          <webHttp />
        </behavior>
      </endpointBehaviors>

    </behaviors>
    <protocolMapping>
        <add binding="basicHttpsBinding" scheme="https" />
    </protocolMapping>    
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
  </system.serviceModel>


Finally, to test out this Audio Service you can either enter the url of an audio file the WCF service recognizes into a browser to download the .WAV file or for example use an HTML 5 <audio> element. Here is a sample page I created to test out the Audio service (a simple ASP.NET Web form)!

<%@ Page Language="C#" %>

<script runat="server">

</script>

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title>Demo Audio Stream wcf</title>
</head>
<body>
<form id="HtmlForm" runat="server">
    <div>
        
        <h3>WCF Sample Audio streaming example</h3>
        
        <p>Playing an audio file (.wav) using a custom WCF IDispatchMessageFormatter with REST (webHttpBinding).</p>
        
        <audio controls>
            <source src="http://localhost/WcfStreamAudioDemo.Host/AudioService.svc/media/Sample" type="audio/wav" />
            <p>Your browser doesn't support HTML5 audio. Here is a <a href="http://localhost/WcfStreamAudioDemo.Host/AudioService.svc/media/Sample.wav">link to the audio</a> instead.</p> 
        </audio>

    </div>
</form>
</body>
</html>