Loading HTML5 Audio from byte array in WCF

Update:

Live demo!

Wcf Audio Demo - HTML 5 and Byte array!

The previous article presented a solution where the user can retrieve data from a WCF operation that returns a Stream. This approach is great, since the user can quickly load data and progressively retrieve it. This makes an efficient use of bandwidth and avoids latency. However, there is a downside to this approach. It is not possible to skip parts of the loaded Stream, so the user can only start from the beginning of the Stream. The article presented loading a Waveform Audio Format file (audio/wav). Another approach is instead to let the user play, rewind and skip to parts of the data, be it a video or audio file for example. This article will present a solution of this and also solve it in memory by returning a byte array from WCF and load it into the WebAudio AudioContext of a webpage and then set the src of a HTML5 Audio element. First off, you can clone my repository which I have prepared from Bitbucket and switch to the branch supporting a byte array.

 git clone https://toreaurstad@bitbucket.org/toreaurstad/wcfaudiostreamdemo.git
 git checkout ByteArrayWise_03082018

Let's first return a byte array to the client in a WCF service contract. We will return a byte array of a sample ogg/vorbis (mime: audio/ogg) file. The WCF Service contract now got a new method GetAudioBytes :

    [ServiceContract]
    public interface IAudioServiceContract
    {

        [OperationContract]
        [ContentType("audio/ogg")]
        [WebGet(UriTemplate = "media/{trackName}")]
        Stream GetAudio(string trackName);

        [OperationContract]
        [WebGet(UriTemplate = "mediabytes/{trackName}", ResponseFormat = WebMessageFormat.Json)]
        byte[] GetAudioBytes(string trackName);

    }

Next off, we implement the method GetAudioBytes :

        private static Stream GetStream(string trackName, string fileExtension)
        {
            var mediaDirectory = HttpContext.Current.Server.MapPath("~/media/");
            string track = string.Format("{0}.{1}", trackName, fileExtension);
            string filePath = Path.Combine(mediaDirectory, track);
            return File.OpenRead(filePath);
        }

        public byte[] GetAudioBytes(string trackName)
        {
            Stream track = GetStream(trackName, "ogg");

            long streamLengh = track.Length;
            byte[] buffer = new byte[32*1024];
            int count;

            var memoryStream = new MemoryStream();
            while ((count = track.Read(buffer, 0, buffer.Length)) > 0)
            {
                memoryStream.Write(buffer, 0, count);
            }

            return memoryStream.ToArray();
        }

The reading of the Stream into a byte array is done via a memory stream with a buffer of size 32 kB. I wrote the method above and checked with one of Jon Skeet's advice and it was identical :) e We now have got a byte array, then the client must get this byte array. Since I decided to create a web client, it is necessary to make use of a client library. I will use HTML 5 audio and WebAudio API for this. First off, we make an XMLHttpRequest to fetch the data, we set the responseType of the request to 'arraybuffer'. I encountered trouble in Javascript sending the byte array from WCF and ended up to force Js to send the data as plain text raw format. Then I had to manually build an arraybuffer by using an Uint8Array as a view to do the actual insertion. There is a lot of manual work done here to ultimately populate the ArrayBuffer with exact same bytes as returned from the WCF service. As I said, by first trying a manual retrieval, I ended up with bloated data that I had a to bake into shape by forcing a plain retrieval of data and then molding the data into an ArrayBuffer. Not very elegant, but it works.

    
    <script src="Scripts/FileSaver.js" type="text/javascript"></script>
    
    <script type="text/javascript">

        window.onload = init;

        var context; //AudioContext
        var source; //Audio buffer 

        function init() {
            if (!window.AudioContext) {
                alert("Your browser does not support any Audio Context and cannot play back this audio");
                return;
            }
            window.AudioContext = window.AudioContext || window.webkitAudioContext;

            context =  typeof AudioContext !== 'undefined' ? new AudioContext() : new webkitAudioContext();;

            loadByteArray('http://myserver/WcfStreamAudioDemo.Host/AudioService.svc/mediabytes/ACDC_-_Back_In_Black-sample');
        }

        function loadByteArray(url) {
            source = context.createBufferSource();
            var request = new XMLHttpRequest();
           
            request.overrideMimeType('text\/plain; charset=x-user-defined');
            //request.responseType = 'arraybuffer';
            request.open('GET', url, true);
            //request.setRequestHeader("Content-Type", "audio/ogg");


            request.onload = function() {
                //console.log(request.response);
                debugger;

                var audioByteArray = request.response.replace('[', '').replace(']','').split(',');
                source = new ArrayBuffer(audioByteArray.length);

                var audioByteArrayView = new Uint8Array(source);


                for (var i = 0; i < audioByteArray.length; i++) {
                    audioByteArrayView[i] = audioByteArray[i];
                }

                var blob = new Blob([audioByteArrayView], { type: "text/plain;charset=utf-8" });

                var blobUrl = URL.createObjectURL(blob);

                var audioCtrlFedByByteArray = document.getElementById("audioCtrlFedByByteArray");

                audioCtrlFedByByteArray.setAttribute("src", blobUrl);



                saveAs(blob, "ExampleReceived.ogg");
             
                context.decodeAudioData(source,
                    function (buffer) {
                        //Play off the audio file automatically
                        var innersource = context.createBufferSource();
                        innersource.buffer = buffer;
                        innersource.connect(context.destination);

                        innersource.start(0);

                      
                        //Hook up the HTML 5 audio control using MediaElementSource connecting to the AudioContext
                        var audioCtrlFedByByteArray = document.getElementById("audioCtrlFedByByteArray");
  
                        var mediaSource = context.createMediaElementSource(audioCtrlFedByByteArray);
                        mediaSource.connect(context.destination);
                        audioCtrlFedByByteArray.play();

                    },
                    function(e) {
                        console.log("Error with decoding audio data" + e.err);
                    }
            );

        }

            request.send();

        }

    </script>

There are three techniques presented into the client-side script. First off I show how you automtically can play downloaded audio into a web page! I use the AudioContext first and decodeAudioData by creating a bufferSource and connect that source to the AudioContext's destination (i.e. speakers). Then i choose start(0) of the bufferSource to automatically playback! Also demonstrated in the Javascript above is how I grab hold of the audio control and then setting the src attribute to a blobUrl created by URL.createObjectUrl feeding it a Blob (Binary large object) of the created Uint8Array populated by the raw data. This will set dynamically the source of the HTML 5 Audio media element on the page and let the user finally play the audio file and rewind or skip to different parts! It is much more user friendly, but the user will have to download the entire file first and the manual process by molding the retrieved raw data into a working ArrayBuffer with correct data will be slow when dealing with large files. The web page of the client defines the audio controls like this:

rm id="HtmlForm" runat="server">
    <div>
        
        <h3>WCF Sample Audio streaming example (<strong>System.IO.Stream</strong>)</h3>
        
        <p>Playing an audio file (.wav) using a custom WCF IDispatchMessageFormatter with REST (webHttpBinding).</p>
        
        <audio controls>
            <source src="http://myserver/WcfStreamAudioDemo.Host/AudioService.svc/media/Example" type="audio/ogg" />
            <p>Your browser doesn't support HTML5 audio. Here is a <a href="http://myserver/WcfStreamAudioDemo.Host/AudioService.svc/media/Example">link to the audio</a> instead.</p> 
        </audio>
        
        <h3>WCF Sample Audio streaming example (<strong>byte[] array</strong>)</h3>
        
        <p>Playing an audio file (.wav) using a custom WCF IDispatchMessageFormatter with REST (webHttpBinding).</p>
        
        <audio id="audioCtrlFedByByteArray" controls>         
            <p>Your browser doesn't support HTML5 audio. Here is a <a href="http://myserver/WcfStreamAudioDemo.Host/AudioService.svc/mediabytes/ACDC_-_Back_In_Black-sample">link to the audio</a> instead.</p> 
        </audio>
        
    </div>
    

Make note that my sample page also includes the library FileSaver to download the ogg sample file. This library is quite handy and is available in Nuget (package id: filesaver). We finally have a way to load data from WCF into a HTML 5 Audio control in memory! This means we could for example have audio tracks saved into a database and fetch them into memory and return them to our clients. An alternative is of course to just return a FileStream and have a server backend written in ASP.NET MVC Core, but then you will also be missing a lot of the nice features of WCF! All in all, it is always nice to know alternatives before sticking to the mainstream solutions. I hope you found this article interesting.