Map and Reduce in PLinq

This article will present some code that can be used to apply a Map and a Reduce transformation of a provided input array of a given type of objects, using PLINQ. For developers familiar with LINQ, PLINQ is the parallel version of Linq. First, the source code will be presented, then the code will be explained.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace TestParallelMap
{
    class Program
    {

        static void Main(string[] args)
        {
            var nums = Enumerable.Range(0, 1001).ToArray(); 
            Func<int,TaggedResult<int,double>> mapFunction = n => new TaggedResult<int,double>(n, Math.Pow(n, 2)); 
            var result = ParallelMap(nums, mapFunction);

            foreach (var r in result)
            {
                Console.WriteLine("Got result {0} for the num {1}", r.OutputValue, r.InputtedValue);
            }

            Console.WriteLine("Press the any key to continue ...");
            Console.ReadKey();

            var resultSecond = ParallelReduce(nums, 0, (first, second) => first + second);

            Console.WriteLine("Got result {0}", resultSecond);  
         
            Console.WriteLine("Press the any key to continue ...");
            Console.ReadKey();

        }

        private static TValue ParallelReduce<TValue>(TValue[] source, TValue seedValue, Func<TValue, TValue, TValue> reduceFunction)
        {
            return source.AsParallel().AsOrdered().Aggregate(
                seedValue,
                (localResult, nextValue) => reduceFunction(localResult, nextValue),
                (overallResult, localResult) => reduceFunction(overallResult, localResult), 
                 overallResult => overallResult); 
                  
        }

        private static TOutput[] ParallelMap<TInput, TOutput>(TInput[] source, Func<TInput, TOutput> mapFunction) 
        {
            return source.AsParallel().AsOrdered().Select(x => mapFunction(x)).ToArray(); 
        }

        private class TaggedResult<TInput, TOutput>
        {
            public TaggedResult(TInput inputtedValue, TOutput outputtedValue)
            {
                InputtedValue = inputtedValue;
                OutputValue = outputtedValue; 
            }

            public TInput InputtedValue { get; set; }

            public TOutput OutputValue { get; set; }
        }

    }
}

The source code presented above is a simple console application. It is written in C# (obviously) and uses PLINQ to prosess the input array. The Map method will transform each value in the input array with a provided mapfunction. The source code uses Func to provide a function pointer to the method that will be executed to produce each item. The generic arguments control the type of the resulting array:

        private static TOutput[] ParallelMap<TInput, TOutput>(TInput[] source, Func<TInput, TOutput> mapFunction) 
        {
            return source.AsParallel().AsOrdered().Select(x => mapFunction(x)).ToArray(); 
        }

Note that PLINQ is very similar to LINQ and the main difference is the AsParallel() method. Also note that AsOrdered() will mean that the ordering will be maintained in the output array. If the order does not matter, there is a performance gain by omitting the call to AsOrdered().. Further, note that one uses the Select method of PLINQ to perform a projection where one at the same time performes the map method. The Reduce method is a bit more complicated. It uses PLINQ's Aggregate method, which really is a bit tough of understanding. Let's repeat the source code:

  private static TValue ParallelReduce<TValue>(TValue[] source, TValue seedValue, Func<TValue, TValue, TValue> reduceFunction)
        {
            return source.AsParallel().AsOrdered().Aggregate(
                seedValue,
                (localResult, nextValue) => reduceFunction(localResult, nextValue),
                (overallResult, localResult) => reduceFunction(overallResult, localResult), 
                 overallResult => overallResult); 
                  
        }

Actually, the PLINQ aggregate method is very versatile and powerful method when you want to process an entire array and perform a reduction method, in our case just a summation, but this could be anything, for example statistical methods and so on. The first argument is the seed value, the next parameter is a lambda expression that will perform the stepwise part of the reduction, the next parameter is the lambda expression that merges the stepwise result with the overall result - and the final fourth parameter is just a simple lambda which tells we do not want to do anything with the final overall result than propagate the value to the processed final result which then is returned. There are many more patterns that are a bit more complicated too, like MapReduce (which combines map and reduce), but this will be a relatively gentle introduction to the functionality that PLINQ presents. It is impressive to think that all the thread handling will be automatically be handled by PLINQ. From relatively simple code one can perform a multitude of different transformation from this source code with Map and Reduce, powered by PLINQ and generics.

Tracking Prism EventAggregator traffic

Tracking Prism Events

It is possible to track the Prism events that are sent and received through the Prism EventAggregator,
but this is not out of the box of the Prism Framework. Usually, the developer must insert breakpoints
where the CompositePresentationEvent is published and subscribed. 

This is satisfactory for small scenarios, but as your solution and project(s) grow, it can become an increasingly difficult task. It is in many cases desired to see which events are published and where they are subscribed. The Prism 4.x Framework still do not support this, but luckily there is a way to achieve this. 

First off, it is required to change your Prism Events from using the class CompositePresentationEvent, to instead use a derived class presented next, CustomCompositePresentationEvent. I have chosen to display the Prism Event traffic in the Output of Visual Studio. This is done using Debug.WriteLine. It could be perhaps an alternative to log this event traffic or display it in a separate Window if you are writing a WPF application. 

The custom class CustomCompositePresentationEvent looks like this: 

using System;
using System.ComponentModel;
using System.Diagnostics;
using System.Text;
using Microsoft.Practices.Prism.Events;

namespace EventAggregation.Infrastructure
{
    
    /// <summary>
    /// Custom CompositePresentationEvent class, derived from this base class to allow the Prism Events to be displayed in the Debug Output Window. 
    /// </summary>
    public class CustomCompositePresentationEvent<TEvent> : 
    CompositePresentationEvent<TEvent> where TEvent : class 
    {

        /// <summary>
        /// Overload of the publish method of CompositePresentationEvent 
        /// </summary>
        public override void Publish(TEvent payload)
        {
            var declaringType = string.Empty; 
            var callingMethod = new StackTrace().GetFrame(1).GetMethod(); 
            if (callingMethod != null)
                declaringType = callingMethod.DeclaringType.FullName; 
            System.Diagnostics.Debug.WriteLine(string.Format(@"Publishing event {0}
            at time: {1}. Source type: {2}. Event Payload: {3}", 
            typeof(TEvent).Name,DateTime.Now, declaringType, 
            GetContentAsString(payload))); 
            base.Publish(payload);
        }

        /// <summary>Overload of the Subscribe method of  CompositePresentationEvent</summary>
        /// <remarks>
        /// For this overload to be executed, set keepSubscriberReferenceAlive to 
        /// true in the subscribe setup method on the target. 
        /// </remarks>
        public override SubscriptionToken Subscribe(Action<TEvent> action,
        ThreadOption threadOption, bool keepSubscriberReferenceAlive, 
        Predicate<TEvent> filter)
        {

            var loggingAction = new Action<TEvent>(e =>
            {
                System.Diagnostics.Debug.WriteLine(string.Format(
                "Subscribing event {0} at time: {1}. Target type: {2}", 
                typeof(TEvent).Name,
                    DateTime.Now, action.Target.GetType().Name));

                action.Invoke(e);
            });

            var subscriptionToken = base.Subscribe(loggingAction, threadOption, 
            keepSubscriberReferenceAlive, filter);
            return subscriptionToken;
        }


        private string GetContentAsString(TEvent payload)
        {
            if (payload == null)
                return string.Empty;
            else
            {
                StringBuilder sb = new StringBuilder();
                foreach (PropertyDescriptor property in 
                 TypeDescriptor.GetProperties(payload))
                {                    
                    sb.Append(string.Format("{0}={1} ", property.Name, 
                    property.GetValue(payload))); 
                }
                return sb.ToString();                 
            }
        }

    }

}

As you can see from the source code above, the methods Subscribe and Publish are overriden. These are virtual methods of the CompositePresentationEvent base class. I have tested using this derived class from CompositePresentationEvent in the EventAggregator sample, which can be downloaded in the Prism 4.1 library with source code at the following url: Download Prism 4.1 You will find the EventAggregator sample in the \QuickStarts\EventAggregation folder after you have extracted the Prism 4.1 library with source code. I have done some minor changes to this demo solution. To the project EventAggregation.Infrastructure.Desktop, I added the class I made which was just presented, CustomCompositePresentationEvent. Next up, our Prism Events must use this CustomCompositePresentation event class! This is easy to change, as shown in the Prism Event FundAddedEvent, which I changed to the following, still in the project EventAggregation.Infrastructure.Desktop:

namespace EventAggregation.Infrastructure
{
    public class FundAddedEvent : CustomCompositePresentationEvent
    {
    }
}

As you can see, the FundAddedEvent Prism Event, is now inheriting from the class CustomCompositePresentationEvent. There are no changes needed to do for the way the FundAddedEvent is published, but the Subscribe method must be changed. The boolean flag keepSubscriberReferenceAlive must be set to true. The reason is that the Subscribe override method in the CustomCompositePresentationEvent requires this, as you can see in the source code. We modify the action sent back again to the base class to continue with its Subscribe implementation. I could not make the override method to work, i.e. the method override was not called, without setting the boolean flag keepSubscribeReferenceAlive to true. The change I had to make for the subscribe method of the FundAddedEvent is in the project ModuleB.Desktop, in the class ActivityPresenter. The line 67 was changed to this:

  subscriptionToken = fundAddedEvent.Subscribe(FundAddedEventHandler,  
  ThreadOption.UIThread, true, FundOrderFilter); 

The changes required then, is to change the Prism Events to inherit from the class CompositePresentationEvent to instead inherit from (use) the class CustomCompositePresentationEvent. This custom class is basically the CompositePresentationEvent class, but we have overridden the methods Subscribe and Publish. In addition, we must set the keepSubscriberReferenceAlive boolean flag to true in our Subscribe methods. This will potentially mean that you must change to the Subscribe overload method which has this flag. Let's first see the GUI of the EventAggregation sample:







The output from the Visual Studio is the most important feature of this CustomCompositePresentationEvent class. After all, we want to be able to see which classes publish and subscribe events, and the content of the payload of these events, correct? In this sample, the output looks like the following example:

Publishing event FundOrder at time: 18.11.2012 15:31:30. Source type: ModuleA.AddFundPresenter. Event Payload: CustomerId=Customer1 TickerSymbol=FundB 
Subscribing event FundOrder at time: 18.11.2012 15:31:30. Target type: ActivityPresenter

As we can see from the output above, the event FundOrder was published by the AddFundPresenter class. The PayLoad of this Prism event was CustomerId=Customer1 and the TickerSymbol=FundB. The event FundOrder was subscribed, interestingly at the same millisecond (i.e. the publish-subscribe execution obviously performs almost instantly in the Prism framework). If you think the concept of being able to see Prism traffic is both interesting and will help you as a developer, consider taking into to use this CompositePresentationEvent class. You will have to include the changes which I listed above also, but these changes should be relatively easy to overcome in a Prism application solution. The tracking or tracing of Prism Event Traffic will help you understand the application better and catch possible errors by this more convenient debugging scenario. For security reasons, you should consider if putting a Debug.WriteLine call to output the Event Payload is safe or not. Last, I want to mention the fact that Prism Publish-Subscribe scenarios take in my test 0 ms and occur to be happening almost instantly. Of course they take some CPU cycles to execute, but if you in any case wonder how fast is really a Publish-Subscribe, it will usually be the answer - REALLY FAST. However, it is possible that if multiple classes subscribe to a given Prism Event, it might take some milliseconds to execute. I have not tested this yet. In a MVVM Scenario, it is usually the ViewModels that publish and subscribe Prism Events, usually via an ICommand implementation (DelegateCommand) that is data bound in WPF or Silverlight to a Button or similar GUI object.

User-Defined conversion sample in C#

This article will present a sample of how to implement a User Defined Conversions in C#. The article will use a typical conversion example, where temperature measured in Celsius scale (Centigrades) are converted into the Fahrenheit scale (Fahrenheits). A conversion formula is available at the following urls (Wikipedia):

• Wikipedia entry on Fahrenheit 
• Wikipedia entry on Celsius

User Defined Conversions are implemented in C-sharp in a similar manner as operator overloading, actually it is an overload of conversion. These conversions can be either:

• Implicit - An object is converted into another type without a specific cast, i.e implicit cast.
• Explicit - An object is converted into another type with a specific cast, i.e.
  explicit cast.

A class can have multiple implicit and/or explicit user defined conversions. The user defined conversions must all be methods that are marked as public and static. In addition, they must be marked as implicit or explicit to specify which cast is allowed. If a conversion is marked as explicit, converting an object into the specified type without an explicit cast will in other words not call the user defined conversion. 

To specify a User-Defined Conversion in C# which is implicit, the following signature must be used for a implicit User-Defined Conversion:

public static implicit operator TReturnType (TInputType objectToConvert) {

 .. //Implementation elided 

A User-Defined Conversion in C# which is explicit, uses the following signature: 

public static explicit operator TReturnType (TInputType objectToConvert) {

 .. //Implementation elided 

The C-sharp programmer who has experience in operator overloading will see the similarities to operator overloading. Operator overloading follows much the same signature as shown here, but without the keywords explicit or implicit. In addition, operator overloading will specify the same keyword operator, but also specify the operator to overload. User-Defined Conversion do not specify an operator to overload, but implicitly or explicitly we are overloading casting (type conversion). 

Demonstration code 

An example of how to implement User-Defined Conversion follows. In this example, a base class called Temperature is used and there are two derived classes, Celsius and Fahrenheit. Both these derived classes defines an explicit user defined conversion which will take an instance of their own type and return an instance of the converted type. This means that the Fahrenheit class will contain an explicit User-Defined Conversion which will receive a Fahrenheit instance and return a Celsius instance. In return, the Celsius class will contain an explicit User-Defined Conversion which will receive a Celsius instance and convert this into a Fahrenheit instance. In the code, the mathematical conversion between the two temperature scales is performed inside the explicit user defined conversion. This will let the programmer control the way a cast is performed and allow the client or consumer of the class to convert the type into another type through a cast and logically fill the expectation that the converted object is correctly converted or casted. 

As with operator overloading, just because C-sharp allows User Defined Conversions, does not mean that the programmer should use this functionality in the C#-language wherever this strategy might fit. User-Defined Conversions are a programming capability many C#-programmers are not aware of, and the coder which adopt this strategy might confuse his fellow programmers or quickly understand that the concepts must be explained to the other coders. But then again, just as operator overloading allows very logical and powerful expressibility in code, user defined conversion will let the programmer control the way explicit and implicit type conversions, that is - casts - are performed. If you expect the class you are writing to be casted into different types or will provide consumers or clients of your class with the benefits of simple expressibility and determined behavior, you should also add explicit user defined conversions. Implicit User-Defined Conversions are sometimes hard to see which User-Defined Conversion will be called. Therefore, if a choice is available between either implicit or explicit User-Defined Conversion, I would suggest to go for explicit. This is the most specific and clear-intentioned casting overload of the two. 

The code for the base class Temperature is shown below:     

    /// Base class for temperatures 
    /// 
    public class Temperature
    {
        protected float _value; 

        public Temperature(float temperature)
        {
            _value = temperature; 
        }

        public float Value
        {
            get { return _value; } 
        }

    }

The derived class Celsius is shown next. Note that this class defines one explicit User-Defined conversion: 

    public class Celsius : Temperature 
    {

        public Celsius(float temperature) : base(temperature)
        {
        }

        public static explicit operator Fahrenheit(Celsius temperature)
        {
            float convertedTemperature = (temperature.Value * (9.0f / 5.0f)) + 32; 
            return new Fahrenheit(convertedTemperature); 
        }

    }

The derived class Fahrenheit is shown below. This class also defines one explicit User-Defined conversion. 

    public class Fahrenheit : Temperature
    {

        public Fahrenheit(float temperature) : base(temperature)
        {
            
        }

        public static explicit operator Celsius(Fahrenheit temperature)
        {
            float convertedTemperature = (5.0f / 9.0f)*(temperature.Value - 32);
            return new Celsius(convertedTemperature); 
        }

    }

The client or consumer class is shown next:

class Program
    {

        static void Main(string[] args)
        {
            Console.WriteLine("Centigrades to Fahrenheits conversion in range [-40,40] Celsius: ");

            Celsius celsius = null;
            Fahrenheit fahrenheit = null;

            for (int i = -40; i <= 40; i++)
            {
                celsius = new Celsius(i);
                fahrenheit = (Fahrenheit)celsius;
                Console.WriteLine("{0} Centigrades is measured into {1:f1} Fahrenheits.", i, fahrenheit.Value);
            }

            Console.WriteLine("Press any key to continue ...");
            Console.ReadKey();
        }

    }

The output in the console window when running the Main method in the consumer class 
above is shown next: 

Centigrades to Fahrenheits conversion in range [-40,40] Celsius:
-40 Centigrades is measured into -40,0 Fahrenheits.
-39 Centigrades is measured into -38,2 Fahrenheits.
-38 Centigrades is measured into -36,4 Fahrenheits.
-37 Centigrades is measured into -34,6 Fahrenheits.
-36 Centigrades is measured into -32,8 Fahrenheits.
-35 Centigrades is measured into -31,0 Fahrenheits.
-34 Centigrades is measured into -29,2 Fahrenheits.
-33 Centigrades is measured into -27,4 Fahrenheits.
-32 Centigrades is measured into -25,6 Fahrenheits.
-31 Centigrades is measured into -23,8 Fahrenheits.
-30 Centigrades is measured into -22,0 Fahrenheits.
-29 Centigrades is measured into -20,2 Fahrenheits.
-28 Centigrades is measured into -18,4 Fahrenheits.
-27 Centigrades is measured into -16,6 Fahrenheits.
-26 Centigrades is measured into -14,8 Fahrenheits.
-25 Centigrades is measured into -13,0 Fahrenheits.
-24 Centigrades is measured into -11,2 Fahrenheits.
-23 Centigrades is measured into -9,4 Fahrenheits.
-22 Centigrades is measured into -7,6 Fahrenheits.
-21 Centigrades is measured into -5,8 Fahrenheits.
-20 Centigrades is measured into -4,0 Fahrenheits.
-19 Centigrades is measured into -2,2 Fahrenheits.
-18 Centigrades is measured into -0,4 Fahrenheits.
-17 Centigrades is measured into 1,4 Fahrenheits.
-16 Centigrades is measured into 3,2 Fahrenheits.
-15 Centigrades is measured into 5,0 Fahrenheits.
-14 Centigrades is measured into 6,8 Fahrenheits.
-13 Centigrades is measured into 8,6 Fahrenheits.
-12 Centigrades is measured into 10,4 Fahrenheits.
-11 Centigrades is measured into 12,2 Fahrenheits.
-10 Centigrades is measured into 14,0 Fahrenheits.
-9 Centigrades is measured into 15,8 Fahrenheits.
-8 Centigrades is measured into 17,6 Fahrenheits.
-7 Centigrades is measured into 19,4 Fahrenheits.
-6 Centigrades is measured into 21,2 Fahrenheits.
-5 Centigrades is measured into 23,0 Fahrenheits.
-4 Centigrades is measured into 24,8 Fahrenheits.
-3 Centigrades is measured into 26,6 Fahrenheits.
-2 Centigrades is measured into 28,4 Fahrenheits.
-1 Centigrades is measured into 30,2 Fahrenheits.
0 Centigrades is measured into 32,0 Fahrenheits.
1 Centigrades is measured into 33,8 Fahrenheits.
2 Centigrades is measured into 35,6 Fahrenheits.
3 Centigrades is measured into 37,4 Fahrenheits.
4 Centigrades is measured into 39,2 Fahrenheits.
5 Centigrades is measured into 41,0 Fahrenheits.
6 Centigrades is measured into 42,8 Fahrenheits.
7 Centigrades is measured into 44,6 Fahrenheits.
8 Centigrades is measured into 46,4 Fahrenheits.
9 Centigrades is measured into 48,2 Fahrenheits.
10 Centigrades is measured into 50,0 Fahrenheits.
11 Centigrades is measured into 51,8 Fahrenheits.
12 Centigrades is measured into 53,6 Fahrenheits.
13 Centigrades is measured into 55,4 Fahrenheits.
14 Centigrades is measured into 57,2 Fahrenheits.
15 Centigrades is measured into 59,0 Fahrenheits.
16 Centigrades is measured into 60,8 Fahrenheits.
17 Centigrades is measured into 62,6 Fahrenheits.
18 Centigrades is measured into 64,4 Fahrenheits.
19 Centigrades is measured into 66,2 Fahrenheits.
20 Centigrades is measured into 68,0 Fahrenheits.
21 Centigrades is measured into 69,8 Fahrenheits.
22 Centigrades is measured into 71,6 Fahrenheits.
23 Centigrades is measured into 73,4 Fahrenheits.
24 Centigrades is measured into 75,2 Fahrenheits.
25 Centigrades is measured into 77,0 Fahrenheits.
26 Centigrades is measured into 78,8 Fahrenheits.
27 Centigrades is measured into 80,6 Fahrenheits.
28 Centigrades is measured into 82,4 Fahrenheits.
29 Centigrades is measured into 84,2 Fahrenheits.
30 Centigrades is measured into 86,0 Fahrenheits.
31 Centigrades is measured into 87,8 Fahrenheits.
32 Centigrades is measured into 89,6 Fahrenheits.
33 Centigrades is measured into 91,4 Fahrenheits.
34 Centigrades is measured into 93,2 Fahrenheits.
35 Centigrades is measured into 95,0 Fahrenheits.
36 Centigrades is measured into 96,8 Fahrenheits.
37 Centigrades is measured into 98,6 Fahrenheits.
38 Centigrades is measured into 100,4 Fahrenheits.
39 Centigrades is measured into 102,2 Fahrenheits.
40 Centigrades is measured into 104,0 Fahrenheits.

Press any key to continue ...

Most Europeans are familiar with the Centigrade system, but US-English citizens use Fahrenheit scale instead for describing the temperature. The freezing point of water, 0 degrees Celsius, is 32 Fahrenheit and about -17,78 Centigrades is 0 Fahrenheit. The boiling point of water, 100 degrees Centigrades, is 212 degrees Fahrenheit. As we can see, the temperature scales grow differently and are shifted somewhat. The User-Defined conversion allows to convert the two inheriting classes of the temperature base class through explicit casts. Some, but not all programmers, believe this is a natural way of performing a conversion. Often, one can see code which actually does the same kind of functionality but does not use User-Defined conversion, i.e. a public static method which receives an instance of type TInput and returns a converted object of type TResult, but without the keywords implicit or explicit and operator. This is of course allowed, but note that this will demand that the user specifically calls this method to perform the conversion routine to get the converted instance. Instead, specifying an implicit and/or expliciting User-Defined Conversion will in many cases be less ambigious and more precise. In the end, judge yourself if User-Defined Conversions are something to use in C#. But be aware that they exist and what they can be used for. More explanation on this topic available on the MSDN page:
User Defined Conversions
If no explicit cast is performed using the cast operator (parentheses), the implicit User Defined Conversion will be called, if both a matching explicit and implicit User Defined Conversion is defined. Note that if the assignment operator = is used without a cast, the implicit conversion is used and therefore the implicit User Defined Conversion will be utilized.