MEF and memory

For those of you that are coding in .NET today, chances are high that you are doing it in using composition through usually a container and using Inversion of Control (IoC). Many will use the Managed Extensibility Framework or MEF, for achieving IoC. MEF is not entirely a true IoC container, as the framework supports non-specific import and export of interfaces bound to implementation(s) using inheritors of ComposablePartCatalog such as AssemblyCatalog.

To download versions of MEF, visit the site http://mef.codeplex.com However, most of us will use the stable version of MEF available in .NET 4, situated in the System.ComponentModel.Composition namespace.

MEF is very easy to get started with and the concepts of importing and exporting parts are really great.
However, there is big flaw in MEF which is well known to its users and contributors, which is the way MEF handles memory. Without getting too much into detail, MEF will generally keep a reference to all parts in its container as long as the application or system is running. This results in a high and observable memory leak. To take an example, I have put together a downloadable solution at the end of this article which shows a way to avoid the memory leak. The test solution will be explained throughout this article with the most relevant parts underlined.

First off, the base of this test solution is not mine. I needed to get a sample of a WPF/MVVM/MEF/Prism solution where I could test out the concepts at home and googled for some simple demo project. I found one at the top, Prism 4 MEF application. It is available at the following URL:

Prism 4 MEF application:
http://www.codeproject.com/Articles/254120/Prism-4-MEF-Application

Code Project Open License terms:
http://www.codeproject.com/info/cpol10.aspx

I will understate that my sample is based on the solution written by Danny Siew from Wellington, New Zealand. Thanks to Danny Siew I could focus on finding a solution of the memory leak problem in MEF.

To verify that the memory leak was fixed with my solution, I downloaded the ANTS Memory Profiler from Red Gate. A trial can be downloaded at the following URL:
http://www.red-gate.com/products/dotnet-development/ants-memory-profiler/

Using the ANTS Memory Profiler, one can quickly identity the memory leak through MEF by profiling your application that is using MEF and witness that parts that are removed and is no longer used in the application are kept alive (that is, not disposed so memory resources can become available again). My main concern are the parts that are marked as CreationPolicy.NonShared. The other parts are usually the singletons, with CreationPolicy.Shared. These parts will have at most one instance and should live throughout the lifetime of the application and not become disposed before the container is disposed. It is though a huge issue that those parts that are marked as NonShared will not be dispoed before the container is dispose, because this means that the number of instances will be accumulated in memory. Clearly, not a desired behavior of MEF!

My solution is actually based upon the following discussion at the following thread on the MEF Codeplex site:
http://mef.codeplex.com/discussions/285445

Thanks to the user Palomino for this great tip:
http://www.codeplex.com/site/users/view/palomino

The strategy of Palomino is to create a CustomCatalog inheriting from the usual abstract base class ComposablePartCatalog in MEF. This class will detect when a part is requiring disposing. I have modified Palomino's code slightly, as it will deactivate the non-desired caching of the part in the CatalogExportProvider MEF uses if the part is marked as CreationPolicy.NonShared. This is how I mean the correct behavior should be. The singletons or shared parts should be disposed in the usual "MEF-way", that is when the container is disposed. The non-shared parts should be disposed when parts go out of scope for early release of resources. In no circumstances should MEF keep caching non-shared parts, as the chances for memory leak is 100%. The project manager of MEF, Glenn Block has created ExportFactory classes that are able to dispose parts on demand. This will be available in .NET 4.5 and a port has been made available at Glenn's Skydrive for download (consisting of a single DLL file) where the inherited MEF Bootstrapper can be adjusted to allow this dispose on demand. I spent some time testing out this ExportFactory class but using ANTS Memory Profiler I could not see the memory leak go away..

It is time present some code for how to avoid this memory leak problem in ANTS, since any real application or system out there using MEF today will have this huge memory leak problem and cause degraded performance and larger and larger memory footprint as the application is running.

Lets start off with this new custom catalog, for any reason of better naming, I decided to change as little of Palomino's code, so let's just call it CustomCatalog here too:

  public class CustomCatalog : ComposablePartCatalog
    {
        private readonly ComposablePartCatalog _innerCatalog;

        public CustomCatalog(ComposablePartCatalog catalog)
        {
            _innerCatalog = catalog;
        }

        public override IEnumerable<Tuple<ComposablePartDefinition, ExportDefinition>> GetExports(ImportDefinition definition)
        {
            var exports = _innerCatalog.GetExports(definition);
            return exports.Select(e => new Tuple<ComposablePartDefinition, ExportDefinition>(
                new CustomComposablePartDefinition(e.Item1), e.Item2));
        }

        public override IQueryable<ComposablePartDefinition> Parts
        {
            get
            {
                return _innerCatalog.Parts
                    .Select(p => new CustomComposablePartDefinition(p))
                    .AsQueryable();
            }
        }
    }

To explain this class in short terms, we pass in an inheritor of the abstract ComposablePartCatalog, such as AssemblyCatalog, in the constructor. Then we override the method GetExports and readonly property Parts. Instead of just returning the implementation of the passed in ComposablePartCatalog inheritor instance, a wrapping CustomComposablePartDefinition is used. This class will be presented next:

    public class CustomComposablePartDefinition : ComposablePartDefinition
    {
        private readonly ComposablePartDefinition _innerPartDefinition;

        public CustomComposablePartDefinition(ComposablePartDefinition innerPartDefinition)
        {
            _innerPartDefinition = innerPartDefinition;
        }

        public override ComposablePart CreatePart()
        {
            var innerPart = _innerPartDefinition.CreatePart();
            if (ReflectionModelServices.IsDisposalRequired(_innerPartDefinition) && 
                _innerPartDefinition.Metadata.ContainsKey(CompositionConstants.PartCreationPolicyMetadataName) && 
                (CreationPolicy)_innerPartDefinition.Metadata[CompositionConstants.PartCreationPolicyMetadataName] == CreationPolicy.NonShared)
            {
                return new NotDisposableComposablePart(innerPart);
            }
            return innerPart;
        }

        public override IEnumerable<ExportDefinition> ExportDefinitions
        {
            get { return _innerPartDefinition.ExportDefinitions; }
        }

        public override IEnumerable<ImportDefinition> ImportDefinitions
        {
            get { return _innerPartDefinition.ImportDefinitions; }
        }

        public override IDictionary<string, object> Metadata
        {
            get { return _innerPartDefinition.Metadata; }
        }

        public override string ToString()
        {
            return _innerPartDefinition.ToString();
        }
    }

The class CustomComposablePartDefinition shown above inherits from the usual ComposablePartDefinition and provides a custom functionlity for the case where the part is a NonShared part and requires disposing, which usually means the part implements IDisposable. This is the source of the memory leak, i.e. where MEF will keep its cached reference to the part in the CatalogExportProvider and stop the Garbace Collector (GC) from being able to dispose the part on a timely manner. The code for the NotDisposableComposablePart follows:

public class NotDisposableComposablePart : ComposablePart
    {

        private ComposablePart _composablePart; 

        public NotDisposableComposablePart(ComposablePart composablePart)
        {
            _composablePart = composablePart; 
        }

        public override IEnumerable<ExportDefinition> ExportDefinitions
        {
            get { return _composablePart.ExportDefinitions; }
        }

        public override object GetExportedValue(ExportDefinition definition)
        {
            return _composablePart.GetExportedValue(definition); 
        }

        public override IEnumerable<ImportDefinition> ImportDefinitions
        {
            get { return _composablePart.ImportDefinitions; }
        }

        public override void SetImport(ImportDefinition definition, IEnumerable<Export> exports)
        {
            _composablePart.SetImport(definition, exports); 
        }
    }

The NotDisposableComposablePart inherits from the base class ComposablePart. It does not implement IDisposable and will not therefore be cached by the ExportCatalogProvider of MEF. Hence, the memory leak would not occur anymore where the container keeps reference. By using the ANTS Memory Profiler I can verify this behavior. An example how of to adjust an AssemblyCatalog in the MEF Bootstrapper follows, using this CustomCatalog:

           var assemblyCatalog = new AssemblyCatalog(typeof(ItemViewModel).Assembly);
            this.AggregateCatalog.Catalogs.Add(new CustomCatalog(assemblyCatalog)); 

This wrapping of the CustomCatalog can be done not only for AssemblyCatalog and TypeCatalog and so on. Wherever a memory leak is shown in large scale in an application using MEF, the wrapping where one uses CustomCatalog is strongly recommended. The following image shows an image from the sample application, which I have adjusted some from Danny's sample on Code Project:

Here is a screenshot of the memory profiling application I use, ANTS Memory Profiler from Red Gate:

To evaluate this memory leak fix of MEF yourself, the sample solution is available for download at the following URL: Prism 4 MEF Sample with memory leak fix [Zip file, 4 MB download]


It is strange that Microsoft choose to have an instance retention policy where the disposing of parts are postponed until the disposing of the container, which is usually done when the application is exited. This actually allows a huge memory leak, especially for the NonShared parts. It is usual in say a WPF/MVVM/Prism/MEF scenario to add and remove view models and other types of instances and maintaining these parts in memory allows a great access to memory leak in a multitude of applications. If you use MEF today, do not be suprised that there are a lot of memory leaks in your managed coded application. Just because it is .NET and managed code does not mean memory leaks are not present, on the contrary side. Thanks to the great tips of Palomino, user at MEF Codeplex, and Danny Siew for providing an easy and great Prism 4 MEF starting solution. Evaluate the code. Make note that the necessary Prism and MEF DLL-s are added. In addition to .NET 4 (obviously), download Microsoft WPF Ribbon 4.0 from MSDN here:


Microsoft Ribbon 4.0 WPF I have compiled the solution in Visual Studio 2012 RC so you will find binaries in the bin folders also to start off the sample right away. Hopefully, this fix will patch MEF until MEF 2 is released. After reading a bit more about MEF 2, it seems they use ExportFactory and the early release or dispose of objects will be an opt in. It will be easier to wrap the different catalogs used with this CustomCatalog to at least try to fix the massive memory leak that MEF almost provides as default behavior. Feel free to comment this article if you have supplementary information. I have used MEF quite a lot in the last two years, but I have less deep knowledge of MEF. I wrote this article because I had to inspect if there really were workarounds of this central bug of MEF.. Happy coding!