Wednesday, 31 July 2019

Consistency guard of enums used in Entity Framework

This is a consistency guard for enums in Entity Framework. It is a mechanism for protecting an entity in Entity Framework or just EF, in case an enum value was loaded from the database with an illeagal value. An illeagal enum value would be any value of an enum that cannot be parsed into an enum. We use Enum.IsDefined method (at first running Convert.ChangeType) to check if the value for the enum is leagal or not. We define a helper class BrokenEnumValue to contain our metadata about enum values that are illeagal or 'broken'. The rest of the code in this article goes into the DbContext class (Or ObjectContext would also work) that EF uses. The ObjectMaterialized event is added in the constructor for example.
            var objectContext = ((IObjectContextAdapter) this).ObjectContext;
            _log = (ILog) AutofacHostFactory.Container.Resolve(typeof(ILog));
            objectContext.ObjectMaterialized += ObjectContext_ObjectMaterialized;
Our helper POCO:

    public class BrokenEnumValue
    {
        public string PropertyName { get; set; }
        public string PropertyTypeName { get; set; }
        public Guid? SchemaGuid { get; set; }
        public string OldValue { get; set; }
        public string CorrectedValue { get; set; }

        public override string ToString()
        {
            return $"{PropertyName} {PropertyTypeName} {SchemaGuid} {OldValue} {CorrectedValue}";
        }
    }



        private void ObjectContext_ObjectMaterialized(object sender, ObjectMaterializedEventArgs e)
        {
            var brokenEnumProperties = FixBrokenEnumProperties(e.Entity);
            if (brokenEnumProperties.Any())
            {
                Type objType = e.Entity.GetType();
                var idProperty = objType.GetProperty("Id");
                Guid? schemaGuid = idProperty?.GetValue(e.Entity, null) as Guid?;
                foreach (var brokenEnum in brokenEnumProperties)
                    brokenEnum.SchemaGuid = schemaGuid;
                string brokenEnumsInfo = string.Join(" ", brokenEnumProperties.Select(b => b.ToString()).ToArray());
                _log.WriteWarning($"Detected broken enum propert(ies) in entity and resolved them to default value if available in enum (None): {brokenEnumsInfo}");
            }
        }

           public IList<BrokenEnumValue> FixBrokenEnumProperties(object obj)
        {
            var list = new List<BrokenEnumValue>();
            try
            {
                if (obj == null) return list;
       
                PropertyInfo[] properties = obj.GetType().GetProperties();
                foreach (PropertyInfo property in properties)
                {
                    if (property.GetIndexParameters()?.Any() == true)
                        continue; //skip indexer properties
                    if (property.PropertyType.IsArray)
                    {
                        Array a = (Array) property.GetValue(obj);
                        for (int i = 0; i < a.Length; i++)
                        {
                            object o = a.GetValue(i);
                            list.AddRange(FixBrokenEnumProperties(o));
                        }
                        continue; //continue to next iteration
                    }
                    object propValue = property.GetValue(obj, null);
                    var elems = propValue as IList;
                    if (elems != null)
                    {
                        foreach (var item in elems)
                        {
                            list.AddRange(FixBrokenEnumProperties(item));
                        }
                    }
                    else
                    {
                        if (property.PropertyType.IsEnum && !IsEnumDefined(propValue, property.PropertyType))
                        {
                            var correctedValue = GetDefaultEnumValue(propValue, property.PropertyType);
                            list.Add(new BrokenEnumValue
                            {
                                CorrectedValue = correctedValue?.ToString(),
                                OldValue = propValue?.ToString(),
                                PropertyName = property.Name,
                                PropertyTypeName = property.PropertyType.FullName,
                            });
                            property.SetValue(obj, correctedValue);
                            
                        }
                        if (property.PropertyType.IsClass && (property.PropertyType.GetCustomAttributes(typeof(DataContractAttribute))?.Any() == true)
                                                          && !(property.PropertyType == typeof(string)) && !property.PropertyType.IsValueType)
                        {
                            list.AddRange(FixBrokenEnumProperties(propValue));
                        }
                    }
                }
            }
            catch (Exception err)
            {
                _somelog.WriteError($"Expection occurred trying to fix broken enum properties: {err}");
            }
            return list;
        }

           private static T GetDefaultEnumValue<T>(T entity, Type propertyType)
           {
               foreach (var enumValue in propertyType.GetEnumValues())
               {
                   if (String.Compare(enumValue.ToString(), "None", StringComparison.OrdinalIgnoreCase) == 0)
                   {
                       return (T)enumValue;
                   }
               }
               return entity;
           }

        private static bool IsEnumDefined(object entity, Type propertyType)
        {
            var castedValue = Convert.ChangeType(entity, propertyType);
            return Enum.IsDefined(propertyType, castedValue);
        }


With this guard, we can avoid that the entity does not load in case an illeagal value was loaded for a given enum. Note that our fallback is looking for the enum value mapping to the [None] enum member, so we fallback to the [None] enum value, if it exists. Mosts enum should have a [None] member mapping to the enum integer value 0. You can of course adjust the strategy used here. I believe such a consistency guard would be helpful for many applications using EF.

Tuesday, 30 July 2019

Slide-in animation of forms in Angular 8

This is a cool feature in Angular 8, the support of animations. First off, we define our animation in a Typescript file like for example a sliding in translation of the X-axis. app-animation.ts
import { trigger, animate, transition, style, group, query } from '@angular/animations';

export const slideInAnimation = trigger('slideInAnimation', [
  // Transition between any two states
  transition('* <=> *', [
    // Events to apply
    // Defined style and animation function to apply
    // Config object with optional set to true to handle when element not yet added to the DOM
    query(':enter, :leave', style({ position: 'fixed', width: '100%', zIndex: 2 }), { optional: true }),
    // group block executes in parallel
    group([
      query(':enter', [
        style({ transform: 'translateX(100%)' }),
        animate('0.5s ease-out', style({ transform: 'translateX(0%)' }))
      ], { optional: true }),
      query(':leave', [
        style({ transform: 'translateX(0%)' }),
        animate('0.5s ease-out', style({ transform: 'translateX(-100%)' }))
      ], { optional: true })
    ])
  ])
]);


In our app.component.ts, we import this animation slideInAnimation like this:
import { slideInAnimation } from './app.animation';
@Component({
  selector: 'pm-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css'],
  animations: [ slideInAnimation ]
})
export class AppComponent {
  pageTitle = 'SomeAcme Product Management';


Note the association of the slideInAnimation above. And in the app component template (app.component.html) we just declare a template variable for the router outlet and set it up to trigger on route activation.
<div class="container" [@slideInAnimation]="o.isActivated ? o.activatedRoute: ''">
  <router-outlet #o="outlet"></router-outlet>
</div>
This video shows the effect. Note that this is an excerpt from Deborah Kurata's course on Angular routing from Pluralsight. Video of this technique:

Sunday, 28 July 2019

Angular 8 - Datepicker and timepicker

I have been looking at the ngx-bootstrap components for Angular lately. These components are very nice and similar to the components that Bootstrap provides, for Angular. I have tested it out with Angular 8. First off, we import the datepicker and time picker of this library like this: ng add ngx-bootstrap --components datepicker ng add ngx-bootstrap --components timepicker Inside your app.module.ts, you should see the two components imported:
import { BsDatepickerModule } from "ngx-bootstrap/datepicker";
import { TimepickerModule } from 'ngx-bootstrap/timepicker';

It is also included to the imports. My sample projects app.module.ts looks like this:
import { BrowserModule } from "@angular/platform-browser";
import { FormsModule } from "@angular/forms";
import { NgModule } from "@angular/core";
import { HttpClientModule } from "@angular/common/http";
import { AppComponent } from "./app.component";
import { UserSettingsFormComponent } from "./user-settings-form/user-settings-form.component";
import { AppFocusDirective } from "./data/app-focus.directive";
import { ButtonsModule } from "ngx-bootstrap/buttons";
import { BsDatepickerModule } from "ngx-bootstrap/datepicker";
import { BsLocaleService } from 'ngx-bootstrap/datepicker';
import { BrowserAnimationsModule } from "@angular/platform-browser/animations";
import { defineLocale } from "ngx-bootstrap/chronos";
import { nbLocale } from "ngx-bootstrap/locale";
import { TimepickerModule, TimepickerConfig } from 'ngx-bootstrap/timepicker';
import { getTimepickerConfig  } from './TimepickerConfig';

defineLocale("nb", nbLocale);

@NgModule({
  declarations: [AppComponent, UserSettingsFormComponent, AppFocusDirective],
  imports: [
    BrowserModule,
    FormsModule,
    HttpClientModule,
    ButtonsModule.forRoot(),
    BsDatepickerModule.forRoot(),
    BrowserAnimationsModule,
    TimepickerModule.forRoot()
  ],
  providers: [ BsLocaleService,
   { provide: TimepickerConfig, useFactory: getTimepickerConfig }
  ],
  bootstrap: [AppComponent]
})
export class AppModule {

  constructor(private bsLocaleService: BsLocaleService) {
    this.bsLocaleService.use('nb');
  }
}

Observe the localization adjustments I have included here, for Norwegian localization of the date picker. I also include a custom factory function for setting up the time picker to not show the meridian button, i.e. show a 24 hours format of the time.
import { TimepickerConfig } from "ngx-bootstrap/timepicker";

// such override allows to keep some initial values

export function getTimepickerConfig(): TimepickerConfig {
  return Object.assign(new TimepickerConfig(), {
    hourStep: 2,
    minuteStep: 5,
    showMeridian: false,
    readonlyInput: false,
    mousewheel: true,
    showMinutes: true,
    showSeconds: false
  });
}


If you want to test out the adjustments I made, the Github repo is available here: https://github.com/toreaurstadboss/Angular8FormsNgxBootstrapTesting

Monday, 22 July 2019

HttpInterceptor to cache clientside in Angular 8

I have been working with a sample project of HttpInterceptor to cache data clientside in Angular 8 hosted on Github. The repository is here: https://github.com/toreaurstadboss/AngularHttpInterceptorSample First off, the data is delivered by a small backend using Express.Js.The server.js file looks like this:
const express = require('express');
const path = require('path');
const fs = require('fs');
var favicon = require('serve-favicon');
var logger = require('morgan');
var cookieParser = require('cookie-parser');
var bodyParser = require('body-parser');

var towns = require('./server/routes/towns');

var app = express();

// view engine setup
app.set('views', path.join(__dirname, '/server/views'));
app.set('view engine', 'jade');

app.use(favicon(path.join(__dirname, 'dist/favicon.ico')));
app.use(logger('dev'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cookieParser());
app.use(express.static(path.join(__dirname, 'dist')));

app.use(function (req, res, next) {
    if (!req.url.startsWith('/api')){
      req.url = '/api' + req.url;
    }
    console.log('got requst url: ' + req.url);
    console.log('Server handling request at Time:', new Date())
    next()
  })

// app.use('/', routes);
app.get('/api/towns', function(req, res) {
    let townsFileContents = fs.readFileSync(path.join(__dirname, 'server/data/townsandcities.json'), 'utf-8');
    let townsFound = JSON.parse(townsFileContents);
    console.log(townsFileContents);
    res.json(townsFound);
});

// app.get('*', function(req, res) {
//   res.sendFile(path.join(__dirname, 'dist/index.html'));
// });

// catch 404 and forward to error handler
// app.use(function(req, res, next) {
//     var err = new Error('Not Found');
//     err.status = 404;
//     next(err);
// });

// error handlers

// development error handler
// will print stacktrace
if (app.get('env') === 'development') {
    app.use(function(err, req, res, next) {
        res.status(err.status || 500);
        res.render('error', {
            message: err.message,
            error: err
        });
    });
}

// production error handler
// no stacktraces leaked to user
app.use(function(err, req, res, next) {
    res.status(err.status || 500);
    res.render('error', {
        message: err.message,
        error: {}
    });
});

var debug = require('debug')('server');

app.set('port', process.env.PORT || 3000);

app.listen(app.get('port'));

console.log('Listening on port: ' + app.get('port'));

module.exports = app;

Angular bits is written in Typescript instead of just Js. The Http Interceptor itself is a ES 6 class which is exported and caches http responses according to route urls.
import { HttpInterceptor, HttpRequest, HttpHandler, HttpEvent, HttpResponse } from '@angular/common/http';
import { Observable, of } from 'rxjs';
import { tap } from 'rxjs/operators';
import { CacheService } from './cache.service';


export class CacheInterceptor implements HttpInterceptor {

    constructor(private cacheService: CacheService) {
    }

    intercept(req: HttpRequest<any>, next: HttpHandler): Observable<HttpEvent<any>> {


        if (req.method !== 'GET') {
            this.cacheService.invalidateCache();
        }
        const cachedResponse: HttpResponse<any> = this.cacheService.get(req.url);
        if (cachedResponse !== undefined) {
         return of(cachedResponse);
        }

        return next.handle(req).pipe(
            tap(event => {
                if (event instanceof HttpResponse) {
                    this.cacheService.put(req.url, event);
                    let reponse: HttpResponse<any>;
                    reponse  = event;

                }
                return of(event);
            })
        );
    }

}

The CacheService caches responses to the client by route url as mentioned. It looks like this:
import { Injectable } from '@angular/core';
import { ICacheservice } from './icacheservice';
import { HttpResponse } from '@angular/common/http';

@Injectable({
  providedIn: 'root'
})
export class CacheService implements ICacheservice {
  private requests = { };
  put(url: string, response: HttpResponse): void {
    this.requests[url] = response;
  }
  get(url: string): HttpResponse | undefined {
    return this.requests[url] ? this.requests[url] : undefined;
  }
  invalidateCache(): void {
    this.requests = {};
  }
  expireItem(url: string, timeoutInMilliseconds: number) {
    const itemFound = this.get(url);
    if (itemFound !== undefined) {
      setTimeout(() => {
        this.requests[url] = undefined;
      }, timeoutInMilliseconds);
    }

  }

  constructor() { }

}

We inject the HttpInterceptor to the app module like this:
import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';
import { AppComponent } from './app.component';
import { CacheInterceptor } from './core/cache.interceptor';
import { HTTP_INTERCEPTORS, HttpClientModule } from '@angular/common/http';
import { BrowserAnimationsModule } from '@angular/platform-browser/animations';
import { MatTableModule } from  '@angular/material';

@NgModule({
  declarations: [
    AppComponent
  ],
  imports: [
    BrowserModule,
    HttpClientModule,
    MatTableModule,
    BrowserAnimationsModule
  ],
  providers: [
    { provide: HTTP_INTERCEPTORS, useClass: CacheInterceptor, multi: true }
  ],
  bootstrap: [AppComponent]
})
export class AppModule { }

The index page then contains some markup including a Material Table using Google's Material UI. The app.component.ts handles the client side call back that triggers the retrieval and clearing of the data. If you click the load link, the data is retrieved and cached. If you open up Developer tools with F12, you will spot that the data is cached if you hit load many times. If you hit clear link, the data is cleared from cache and next click on load will fetch data again and inject into the cache. This sample shows how you can create a cache side feature of Angular 8 and support an HTTP interceptor. To check it out yourself run: git clone https://github.com/toreaurstadboss/AngularHttpInterceptorSample.git Afterwards, run: npm run start This will start both the Angular website on port 4200 and the Node.js Express.js backend on port 3000. The package.json script for npm run start looks like this: "start": "concurrently --kill-others \"ng build --watch --no-delete-output-path\" \"nodemon server.js\" \"ng serve --proxy-config proxy.conf.json \"", You can also just enter: npm start

Monday, 15 July 2019

Http echo service written in Node.js

I have added a small repo on Github that shows how to create a simple HTTP echo service. https://github.com/toreaurstadboss/NodeJsHttpEchoServer The server.js is script is this:
let http = require('http');

console.log('Attempting to start the Node.js echos server on port 8081..')
http.createServer().on('request', function(request, response){
 request.pipe(response);
}).on('close', function(){
    console.log('The Node.js echo server has been closed.')
})
.listen(8081);

console.log('Node.js echo server started on port 8081.')

This will pipe out the requesting data to the response. Test it out with this command for example in a command window: curl -d "Hello dog" http://localhost:8081 Note - use not Powershell for this command as Powershell has aliased curl command for Invoke-WebRequest. Rather install curl with Chocolatey for example, choco install curl

Sunday, 14 July 2019

Using Ndepend to investigate method cycles

Ndepend is a very powerful Code analysis platform for DotNet and I have created a method cycle detection rule. The method cycle detection also supports property setters and getters. A cycle in a property or method will often cause problems, such as stack overflow exceptions due to too many recursive calls. Spotting such cycles can often be hard in real-world scenarios. A classic error is to actually do a cycle through a property getter, by just calling the getter once more or a property setter for that instance. The following C# attribute invokes a Rules extracted from Source code from Ndepend.

using System;
using NDepend.Attributes;

namespace Hemit.OpPlan.Client.Infrastructure.Aspects
{

    /// <summary>
    /// Ndepend attribute to enable cyclic loops in methods in the source code 
    /// </summary>
    [CodeRule(@"// <Name>Avoid methods of a type to be in cycles. Detect also recursive property setter calls</Name>
warnif count > 0

from t in Application.Types
                 .Where(t => t.ContainsMethodDependencyCycle != null && 
                             t.ContainsMethodDependencyCycle.Value)

// Optimization: restreint methods set
// A method involved in a cycle necessarily have a null Level.
let methodsSuspect = t.Methods.Where(m => m.Level == null)

// hashset is used to avoid iterating again on methods already caught in a cycle.
let hashset = new HashSet<IMethod>()


from suspect in methodsSuspect
   // By commenting this line, the query matches all methods involved in a cycle.
   where !hashset.Contains(suspect)

   // Define 2 code metrics
   // - Methods depth of is using indirectly the suspect method.
   // - Methods depth of is used by the suspect method indirectly.
   // Note: for direct usage the depth is equal to 1.
   let methodsUserDepth = methodsSuspect.DepthOfIsUsing(suspect)
   let methodsUsedDepth = methodsSuspect.DepthOfIsUsedBy(suspect)

   // Select methods that are both using and used by methodSuspect
   let usersAndUsed = from n in methodsSuspect where 
                         methodsUserDepth[n] > 0 && 
                         methodsUsedDepth[n] > 0 
                      select n

   where usersAndUsed.Count() > 0

   // Here we've found method(s) both using and used by the suspect method.
   // A cycle involving the suspect method is found!
   let cycle = ExtensionMethodsEnumerable.Append(usersAndUsed,suspect)


   // Fill hashset with methods in the cycle.
   // .ToArray() is needed to force the iterating process.
   let unused1 = (from n in cycle let unused2 = hashset.Add(n) select n).ToArray()

let cycleContainsSetter = (from n1 in cycle where n1.IsPropertySetter select n1).Count()

  
select new { suspect, cycle, cycleContainsSetter }",
        Active = true,
        DisplayStatInReport = true,
        DisplayListInReport = true,
        DisplaySelectionViewInReport = true,
        IsCriticalRule = false)]
    public class MethodCycleDetectionAttribute : Attribute
    {
    }
}


Note that this Ndepend Code rule uses the CQLinq syntax placed into a C# attribute class that inherits from Attribute and itself is attributed with a Ndepend.Attributes.CodeRuleAttribute. I had to adjust the attribute a little bit using the ExtensionMethodsEnumerable fix for .Net 4.6.2 mentioned on Ndepend blog post here: https://blog.ndepend.com/problem-extension-methods/ The following screen shot shows Visual Studio 2019 with Ndepend version 2019.2.5 installed. It shows the method cycle detection code rule I created showing up as a code rule through source code. Selecting that code rule in the menu Ndepend => Rules Explorer that selects Queries and Rules Explorer allows to not only view the code rule but quickly see the code analysis results.
One method cycle I have is shown in the following image, the method ProcessPasRequest of my system. This case shows how advanced Ndepend really is in code analysis. It can detect transitive cycles with multiple function calls. The following two screen shots shows the cycle in question. In this case, it was not a bug, since the cycle will only happend a finite amount of times for a given set of patients, but it still can go into an infinite loop if the external system is not working correct. However in this case, if that external system is not working, the system I have been working with also faults as it relies on that external system. With Ndepend I could spot method cycles with transitive cycles with a breeze! I can also spot property getter and setter cycles, as property getters and setters in C# are methods anyways (compiler generated). I managed to get Ndepend back to the main menu in Visual Studio 2019 using this extension.
https://marketplace.visualstudio.com/items?itemName=Evgeny.RestoreExtensions

Loading tweets from Twitter with Node.Js and Express.Js

This article will present a Git repository I created that can fetch tweets from Twitter using Twitter Api v1.1 and Node and Express. Or also known as Node.js and Express.Js. First off, Twitter now enforces all API calls to be authenticated. To be able to call the Twitter API, there are four values you must retrieve for the App you have created at https://developer.twitter.com/en/apps. These are:
  • Consumer API
  • Consumer API secret
  • Access token
  • Access token secret
The Git repo I created is available here: https://github.com/toreaurstadboss/SimpleTwitterFeedExpressJs The solution is started using: npm run start But for working with the solution, use nodemon command. The server.js is the primary part. Here we create the Express.Js server object for Node which will host our web service calls for the Tweets. We use the following Npm libraries:
  • express (web server)
  • fs (FileStream for Node
  • https (for secured communication
  • Twitter (for Twitter API v1.1 communication
The following Js script is the server.js file:
var express = require("express");
var request = require("request");
let config = require("./config");
const https = require("https");
const fs = require("fs");
var url = require("url");
var Twitter = require("twitter");

var key = fs.readFileSync(__dirname + "/certs/selfsigned.key");
var cert = fs.readFileSync(__dirname + "/certs/selfsigned.crt");
const port = 3000;
var options = {
  key: key,
  cert: cert
};

var app = express();

var client = new Twitter(config);

app.get("/", function(req, res) {
  res.sendFile(__dirname + "/index.html");
});

function getTweets(userName, tweetCount, res) {
  var params = {
    q: userName,
    count: tweetCount
  };

  var responseFromTwitter = {};

  let tweetSearched = false;

  // Initiate your search using the above paramaters
  client.get("search/tweets", params, function(err, data, response) {
    console.log("Found # Tweets: " + data.statuses.length);
    res.charset = "utf-8";
    res.append("Content-Type", "application/json; charset=utf-8");

    // If there is no error, proceed
    if (!err) {
      res.end(JSON.stringify(data.statuses));
    }
  });
}

app.get("/tweets/:username", function(req, res) {
  let tweetsFound = getTweets(req.params.username, 10, res);
  console.log(tweetsFound);
});

app.use("/css", express.static(__dirname + "/node_modules/bootstrap/dist/css"));

var server = https.createServer(options, app);
server.listen(port, () => {
  console.log("server starting on port: " + port);
});
Make note that this sample uses HTTPS communication. It is using self signed certificates. I created these using this command inside Windows Subsystem for Linux: sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./selfsigned.key -out selfsigned.crt This created the key pair of the selfsigned key and the certificate. The HTTPS server is created using this:
var server = https.createServer(options, app);
server.listen(port, () => {
  console.log("server starting on port: " + port);
});
The bulk of the code retrieving the Tweets is the method getTweets. We pass on the response object and use the end method of Express to stringify the JSON data we found. I have not done much error handling here. We return data as Json if we found some Tweets for given username. There is one route defined in Express here: app.get("/tweets/:username", function(req, res) { let tweetsFound = getTweets(req.params.username, 10, res); console.log(tweetsFound); }); The method getTweets will then retrieve data from Twitter using a HTTPS call to
function getTweets(userName, tweetCount, res) {
  var params = {
    q: userName,
    count: tweetCount
  };

  var responseFromTwitter = {};

  let tweetSearched = false;

  // Initiate your search using the above paramaters
  client.get("search/tweets", params, function(err, data, response) {
    console.log("Found # Tweets: " + data.statuses.length);
    res.charset = "utf-8";
    res.append("Content-Type", "application/json; charset=utf-8");

    // If there is no error, proceed
    if (!err) {
      res.end(JSON.stringify(data.statuses));
    }
  });
}

To test out this code, you can enter the following command to clone the repository.
git clone https://github.com/toreaurstadboss/SimpleTwitterFeedExpressJs.git