Powershell - Search for big files and output to Excel

Here is a Powershell script that you can run to search for the ten biggest files in your current file directory and display them in Excel. My version of Powershell is 5.1 according to the built in Powershell variable $PSVersionTable.PSVersion, so if you run the script in earlier versions of Powershell, the script must be changed a bit.

Write-Host 'Look for big files in current directory' 

$alternatingOddRowColor = 19
$alternatingEventRowColor = 20

$filename = ''

#$filename = Read-Host 'Directory to output filelist? (c:\temp) default'


if ([string]::IsNullOrEmpty($filename)){
 $filename = "c:\temp\\" 
}

$filename = $filename + 
 ([TimeSpan](Get-Date).ToLongTimeString()).Ticks.ToString() + '_Bigfiles' + '.csv'

Write-Host $filename


gci -r | sort Length -desc |
Select-Object @{Name='Filesize(MB)'; Expression = { [int]$($_.Length /1MB) }}, Name, FullName, LastWriteTime -First 10 | 
Export-Csv $filename -NoTypeInformation -Encoding ASCII -UseCulture

$excelFileName = $filename.Replace('.csv', '.xlsx')


$excel = New-Object -ComObject Excel.Application 
$excel.Visible = $true
$excel.Workbooks.Open($filename)

$excel.DisplayAlerts = false

$objWorksheet = $excel.Workbooks.Item(1)

$activeRange = $excel.ActiveWorkbook.ActiveSheet.UsedRange

$activeRange.EntireColumn.AutoFit()

For ($i = 1; $i -le $activeRange.Rows.Count; $i++) { 
 $themeColorIndex = $(If ($($i %2) -eq 0) { $alternatingOddRowColor } 
   Else { $alternatingEventRowColor });

 $currentRow = $excel.ActiveWorkbook.ActiveSheet.UsedRange.Rows($i).EntireRow
 $currentRow.Font.Name = 'Comic Sans MS'
 $currentRow.Interior.ColorIndex =  $themeColorIndex;
 if ($i -eq 1){
  $currentRow.Font.Bold = true 
  $currentRow.Font.Size = 14
 } 
}


$excel.SaveAs($excelFileName,51)

#$excel.Quit()

A timestamp can be created in Powershell using the Get-Date cmdlet, converting the Get-Date DateTime object to a string with ToLongTimeString(), then casting that string into a timestamp and performing a ToString. Like this:

 ([TimeSpan](Get-Date).ToLongTimeString()).Ticks.ToString()

It is nice to have a file size in megabyte with Get-ChildItem That can be achieved using a calculated property in Powershell. This is done with the following construct ${ .. }

@{ Name : 'PropertyName', Expression { $( ..calculation here .. ) }}

#Like this! 

gci -r | sort Length -desc |
Select-Object  @{Name='Filesize(MB)'; Expression = { [int]$($_.Length /1MB) }} , Name, FullName, LastWriteTime -First 10 | 
Export-Csv $filename -NoTypeInformation -Encoding ASCII -UseCulture

After running the Powershell script I could find out why my source code repository seemingly had grown so much in size. The repo was not increased, but the folder Test Results contained almost a gigabyte of disk space after running some web tests and load tests. My .hgignore file ignore these files anyways. The picture below shows how the generated file looks. Of course with a Comic Sans MS. Font

Compressing files in a MVC environment

This article will present a way to compress files in a MVC environment. For compression, we will use the DotNetZip Nuget package, which is an open and free compression library hosted on Codeplex and supported also by Xceed. The DotNetZip produces of course Zip files. DotNetZip website We install this compression library by initiating the following Nuget command: Install-Package DotNetZip We then define a simple view in MVC that has got a file upload input and a submit button:

@model ZipAndMvc.Models.HomeViewModel
@{
    ViewBag.Title = "Home Page";
}

<div class="jumbotron">
    <h2>Test out zipping a file</h2>   
</div>

<div class="row">

    @using (Html.BeginForm("ZipIt", "Home", FormMethod.Post, new {  enctype = "multipart/form-data" }))
    {
        <div class="col-md-3">@Html.Label("Zip password") @Html.TextBoxFor(m => m.ZipPassword) </div>
        <div class="col-md-3"><input type="file" name="FileUpload" /> </div>
        <div>  <input type="submit" id="Submit" value="Upload and zip file" /> </div>
    }
</div>

This view allows the user to type in a password for the file to compress, where the user also selects the file to compress. The user then hits the submit button. The HomeViewModel is very simple with a simple property for setting the zip password. Then we define the following code in the MVC controller:

        public FileStreamResult ZipIt(HomeViewModel viewmodel)
        {
            if (Request.Files.Count > 0)
            {
                using (var zip = new ZipFile())
                {
                    zip.Encryption = EncryptionAlgorithm.PkzipWeak;
                    zip.Password = viewmodel.ZipPassword;
                    zip.CompressionLevel = Ionic.Zlib.CompressionLevel.Default; 
                    var memoryStream = new MemoryStream();
                    zip.AddFile(Request.Files[0].FileName, "");
                    zip.Save(memoryStream);
                    memoryStream.Position = 0;
                    return new FileStreamResult(memoryStream, contentType: "application/zip")
                    {
                        FileDownloadName = Path.ChangeExtension(Request.Files[0].FileName, "zip")
                    };
                }
            }
            return null;
        }

The client posts the file to compress. The controller then inspects the Request.Files collection and selects the first file if there is present any files there. Here we return a FileStreamResult where the compressed data inside the memorystream is returned to the client. We use DotNetZip to do the compression. The benefit of DotNetZip compared to .Net built-in support for compression is more functionality. The code above should be sufficient for basic compression scenario in MVC. Feel free to experiment with DotNetZip. As you can see, you can specify compression level. You can also choose to add directories and much more. The reason for the second argument in AddFile method is to ensure that the file to be added to the zip package is put in the root folder of the zipped file. Also, set the values of Encryption and Password before adding files or directories (Folders) in the ZipFile. You can actually use different passwords also in the Zip file.

Finding old Git Branches with WSL and Bash

Finding old branches in Git

I had to find out which branches in a Git repository was old and output it to a file. An old branch is defined to have no commits the last four months. Here is the bash script I ended up with.

#!/bin/bash
resolveOldBranches(){
branchfile="oldbranches.txt"
declare -i branchiteration=0
branchcount=$(git branch -a | wc -l)

if [ ! -e $branchfile ] ; then
 touch $branchfile
fi

#empty the oldbranch file
: > $branchfile

for k in $(git branch -a | sed /\*/d); do


 if [ -z "$(git log -1 --since='4 months ago' -s $k)" ]; then
  echo $k | cut -d/ -f3 >> $branchfile
 fi
 branchiteration=$branchiteration+1
 percentage= bc <<< "scale=2;($branchiteration/$branchcount)*100"


 read -n 1 -t 0.1 input                  # so read doesn't hang
   if [[ $input = "q" ]] || [[ $input = "Q" ]]
   then
      echo # to get a newline after 
echo -e "XXX\n$($percentage)\nAnalyzing $branchiteration of $branchcount $(bc <<< "scale=2;($branchiteration/$branchcount)*100") % done. \n(Exit: Q/q)... \nXXX"

done | whiptail --title "Resolving OpPlan 4 branch ages" --gauge "Analyzing.. (Press Q or q to exit)" 10 60 0


}

resolveOldBranches
cat $branchfile

X11 Subsystem running WSL Windows 10 subsystem for Linux

This article will look at running graphical Linux applications in Windows 10. As many of you know, Windows 10 can offer a subsystem for Linux running inside Windows 10. We need to download and install a X11 server to be able to run graphical applications that need more than the console, that is a graphical user interface. First off, download XMing for Windows Server from here:

XMing X Server for Windows Server
After installation of Xming, start Xming from your start button.
Now we need to install the Linux Subsystem itself, if you have not done this yet.
Inside Powershell, enter the following command as an administrator:

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux 

If you are not a Windows Insider yet, you must join this program. See this page for a description how to obtain WSL or Windows Subsystem for Linux and install it.
Installation guide of WSL - Windows Subsystem for Linux After you have downloaded WSL and installed it using Powershell, select Launch and in the console Window after WSL is further installed and setup, enter a user in Linux you will use as administrator (in Linux terms, this is the root user). You can add another user using the adduser command as root in the Bash console. You can access WSL Linux inside Windows 10 by selecting the menu item Bash on Ubuntu on Windows.

Getting started using WSL

You probably want to update your WSL to the latest version. WSL is a specialized Ubuntu Linux distribiton. Enter the following command to upgrade Linux Kernel and applications, note that it will take time to download package and upgrade to latest versions:

apt-get update && apt-get upgrade The version of Linux distro can be found using this command:

lsb_release -irc 

As we see, I am running Ubuntu 16 Xenial. Next off, we are going to support X-Server. Download first the X11 apps.
root@tore# apt-cache search x11-apps x11-apps - X applications
Install X11-Apps. This will also download all required additional pacages. apt-get install X11-Apps
You need to do this as root or switching to a super user and use the sudo command. Next off, edit your ~/.bashrc file. This is similar to the $Profile file that Powershell uses. You are setting up your environment here to make sure you can use X-Server based Graphical User Interfaces. Such as Gimp, Firefox, Stellarium, Quake and so on - requiring a GUI. But we will start off with running a X11-app, such as Xeyes. Make sure your ~/.bashrc file got the following content. First download Nano if you do not want to use Vim or Vi. (apt-get install nano)
nano ~/.bashrc Next off, add the following line at least: export DISPLAY=:0 We can also add more nice colors and some additional information and offer to switch to a non-root user initally.

export DISPLAY=:0
LS_COLORS='rs=0:di=1;35:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:';
export LS_COLORS
PS1='\e[37;1m\u@\e[35m\W\e[0m\$ '
echo "Welcome to Bash on Ubuntu on Windows. To switch to your user Tore :"
echo "su tore && cd /home/tore"
echo "Do this now? [y/n]"

read -rsn1 input
if [ "$input" = "y" ]; then
    cd /home/tore
    su tore
    ls -al
fi


echo "To adjust profile file edit the file ~\.bashrc with Nano"

Last, we can test out everything, running Xeyes. Note that you can test out more advanced Linux applications such as Gimp by running: apt-get install gimp This will install Gimp, the Gnu Image Manipulation Program. Then just enter gimp from the console. You can also run such programs in the background by adding the ampersand (&): gimp &

Sometimes, we need to force an exit of such a Linux app with a GUI running from the console entering Ctrl+C to force exit the process.

Setting up a Git alias with a shell function

This short article will show you how to search in the log of a Git repository with a shell function. This shell function can also receive a positional parameter to use in the shell function. First off, edit the .gitconfig file in your user's home folder. On Linux Mint for example, the folder should reside in the /home directory. You can use nano for example. (apt-get install nano)

.gitconfig : 

[alias]
        searchlog = "!f() { git log --all --decorate --graph -i --grep \"$1\";  }; f"

Note the positonal parameter $1 here, we escape the quote also. The function inside the alias is a shell function.

To use this alias command, type for example:
git searchlog test

This will search the Git log for the parameter passed in (test) and include some flags to decorate the log displayed. Using Git aliases, we can do lenghty Git commands with shorter aliased commands.

Displaying altitude with Google Maps

We will look at displaying additional positional information with Google Maps in this article, such as altitude. First off, we need to add a new object to Google Maps Api v3 - the MarkerWithLabel object. This allows us to add text labels to Google Maps. They are also draggable. I have updated a Plunk below so you can see the end result yourself:

Location in Google Maps with altitude - Plunk

The MarkerWithLabel.js contains the additional javascript code to add the MarkerWithLabel object. We add the marker inside the showPosition method with the following code:

  function showPosition(position){
        
         var mapCanvas = document.getElementById("demo");
  var myCenter = new google.maps.LatLng(position.coords.latitude, position.coords.longitude); 
  var mapOptions = {center: myCenter, zoom: 12};
  var map = new google.maps.Map(mapCanvas,mapOptions);
  var marker = new google.maps.Marker({
    position: myCenter,
    animation: google.maps.Animation.BOUNCE
  });
  marker.setMap(map);
  
  console.log(position.coords);
  
  
   var image = 'https://developers.google.com/maps/documentation/javascript/examples/full/images/beachflag.png';
 
  

     var altitudeMarker = new MarkerWithLabel({
       position: myCenter,
       draggable: true,
       raiseOnDrag: true,
       icon: image,
       map: map,
       labelContent: position.coords.altitude.toString(),
       labelClass: "labels", // the CSS class for the label
       labelStyle: {opacity: 0.75}
     });
     
     
        
      }

We add the altitude, using the position.coords.altitude object.

A good tip here is to add high accuracy of the GeoLocation.

  function getLocation() {
    

    var geo_options = {
     enableHighAccuracy: true, 
     maximumAge        : 30000, 
     timeout           : 27000
    };


         if (navigator.geolocation){
          var position = navigator.geolocation.getCurrentPosition(showPosition, null, geo_options);
        }
      }

Note that the client has got to have a positioning device supporting returning the altitude. Most smartphones today got GPS for example.
As a test - you can change the value displayed to position.coords.accuracy.toString() instead. Accuracy is always return in the Coords object.

HTML 5 Geolocation introduction

This article will introduce you to HTML 5 Geolocation API.

First off, test the browser for support of HTML 5 Geolocation. The first call to get the location of the client / user will also prompt a dialog that the user must usually confirm to to allow getting the location.
The following Javascript is necessary to get the latitude and longitude of the client :

 function getLocation() {
  if (navigator.geolocation){
   var position = navigator.geolocation.getCurrentPosition(showPosition);
  }
 }
        
 function showPosition(position){
  console.log(position);
  $("#demo").html(position.coords.latitude + " " + position.coords.longitude);
 }

Note that you can retrieve additional information such as altitude, heading and speed if the client got positioning hardware supporting this.
Mozilla Developer Network - Coordinates object
In Firefox, you have to use HTTPS to use Geolocation in newer versions of this browser. The client must agree to share the current location. A demo of Geolocation is here:

HTML 5 Geolocation demo - Plunk

Let us also use a map to show the location of the user. We can use Google Maps API. To use this API, request a Google API Key from here:

Google API Key site

We plot the current location with some script below using the Google Maps API, adding a Marker that is bouncing on top of the current location.

  function getLocation() {
   if (navigator.geolocation){
    var position = navigator.geolocation.getCurrentPosition(showPosition);
   }
  }
        
  function showPosition(position){
        
   var mapCanvas = document.getElementById("demo");
   var myCenter = new google.maps.LatLng(position.coords.latitude, position.coords.longitude); 
   var mapOptions = {center: myCenter, zoom: 12};
   var map = new google.maps.Map(mapCanvas,mapOptions);
   var marker = new google.maps.Marker({
    position: myCenter,
    animation: google.maps.Animation.BOUNCE
   });
    marker.setMap(map);
        
  }
      

The current location is displayed using Google Maps as displayed here:

Support for Geolocation in Firefox is limited to HTTPS and recent version of Firefox may not function with Geolocation in Linux, newer than Firefox version 20. If so, downgrade to Firefox Version 20. I have tested the code above using Opera web browser in Linux Mint 15.

Note that we here added Google Maps Javascript source reference and an Google API key.

  <script src="https://maps.googleapis.com/maps/api/js?key=AIzaSyAeymq4mlrbKoBUhU3LdegaudQDnY7MFPo&callback=myMap"></script>
  

Modernizr feature discovery demo

I just made a Modernizr feature discovery demo! It lists up the features Modernizr looks for and tests the browser you are running if it supports that feature! Plunk - Modernizr demo

<html class="no-js">

<head>
<meta charset="utf-8"/>
<title>Modernizr browser feature detection</title>
<script data-require="modernizr@*" data-semver="2.6.2" src="//cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.js"></script>
<script data-require="jquery@*" data-semver="3.1.1" src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script>
<script data-require="underscore.js@*" data-semver="1.8.3" src="//cdnjs.cloudflare.com/ajax/libs/underscore.js/1.8.3/underscore-min.js"></script>
</head>

<style>

.greenlight {

}

.yellowlight {

}

.redlight {

}

.trafficlight {


}

.trafficlight:after {
    background-color: #10AF20;
 border-radius: 10px;
 padding-left: 5px;
 padding-right: 5px;
}

.redlight:after {
 content: "UNSUPPORTED ";
    color: #f0f0af;
    background-color: #AF1020;
}

.yellowlight:After {
 content: "PERHAPS SUPPORTED";
 color: #f0f0af;
    background-color: #AFAF10;
}

.greenlight:after{
 content: "SUPPORTED ";
 color: #f0f0af;
 background-color: #10AF20;
}

.underlight{
 margin-left:30px;
}

li {
 font-family: Trebuchet, Verdana;
 margin: 4px;
}

</style>

<body>
<h2>Modernizr browser feature detection</h2>


<ul id="ModernizrFeatureList">
<script>

function displayfeature(feature, isSubfeature){
   var isFeaturePartiallySupported = false;
   var isFeatureSupported = false; 
   if (eval("Modernizr." + feature) === true){
    isFeatureSupported = true;
   }
   if ((eval("Modernizr." + feature) === "probably") | (eval("Modernizr." + feature) === "maybe")){
    isFeaturePartiallySupported = true;
   }

   //debugger;

   var trafficlight = "trafficlight" + " ";
   if (isFeatureSupported)
    trafficlight += "greenlight"; 
   if (isFeaturePartiallySupported)
    trafficlight += "yellowlight"; 
   if (!isFeatureSupported && !isFeaturePartiallySupported){
    trafficlight += "redlight";
   }

   if (isSubfeature)
    trafficlight += " underlight";

   var featureToShow = "<li class='" + trafficlight + "'>" + feature + ": " + eval("Modernizr." + feature) + " </li>";

   return featureToShow;
}

var modernizrProps = _.sortBy(Object.keys(Modernizr), function(key){ return key; });


modernizrProps.forEach(function(feature, index){

  var modernizrFeatureType = eval("typeof Modernizr." + feature); 

   if (modernizrFeatureType == "boolean"){
     var f = displayfeature(feature, false);
     $("#ModernizrFeatureList").append(f);
   }
   else if (modernizrFeatureType === "object"){
    try {
     //debugger;
     for (var subfeature in  eval("Modernizr." + feature)){
     var subf = displayfeature(feature + "." + subfeature, true);
     $("#ModernizrFeatureList").append(subf);
    }

    }
    catch (Error){

    }
   }
 
});


</script>

</ul>
</body>
</html>