Sunday, 31 March 2013

Searching for text in files with Powershell

This article will present a way of searching for text in files on your hard disk using Powershell. There are of course many ways of achieving this, such as findstr command. I call the function I present here as grep, but this is in no way anything similar to the sophisticated grep tool many users from Unix-based operative systems are familiar with. The output this command gives gives a nice summary of line number and file
function grep($searchPattern, $fileMatch = '*.*', [bool] $ignoreCase = $true){            
 if ($ignoreCase){            
  Get-ChildItem -Recurse -Filter $fileMatch | Select-String -Pattern $searchPattern |            
  Format-Table -GroupBy Path -AutoSize            
 else {            
  Get-ChildItem -Recurse -Filter $fileMatch |             
  Select-String -Pattern $searchPattern -CaseSensitive |            
  Format-Table -GroupBy Path -AutoSize            
grep AsFullName *.cs $true            

The cmdlet above is called grep, but this does not apply to the verb-noun standard of Powershell. A better name could be Search-Files or something similar. The function uses the Get-ChildItem cmdlet with the recurse option to search all files (and folders) recursively from the current working directory and all its subfolders. This is then piped to Select-String and the flag -CaseSensitive is set if $ignorecase is set to $false. As you can see, the default value of the second and third parameter to the function /cmdlet is '*.*' for $fileMatch and $true for $ignoreCase. A function call is done in the code shown above as an example. Make note that only the first parameter is required, the search string. If a case sensitive search is desired, one must specify all three parameters. If it is desired to search through all files, use '*.*'. In my example, I am looking through some source code, and I am only interested in looking at the files with the extension '.cs' (C# source code files).

It is possible to compact the function or cmdlet above by making use of emulating the ternary operator, but the script above is easy should be easy to debug. If $ignoreCase is set to $false, the -CaseSensitive flag is used in the Search-String cmdlet in the script above.

The output of the command is shown below, when the working directory is set to the source code folder where I have cloned the source code of ShellStudio:

PS C:\toaurs-he\StudioShell\studioshell[ default ]> grep AsFullName *.cs

   Path: C:\toaurs-he\StudioShell\studioshell\src\CodeOwls.StudioShell\CodeOwls.StudioShell.Paths\Items\CodeModel\ShellCodeTypeReference.cs

IgnoreCase LineNumber Line                                            Filename                  Path                                                                              
---------- ---------- ----                                            --------                  ----                                                                              
      True         54         public string AsFullName                ShellCodeTypeReference.cs C:\toaurs-he\StudioShell\studioshell\src\CodeOwls.StudioShell\CodeOwls.StudioSh...
      True         56             get { return _typeRef.AsFullName; } ShellCodeTypeReference.cs C:\toaurs-he\StudioShell\studioshell\src\CodeOwls.StudioShell\CodeOwls.StudioSh...

Using Studioshell to prefix booleans with Is

An often used convention used in coding is to prefix boolean properties that are public with the prefix Is. In this article, an automatic refactoring technique will be explained using Powershell. Alternatives could be using Resharper or Roslyn. Since Powershell will tackle this on a file level, it will execute very fast. Using Powershell alone will not be sufficient, since Powershell does not have the needed code analysis capabilities. To tackle this, we will useStudioshell and execute the script presented in this article.
To download Studioshell, visit the following url:
Studioshell codeplex page

As an alternative, download and install StudioShell via NuGet. Open up a new solution in Visual Studio, then choose the menu option Tools, Library Package Manager and Package manager console. Choose your main project or what's best suited and run the command:
Install-Package StudioShell.Beta

When StudioShell is installed, check that the Execution Policy is remote signed. Start Visual Studio as an administrator and run the following cmdlet: Set-ExecutionPolicy RemoteSigned . StudioShell has its own profile file. Let's first create the profile file, if necessary. Run the following cmdlet: New-Item -ItemType File $profile Add the following function to the $profile file, by running the following cmdlet to open up the $profile file in Notepad: Invoke-Item $profile The automatic refactoring function in Powershell with the help of StudioShell then looks like this:

function PrefixWithIs-Booleans(){            
 Set-Location dte:\solution\codemodel            
        dir -recurse | where { $_ -match 'property' -and $ -notMatch '^Is' -and $_.access -match 'public' -and             
        $_.type.asfullname -match 'bool' } | foreach { $newname = "Is" + $; $_.renamesymbol($newname); }            
Of course, this function could be put in a .ps1 or .psm1 file instead of residing in your StudioShell profile file. By putting the function or cmdlet into the $profile file, it will always be available in StudioShell. As you can also see, the cmdlet is named with verb-noun, as is the convention in Powershell. To run this cmdlet, just type: PrefixWithIs-Booleans This cmdlet will automatically rename all public properties that return a boolean and that has not have the prefix Is, and rename this public property to have the prefix Is. This will result that all public properties start with Is. The cmdlet PrefixWithIs-Booleans starts with running the dir -recurse command to recursively look at all files (and folders) in subdirectories. But first the location is changed to dte:\solution\codemodel. Studioshell has a folder structure which is used for the analysis of code of the currently loaded solution. By then looking at all files (and folders) in the folder dte:\solution\codemodel, effectively one is looking at the entire Visual Studio solution. The property of the items one is looking at is CodeOwls.StudioShell.Paths.Items.CodeModel.ShellCodeProperty2. The command dir -recurse is piped to the cmdlet where and the long where condition effectively looks for boolean properties that are public and does not have a name starting with Is. The result set is then furter piped to foreach which will iterate on the current object in Powershell, $_ and then call the renamesymbol method of the current object. The new name is then set to Is + name member of current object. This shows how Studioshell can refactor a Visual Studio solution using the dte: folder structure of Studioshell and then to code manipulation on the Visual Studio. It is important to be careful when using Studioshell, I would suggest using a versioning system such as Mercurial. Studioshell can also automate a multitude of features in Visual Studio besides code manipulation and code analysis. For example, it is possible to save window layout and load up a window layout with a single command from the shell command line of Studioshell inside Visual Studio. It is also possible to toggle on and off all breakpoints and so on. If you find yourself doing repetitive tasks in Visual Studio or changing the code in a repetitive manner, consider using StudioShell for automating these tasks.

Saturday, 30 March 2013

Creating a Mercurial-aware Powershell command line

This article will describe how we can create a Mercurial-aware Powershell command line. Mercurial is a (distributed) source control. It supports branch per feature and multiple other nice features that gives powerful control of the source code. To get Mercurial, check out the following url: Mercurial The Mercurial-aware Powershell command line will detect when the user enters a Mercurial repository folder on disk. This will be done via the hg branch command, which will return null if the user is not in a Mercurial folder and the name of the branch if the user is inside a Mercurial repository. Let's look at the needed code first, I put this code in my $profile file, use the cmdlet inside Powershell to open the $profile file: Invoke-Item $profile This will edit the profile file of your Powershell by launching Notepad. The $profile file is not perhaps created yet. If that is the case, enter the following command: new-item -ItemType file -path $profile Now retry the command Invoke-Item $profile if this command failed. If you just type: $profile at the Powershell prompt, you will get the file location of the profile file. Make note that Powershell ISE and Powershell prompt has different profile files.. Before I can show you the script, I must get the script transformed into readable colored HTML right? I type Import-Module PowershellPack, now I can use the nice cmdlet Copy-ColoredHTML. This copies the script as colored HTML into the clipboard. I can then paste the
function get-hgStatus(            
    $status = @('M','A','R','C','!','?','I')            
    hg status --all |            
        where { $_ -match "^\s*[$Status]" } |            
        foreach { $_ -replace "\s+", ',' } |            
        ConvertFrom-Csv -Header Status, Path             
function prompt {            
 $branch = (hg branch);            
 $status = get-hgStatus | Group-Object Status;            
 $modified = $status | where { $ -eq 'M' } | select -expand count;             
 $added = $status | where { $ -eq 'A' } | select -expand count;             
 $removed = $status | where { $ -eq 'R' } | select -expand count;             
 $untracked = $status | where { $ -eq '?' } | select -expand count;             
 $missing = $status | where { $ -eq '!' } | select -expand count;             
 Write-Host "PS $pwd" -NoNewline            
 if ($branch){            
  write-host "[" -NoNewline            
  write-host " $branch " -ForegroundColor White -NoNewline            
  if ($added){            
    write-host " " -NoNewline            
    write-host "+$added" -ForegroundColor green -NoNewline;            
  if ($modified){            
    write-host " " -NoNewline            
    write-host $modified -ForegroundColor yellow -NoNewline;            
  if ($removed){            
    write-host " " -NoNewline            
    write-host "-$removed" -ForegroundColor magenta -NoNewline;            
  if ($missing){            
    write-host " " -NoNewline            
    write-host "!$missing" -ForegroundColor red -NoNewline;            
  if ($untracked){            
    write-host " " -NoNewline            
    write-host "?$untracked" -ForegroundColor gray -NoNewline;            
  write-host "]" -NoNewline;            
 "> "            
Note the use of a default value in the passed in $status variable, which is default set to an array of all the possible hg status flags. The hg-GetStatus function can be called individually like hg-GetStatus A to show all the added files in the repo. The script above uses the hg status command to get the status of the current hg repository in the folder. If the current folder is not a hg repository, a default prompt is shown instead. The output of the command hg status is converted into a structured object and then grouped by status using the Group-Object cmdlet. A pipeline is used to prepare the output of the hg status before it is transformed into the structured object. The count of each group is retrieved with the Select cmdlet, and using the -extract flag. The group counts are then shown to the user in the prompt and formatted with color. Make note of the use of -NoNewLine and -ForegroundColor. In addition, the Powershell function prompt will control how your prompt in the shell will look like, i.e. the command line. I put this prompt in the $profile file such that this is inited each time.

This is just an introduction to customizing the Powershell command line to a more suited hg aware prompt for developers using hg. There is actually already a better package available on the Internet for displaying the state of the hg repository in the folder displayed in Powershell, which is called hg posh. Check out hg posh on the following url:
hg posh

However, although hg posh is more correct, I like the look of this command prompt better ... For many hg users, this will suffice.. And here is our nice new Mercurial aware command line:

Friday, 29 March 2013

Hashtables in Powershell

Hashtables are good for creating dictionary-like data structures in Powershell, for look-ups and related functionality. The following powershell script code shows how to create a Hashtable in Powershell and to use and modify this datastructure:
$countries = @{            
    "Norway" = "Oslo";            
    "Denmark" = "Copenhagen";            
    "Sweden" = "Stockholm";            
    "Germany" = "Berlin";            
    "Italy" = "Rome";            
    "Burkina Faso" = "Ougadougou";            
$countries.'Burkina Faso'            
$countries.Add('France', 'Paris')             
$countries.Remove('Burkina Faso')             
$countries.ContainsKey('Burkina Faso')
The code above creates a hashtable using the @{ and } literals, where each key and value is defined by key = value followed with a semicolon, note that this is optional and can be omitted. Hashtables are very user-friendly in Powershell. One can address a value in the hashtable by passing in the key in square brackets, like an index. It is also possible to perform manipulation of the hashtable, using Remove and Add. For the Add method, one supplies the key and value pair to add. For the Remove method, one supplies to the key to look for and then remove (this will of course also remove the value of the key-value pair).

It is also possible to query the hashtable using ContainsKey and ContainsValue methods, looking for keys or values. Note also that members of the hash table is available and shown in Intellisense using the Powershell ISE (Integrated Shell Environment). It is possible to say $countries.France, or $countries.'Burkina Faso'. The last hash table key address is encapsulated in quotes, since there is a space in the key. When calling GetType() on the hashtable instance $countries, the name Hashtable is shown.

To list up the keys and the values of a hashtable, use the Keys or Values member of the hashtable. Example:

Keyboard input in Powershell

Keyboard input in Powershell

Asking users for input in Powershell is very straight-forward. Use the Read-Host cmdlet in Powershell and add a string value to the -Prompt switch of the cmdlet to specify which text to display to the user. Assign the result from Read-Host to a Powershell variable, e.g. $myvariable. This variable can then be treated logically and control the further program execution and logical flow of the program.

The following code shows how this can be done:

function Ask-FavoriteColor ([int] $personsToAsk){            
    Ask about the favorite color
    The favorite color was asked in the movie "Holy Grail"
    .PARAMETER personsToAsk
    The number of persons to ask
 foreach ($i in 1..$personsToAsk){            
  [string] $color = Read-Host -Prompt "What is your favorite color?"            
  if ($color.ToLower() -ne "blue"){            
     Write-Host "You didn't say blue!"             
  else {            
     Write-Host "Good choice!"             
Ask-FavoriteColor 2

If you see the text at the top of the function, you will see how one can document the function using meta keywords such as .SYNOPSIS and .DESCRIPTION. When using the Powershell cmdlet Get-Help, e.g. Get-Help Ask-FavoriteColor -detailed, one will see the (detailed) documentation in the Powershell help page. This is rather equivalent to the obiqutous man command seen in Unix-based systems.

When creating Powershell functions, your functions should be documented using this technique. There are lots of other meta keywords that can be used. See this overview for meta keywords for documentation.

PS C:\Users\Tore Aurstad\Documents\Powershell Scripts\files> C:\Users\Tore Aurstad\Documents\Powershell Scripts\Input\KeyboardInput.ps1
What is your favorite color?: red
You didn't say blue!
What is your favorite color?: blue
Good choice!

PS C:\Users\Tore Aurstad\Documents\Powershell Scripts\files> Get-Help Ask-FavoriteColor -Detailed NAME Ask-FavoriteColor SYNOPSIS Ask about the favorite color SYNTAX Ask-FavoriteColor [[-personsToAsk] ] [] DESCRIPTION The favorite color was asked in the movie "Holy Grail" PARAMETERS -personsToAsk The number of persons to ask This cmdlet supports the common parameters: Verbose, Debug, ErrorAction, ErrorVariable, WarningAction, WarningVariable, OutBuffer and OutVariable. For more information, see about_CommonParameters ( REMARKS To see the examples, type: "get-help Ask-FavoriteColor -examples". For more information, type: "get-help Ask-FavoriteColor -detailed". For technical information, type: "get-help Ask-FavoriteColor -full".

Exception handling in Powershell scripts

Exception handling in Powershell

Powershell is capable of handling exceptions in your Powershell scripts. Instead of the familiar try-catch construct, you use the trap keyword. The trap keyword followed by a block clause for handling the exception can be anywhere inside a Powershell function, at the top, middle or bottom of the method. I would suggest keeping the trap keywords at the bottom for readability. After all, your function's code is what is most important and exception handling is secondary. Still, the scripts should contain good exception handling for expected errors. In addition, a universal exception handler can be achieved by trapping the [System.Exception] exception, i.e. any exception. It is though better to trap the expected exceptions and unexpected exceptions should be thrown. 
#clear screen first using Clear-Host             
function Divide([int]$a, [int]$b){            
 $answer = $a / $b;            
 Write-Host $answer             
 trap [System.DivideByZeroException] {            
    Write-Host Attempted to divide by zero!            
 trap [System.Exception] {            
    Write-Host Unexpected Exception. Breaking out of function.            
Divide 15 3            
Divide 75 15            
Divide 95 5            
Divide 111 0            
Divide 64 8            
Divide 4 2.3             

As seen in the Powerscript code above, the trap keyword is followed by the error handling code. Usually, doing a Write-Host or logging the error is what will be done here (Write-Host corresponds to Console.WriteLine) in .NET based programming languages.

It is possible to have multiple trap statements with individual Exception types. In this example, the [System.DivideByZeroException] is handled. The keyword continue basically means that execution is permitted to continue. In the other trap statement, the keyword break is used. This basically is the same as a throw statement in a corresponding try-catch clause, where program execution is halted in the function. Judge which exceptions should be allowed to continue, and which exceptions should be allowed to break.

This is just another example of how powerful Powerscript is when it comes to a shell script. Of course, shell scripting languages such as Perl sports most of the functionality. The ability of Powershell to make use of .NET (after all Powershell is built upon .NET), means many .NET developers will quickly find new uses of Powershell by making use of previous knowledge and experience in .NET.

To sum up, your Powershell scripts should have the necessary trap statements for error-handling. As a sidenote, I do not like the keyword trap, they should have called it catch or something more trustworthy ..

Thursday, 28 March 2013

Using Powershell to download multiple files


Powershell is a shell scripting language with command line access, based on .NET, which  gives
the user and the developer access to an abundance of system resources, plus intra- and internetworked resources. In this article, an example of how Powershell can be used to download multiple files from the internet, and save these files to a local file folder, will be given.

Powershell is as previously mentioned based upon .NET. This gives Powershell many capabilities as a command line shell and scripting language. Syntaxwise, Powershell in many respects are similar to Perl or PHP, but there are also simliaries to other .NET Languages. At the same time, this is a command line shell script. It's role is to inherit the role that the Command Line Prompt (CMD.exe) and MS-DOS previously covered. Instead of the .BAT files of CMD and  MS-DOS, scripts are written in Powershell scripts, with the extension .PS1. There are more advanced topics to cover, such as modules of Powershell, but this article will cover some basic use of Powershell.

I have used the Windows Powershell ISE, which is the Integrated Script Environment.
The term ISE is similar to IDE, which stands for Integrated Development Environment. Powershell ISE is included in Windows 8, which I have used. Windows 7 users can download the Windows Management Framework 3.0 package, which is available at the following URL:

The above package is available also for Windows 2008 Server R2 and Windows 2012 Server.

Downloading files with Powershell

Let's first review the script which downloads some files (in my example I create an Object array of Url addresses which points to Flickr thumbnail images of mine).

$flickrImages = '',             
$targetDir = 'C:\users\Tore Aurstad\Documents\Powershell Scripts\WebClient\'            
function DownloadFile([Object[]] $sourceFiles,[string]$targetDirectory) {            
 $wc = New-Object System.Net.WebClient            
 foreach ($sourceFile in $sourceFiles){            
  $sourceFileName = $sourceFile.SubString($sourceFile.LastIndexOf('/')+1)            
  $targetFileName = $targetDirectory + $sourceFileName            
  $wc.DownloadFile($sourceFile, $targetFileName)            
  Write-Host "Downloaded $sourceFile to file location $targetFileName"             
DownloadFile $flickrImages $targetDir            

Many .NET developers are unfamiliar with Powershell scripting language syntax. Variables are declared by prefixing the variable name with the $-sign. I declare an Object Array, which is basically a comma separated list. This syntax will be familiar to Perl Developers. Also note the absence of semi-colons. Semi-colons are not obligatory in Powershell, and these are therefore omitted.

Further on, I declare a variable for where to put the downloaded files. If you want to test out this script yourself, you have to change this path, obviously.
Next on, I declare a function in Powershell for downloading the files. This function takes in an Object array and a string for the target Directory or folder.

The function is called without using commas or parentheses. In Powershell every function argument is passed in using spaces. This is where the Powershell syntax feels a bit different to other Microsoft Languages. If you use commas when calling the function in this example, the first argument will receive an Object Array, while the second argument will be empty. Be aware of this Powershell gotcha - use spaces when calling a function with multiple arguments.

Let's investigate the function a bit closer. As you can see, although Powershell allows mutable types (which means that a variable containing for example a string can be redefined to an integer), prefixing the arguments with square brackets and a type, such as [Object[]] or [string], makes your Powershell scripts more type-safe. Powershell obviously in many cases feels like Javascript for example, where also a variable can be changed into another type (redefined). If you want to have more type safety, prefix your function arguments with the strongly type you expect being passed into the Powershell function. The variable $wc is instantiated into a new System.Net.WebClient instance, using the New-Object cmdlet (Command let) in Powershell. To perform the download of each file in the passed in Object Array, I use a foreach loop.

This is where Powershell shows itself as a convenient scripting language. Iterating through a collection using foreach loop makes coding much easier, note though that Perl already implemented this functionality some years ago. Inside the foreach loop, the call to the function or method $wc.DownloadFile downloads the file from the Internet using the instantiated WebClient. I am not sure if .BAT files could achieve this, but this shows how much functionality is available to Powershell users and Developers. Powershell is well suited for system maintenance and administration, plus actually are larger part of target uses than one might think of a shell script language and command line tool.

The fact that much of .NET is available means that .NET Developers in some cases must rethink their use of the relationship between compiled programs and scripts, now that Powershell is so available. The added convenience and flexibility of Powershell, plus functionality makes Powershell a contender among the more mature .NET Programming Languages such as VB and C#. If you implemement something in a Programming language, think of doing the same in a script. Sometimes going for a script based solution will be better, more flexible and optimal.

For developers using .NET, the use of SubString and LastIndexOf shows that many .NET methods and types are available for us in Powershell.

If you wonder how I formatted the Powershell code above, I downloaded the PowerShellPack utility from here:

Powershell pack

After you have downloaded PowershellPack and installed it, follow the following guide:

Copy selected text in Powershell ISE to colored Html

Click the thumbnail below to see the script running inside Windows Powershell ISE:

Windows Powershell ISE has an editor and a command line below, where Powershell scripts can be executed and Powershell itself can be used. To start the PowerShell script, I type .\WebClient.ps1, since I saved the script above in a file called WebClient.ps1.

Users of BASH and other shell scripting Languages will feel familiar using Powershell, as multiple commands are linked to Unix familiar commands, such as DIR in MS-DOS and CMD is aliased to the command ls. More commands can be aliased, such that Unix Developers can manage part of Windows Clients and servers and still have that familiar Unix syntax feel of it ... This concludes our introductory tour of Powershell. I would like to say happy coding, but it is perhaps more correct to say happy (Powershell) scripting?