Setting up Jenkins on Windows with Git, Mercurial and SSH

This guide will detail the steps required to correctly setup and configure Jenkins on Windows using both Git and Mercurial as the version control tools and using SSH with both in order to authenticate with repositories hosted on the BitBucket service.

  • Download and install Jenkins, Git & Mercurial to their default locations. Ensure you get the 64-bit versions of all of these tools.
  • First, we need to create an SSH key pair, using OpenSSH which comes bundled with Git, that will allow Git to communicate with Bitbucket via SSH.
  • Next, we’ll configure OpenSSH (which is used by Git), so follow the steps under the section “Set Up SSH for Git” from here: https://confluence.atlassian.com/bitbucket/set-up-ssh-for-git-728138079.html
  • This primarily involves creating a new ssh keypair from a Git Bash shell, using ssh-keygen, ensuring the resulting keys are stored in your user’s “home” directory (which on Windows is usually, C:\Users\xxxxx\ - where xxxx is your logged on Windows username) within an .ssh directory and that you have a config file within that same folder that tells Git/SSH which key to use for a specific host (i.e. Host bitbucket.org
    IdentityFile ~/.ssh/<privatekeyfile>
    )
  • Next, we need to configure Mercurial. Since Mercurial is a more “windows-y” tool, by default, it wants to use PuTTY (and its related tools of Plink and Pageant), however, we’re going to tell Mercurial to use OpenSSH instead. Normally, you would edit a mercurial.ini file inside the installation folder of TortoiseHg (usually, C:\Program Files\TortoiseHg\) however, this didn’t seem to work for me, as Hg insisted on pulling the required config from a different file which constantly overrode anything I had set in the mercurial.ini file! The file in question that is likely to need to be edited is C:\Program Files\TortoiseHg\hgrc.d\Paths.rc Within this file, you’ll need to add or amend the [ui] section to configure ssh:
    [ui]
    ssh = ssh -2 -C -x
  • Note that the above assumes that the path for SSH (which, since it’s installed with Git, is usually C:\Program Files\Git\usr\bin. If this path isn’t added to your PATH environment variable for SYSTEM (not for a specific user) then you’ll need to add it. Open Control Panel, go to “System”, click “Advanced System Settings” in the left-hand menu, then click the button “Environment Variables” on the resulting dialog. Remember to edit the System Variables not the ones for your user).
  • Jenkins, when installed on Windows, is by default configured to run as a Windows Service. The service is configured to run under the Local System account. As a result of this, Mercurial will invoke OpenSSH in this context, and so OpenSSH will now look to the Local System account’s home directory for it’s SSH keys (Git seems perfectly happy looking for the keys in the user’s home folder). So, take the entire .ssh folder from the user’s home folder (the same folder as used earlier when creating the ssh-keys initially) and copy them to the Local System account’s home folder. Where is this? Well, it’s not within the C:\Users\ area, not even as a hidden/system folder, oh no, that would be too logical. So, Windows, in it’s infinite wisdom decides to place the Local System account’s home folder here: C:\Windows\System32\config\systemprofile\
  • After this, you should be able to start Jenkins and add some build jobs. When creating the jobs, you’ll set the relevant Source Code Management to wither Git or Mercurial, and you’ll specify an ssh:// protocol address for the Repository URL . Note that you do NOT need to specify anything within the credentials (i.e. leave it set to it’s default value of --none--) section here as when Jenkins “shells” out to the relevant SCM tool, that tool will be given an ssh:// address to connect to and that tool’s configuration will look to see how it is configured to access ssh:// URLs. It’s that configuration that will provide the necessary credentials to connect to the remote repository.

MVC Razor Views and automated Azure deployments

The other day, I decided that I’d publish a work-in-progress website to an Azure Website.  This was a free Website as part of the free package that Azure subscribers can use.  The website was a plain old vanilla ASP.NET MVC site.  Nothing fancy, just some models, some controllers , some infrastructure code and of course, some views.

I was deploying this to Azure via a direct connection to a private BitBucket Git repository I had within my BitBucket account.  This way, a simple “git commit” and “git push” would have, in a matter of seconds, my latest changes available for me to see in a real “on-the-internet” hosted site, thus giving me a better idea of how my site would look and feel than simply running the site on localhost.

An issue I almost instantly came up against was that, whenever I’d make a tiny change to just a view file – i.e. no code changes, just tweaks to HTML markup in the .cshtml Razor view file, and commit and push that change – the automated deployment process would kick in and the site would be successfully deployed to Azure, however, the newly changed Razor View would remain unchanged.

What on earth was going on?   I could run the site locally and see the layout change just fine, however, the version deployed to Azure was the older version, prior to the change.   I first decided that I now had to manually FTP the changed .cshtml files from my local machine to the relevant place in the Azure website, which did work, but was an inelegant solution.  I needed to find out why the changed Razor views were not being reflected in the Azure deployments.

image After some research, I found the answer.

Turns out that some of my view files (the .cshtml files themselves) had the “Build Action” property set to “None”.  In order for your view to be correctly deployed when using automated Azure deployments, the Build Action property must be set to “Content”.

Now, quite how the Build Action property had come to be set that way, I have no idea.  Some of the first views I’d added to the project had the Build Action set correctly, however, some of the newer views I’d added were set incorrectly.  I had not explicitly changed any of these properties on any files, so how some were originally set to Content and others set to None was a mystery.  And then I had an idea…..

I had been creating Views in a couple of different ways.  Turns out that when I was creating the View files using the Visual Studio context menu options, the files were being created with the correct default Build Action of Content.  However, when I was creating the View files using ReSharper’s QuickFix commands by hitting Alt + Enter on the line return View(); which shows the ReSharper menu allowing you to create a new Razor view with or without layout, it would create the View file with a default Build Action of None.

Bizarrely, attempting to recreate this issue in a brand new ASP.NET MVC project did not reproduce the issue (i.e. the ReSharper QuickFix command correctly created a View with a default Build Action of Content!)

I can only assume that this is some strange intermittent issue with ReSharper, although it’s quite probably caused by a specific difference between the projects that I’ve tested this on, thus far I have no idea what that difference may be…   I’ll keep looking and if I find the culprit, I’ll post an update here.

Until then, I’ll remain vigilant and always remember to double-check the Build Action property and ensure it’s set to Content for all newly created Razor Views.

SSH with PuTTY, Pageant and PLink from the Windows Command Line

I’ve recently started using Git for my revision control needs, switching from Mercurial that I’ve previously used for a number of years.  I had mostly used Mercurial from a GUI, namely TortoiseHg, only occasionally dropping to the command line for ad-hoc Mercurial commands.

In switching to Git, I initially switched to an alternative GUI tool, namely SourceTree, however I very quickly decided that this time around, I wanted to try to use the command line as my main interface with the revision control tool.  This was a bold move as the Git syntax is something that had always put me off Git and made me heavily favour Mercurial, due to Mercurial’s somewhat nicer command line syntax and generally “playing better” with Windows.

So, I dived straight in and tried to get my GitHub account all set up on a new PC, accessing Git via the brilliant ConEmu terminal and using SSH for all authentication with GitHub itself.  As this is Windows, the SSH functionality was provided by PuTTY, and specifically by the PLink and Pageant utilities within the PuTTY solution.

imageI already had an SSH Key generated and registered with GitHub, and the private key was loaded into Pageant, which was running in the background on Windows.  The first little stumbling block was to get the command line git tool to realise it had to use the PuTTY tools in order to retrieve the SSH Key that was to be used for authentication.

image This required adding an environment variable called GIT_SSH which points to the path of the PuTTY PLINK.exe program.  Adding this tells Git that it must use PLink, which acts as a kind of “gateway” between the program that needs the SSH authentication, and the other program – in this case PuTTY’s Pageant – that is providing the SSH Key.  This is a required step, and is not the default when using Git on Windows as Git is really far more aligned to the Unix/Linux way of doing things.  For SSH on Unix, this is most frequently provided by OpenSSH.

After having set up this environment variable, I could see that Git was attempting to use the PLINK.EXE program to retrieve the SSH key loaded into Pageant in order to authenticate with GitHub, however, there was a problem.  Although I definitely had the correct SSH Key registered with GitHub, and I definitely had the correct SSH Key loaded in Pageant (and Pageant was definitely running in the background!), I was continually getting the following error:

image

The clue to what’s wrong is there in the error text – The server that we’re trying to connect to, in this case it’s github.com, does not have it’s RSA key “installed” on our local PC.  I say “installed” as the PuTTY tools will cache remote server RSA keys in the Windows registry.  If you’re using OpenSSH (either on Windows or more likely on Unix/Linux, they get cached in a completely different place). 

Although the error indicates the problem, unfortunately it gives no indication of how to correct it.

The answer lies with the PLINK.exe program.  We have to issue a special one-off PLINK command to have it connect to a remote server, retrieve that server’s RSA key, then cache (or “install”) the key in the registry to allow subsequent usage of PLINK as a “gateway” (i.e. when called from the git command line tool) to be able to authenticate the server machine first, before it even attempts to authenticate with our own SSH key.

The plink command is simply:

plink.exe -v -agent git@github.com

or

plink.exe -v -agent git@bitbucket.org

(the git@github.com or git@bitbucket.org parts of the command are the specific email addresses required when authenticating with the github or bitbucket servers, respectively).

The –v simply means verbose output and can be safely omitted.  The real magic is in the –agent command which instructs plink to use Pageant for the key:

image

Now we get the opportunity to actually “store” (i.e. cache or install) the key.  If we say yes, this adds the key to our Windows Registry:

image

Once we’ve completed this step, we can return to our command window and attempt our usage of git against our remote repository on either GitHub or BitBucket once more.  This time, we should have much more success:

image

And now everything works as it should!