NDepend In Review

Back in September of this year, I was contacted by Patrick Smacchia, the lead developer from the NDepend team.  If you're not familiar with NDepend, it's a static analysis tool for .NET.  Patrick had asked me if I would be interested in reviewing the software.  I don't normally write software reviews on my blog, but on this occasion, I was intrigued.  I'm a long time user of ReSharper in my day to day work and ReSharper offers some amount of static analysis of your code whilst you write it.  I'd also been watching certain open source projects that had appeared since the introduction of the Roslyn compiler technology such as RefactoringEssentials, Code Cracker & SonarAnalyzer.  I've been thinking a lot about static analysis of code recently, especially since I discovered Connascence and started using that as a way to reason about code.

I had previously heard of the NDepend tool, but had never used it, so when Patrick contacted me and asked if I'd like to review his static analysis tool, I was more than happy to do so.

Full Disclosure:
NDepend is a commercial tool with a trial period.  Patrick agreed to provide me with a complimentary professional license for the software if I wrote a review of the software on my blog.  I agreed on the condition that the review would be my own, honest opinion of the software, good or bad.  What follows is that personal opinion.

This is a fairly long blog post so if you want the TL;DR skip to the end and the "Conclusion" section, but I suggest that you do read this whole post to better understand my complete experience using the NDepend product.


Getting Started

The first thing to getting up and running with NDepend was to download the software from the NDepend website.  This was easy enough, simply by following the relevant links from the NDepend homepage.  The version of NDepend that is current as I write this is v2017.3.2.  As I already had a license, I was able to enter my license key that had been previously supplied to me via email and begin the download.  NDepend comes as a .zip archive file rather than an MSI or other installer program.  This was somewhat unusual as a lot of software is delivered via a setup installer package these days, but I'm an old-school, command line kind of guy and I liked that the program was supplied as a .zip file that you simply extract into a folder and go.  After having done that, there's a couple of ways you can launch NDepend.  There's a few executable files in the extracted folder, so it's not immediately obvious what you execute first, but NDepend does have a helpful getting started guide on their website that's easy enough to follow along with.

For usage on a development machine, you're likely to be using either the stand alone Visual NDepend IDE (VisualNDepend.exe) or, perhaps more likely, the Visual Studio plugin for interacting with NDepend (installed by running the NDepend.VisualStudioExtension.Installer.exe installer).  As well as being used on a developer's machine, NDepend can also run on your build machine, and this is most likely to be integrated into your build process via the NDepend command line tool (NDepend.Console.exe). 

NDepend also ships with an executable called NDepend.PowerTools.exe and its source code in the NDepend.PowerTools.SourceCode folder.  This is a kind of optional extra utility which is run from the command line and contains a number of predefined metrics that can be run against some .NET code.  NDepend also provides an API with which we can integrate and use the NDepend functionality from our own code.  Many of the metrics of the NDepend Powertools are also used within the "main" NDepend tool itself, but as the source code is supplied to the Powertools utility, we can see exactly how those metrics are calculated and exactly how the various statistics around the code that NDepend is analysing is gathered from the NDepend API.  In essence, as well as providing some handy metrics of its own, the NDepend Powertools also serves as kind of demonstration code for how to use the NDepend API.


Launching NDepend For The First Time

After installing the Visual Studio plugin and launching Visual Studio, I loaded in a solution of source code that I wanted to analyse.  I opted to initially launch NDepend via the installed Visual Studio plugin as I figured that is the place I'd be interacting with it most.  NDepend installs itself as a extra menu option in the Visual Studio menu bar, much like other extensions such as ReSharper does.  When you first view this menu on a new solution, most of the menu options are greyed out.  Initially I was slightly confused by this but soon realised that as well as you having a Visual Studio solution and one or more project files for your own code, so too does NDepend have its own project files, and you need to create one of these first before you can perform any analysis of the current loaded solution.  NDepend helpfully allows this with one simple click on the "Attach New NDepend project to current VS Solution" menu item.  Once selected, you're asked which assemblies within your solution you wish to have analysed and then NDepend will go off and perform an initial analysis of your code.

NDepend uses both your solution's source code as well as the compiled assemblies to perform its analysis.  Most of the analysis is done against the compiled code and this allows NDepend to provide statistics and metrics of the IL (or Common Intermediate Language) that your code will compile to as well as statistics and metrics of other code areas.  NDepend uses the solution's source code during analysis in order to gather metrics relating to things not present inside the IL code such as code comments and also to gather statistics allowing NDepend to locate code elements such as types, methods and fields when these are shown as part of a query or search (see later for further details).

Once the analysis is complete, you'll get a pop-up window informing you of the completion of the initial analysis and asking you what to do next.  You'll probably also get a new browser tab open up in your default browser containing the NDepend Analysis report.   It was at this point, I was initially quite confused.  Having multiple things pop-up after the analysis was complete was a little startling and quite overwhelming.  One thing I'd have rather seen is for the browser based report to not be loaded initially, but to have been a button to be clicked on ("View Analysis Report" on something similar) within the pop-up window.  This way, only one "pop-up" is appearing, which to me is a lot less jarring.

The analysis popup allows you to view the NDepend dashboard, the interactive graph or the NDepend Code rules.  I hadn't read the online documentation, so it's probably mostly my own fault, but I'm the kind of person who just likes to dive straight in to something and try to figure things out for myself, but at this point, I was quite stuck.  I didn't quite know what to do next.  The help text on the popup indicates that the Dashboard is the best place to start, and so I selected that and was greeted with the following screen:

Now, this is an incredibly informative dashboard, with an awful lot of information contained within, but for me (and again, quite possibly my own fault for not reading the documentation more thoroughly) this was even more confusing.  There's a few things on the dashboard that seem to be obvious and make sense, such as the number of lines of code, the number of lines of comments and the percentage compared to actual code, but a lot of the other information around the debt level, quality gates and rules didn't seem to make much sense to me at all.  I decided I'd look at a few of the other options that were available in the popup, however that initial popup had now disappeared, so I had to go hunting through the NDepend menus.  Under the "Graph" menu, I found the "View Dependency Graph" option which seemed to coincide with one of the buttons I'd seen on the initial post-analysis pop-up.  Sure enough, opting to view the dependency graph showed me a nice UI of the various assemblies within my solution and how they were all related.

The Dependency Graph is a very nice way to get an overview of your solution.  It should be familiar to developers who have used the Enterprise SKU of Visual Studio and have used the various Architecture features, specifically the Code Map.  What's great about NDepend's Dependency Graph is the ability to alter the box size for the assemblies based upon various metrics for the assembly.  By default, it's based upon lines of code for the assembly, but can be changed to Cyclomatic Complexity, In/Out Edges, two types of coupling and overall "ranking" to name but a few.  The thickness of the arrows can also be altered based upon various metrics of the code such as namespaces, types, methods etc.  This gives the Dependency Graph view a great amount of power in seeing an overview of you solution, and specifically to see where complexity and potential issues with the code may lie. 

An alternative view of the dependency data of your solution can be found with the dependency matrix view.  This shows assemblies by name duplicated across two axes and showing the assembly metrics where the two assemblies intersect.

The assembly matrix view is very handy to use on larger solutions that may contain dozens of different assemblies as the Dependency Graph view of such a solution can be quite busy and difficult to follow due to being overloaded with information.  Another very useful visual representation of solution and project metrics is the Code Metrics View (aka the Treemap Metrics View).  This again shows all assemblies in the solution but this time as a series of coloured boxes.  Each box is further sub-divided into more coloured boxes of various sizes based upon the chosen metric.

The entire collection of boxes can be viewed at assembly, namespace, method, type or even field level with the size of the main boxes being determined by a similar set of metrics as the Dependency Graph, albeit with more options to choose from for method level metrics.  The colour of the boxes can be determined based upon a similar set of metrics as the size and is perhaps most frequently used to determine code coverage of the chosen element.  All of which gives a huge combination of options by which to gain a 100-foot view of the overall solution.  What's most interesting here is that the entire view can be based upon a specific "filter" or "query" performed over the entire codebase.

A "filter" or "query" over the codebase?  Well, this is where something called CQLinq comes in, and it's the very heart of the NDepend product.


Understanding NDepend

It took me a while to overcome the initial confusion and feeling of being overwhelmed by NDepend when I first started using it, however, after some time, I started to understand how the product hangs together, and it all starts with the foundation of everything else that NDepend offers, be that dependency graphs, code metrics, quality rules or other features.  The foundation of all of the impressive metrics and data that you can get from NDepend is CQLinq.

CQLinq stands for Code Query Linq and is at the heart of NDepend's features and functionality and is, essentially, a domain-specific language for code analysis.  It's based upon Linq, which is the Language Integration Query functionality that's been part of the .NET Framework since version 3.5.  For any developer who has used Linq to query in-memory objects, you'll know just have powerful and flexible a tool Linq can be.  It's even more powerful when Linq is used with a Linq Provider that can target some collection of external data such as a database.   Linq allows us to perform incredibly powerful filters, aggregations, projections and more, all from either a SQL-like syntax, or a very intuitive fluent chaining syntax.  For example:

var totalOutstanding = Customers.Where(c => c.Type == 1).Sum(c => c.Balance);

shows a simple but powerful query allowing the developer to, in a single line of code, get the total sum of all balances for a given type of customer from an arbitrary sized collection of customers, based upon properties of the customer objects within that collection.  This is just one example of a simple Linq query, and far more sophisticated queries are possible with Linq without adding too much additional complexity to the code.  The data here, of course, is the collection of customers, but what if that data was your source code?   That's where CQLinq comes in.

CQLinq takes your source code and turns it into data, allowing you, the developer, to construct a Linq query that can filter, aggregate and select metrics and statistics based upon viewing your code as data.  Here's a fairly simple CQLinq query:

from a in Application.Assemblies 
where a.NbLinesOfCode >= 0 
orderby a.NbLinesOfCode descending 
select new { a, a.NbLinesOfCode }

Here we're saying, look in all the assemblies within the solution that have greater than 0 lines of code.  We order those assemblies by the highest count of lines of code descending then we select a projected anonymous type showing the assembly itself and the count of its lines of code.  The result is a list similar to the following:

MyApp.Admin			8554
MyApp.Core			6112
MyApp.Data			4232
MyApp.DataAccess	3095
MyApp.Services		2398

So, even with this simple query, we can see the expressive power of a simple CQLinq query and the aggregate data we can obtain from our source code.

Now, the real power of CQLinq lies in the custom objects and properties that it makes available for you to use.  In the query above we can see the first line is selecting an a from Application.AssembliesApplication is an object provided by NDepend, and Assemblies is a collection of objects associated with the Application.  Further in the query, we can see that the variable a, that represents an assembly, has a property called NbLinesOfCode.  This custom property is also provided by NDepend and allows us to know how many lines of actual code the assembly contains.

NDepend, and CQLinq, provide a large number of custom objects (NDepend calls them custom domains) and properties out of the box giving insight into all sorts of different metrics and statistics on the source code that has been analysed by NDepend.  Custom objects/domains such as Assemblies, Types, Methods, Namespaces and custom properties such as IsClass, DepthOfInheritance, CouldBePrivate, CyclomaticComplexity, NbLinesOfCode, MethodsCalled, MethodsCallingMe, to give only the very briefest of overviews, contain the real power of CQLinq, and ultimately, NDepend.

Once you come to appreciate that underneath the surface of all of the other features of NDepend - such as the large array of metrics shown on the NDepend Dashboard and the NDepend Analysis Report - is one or more CQLinq queries slicing and dicing the data of your source code, everything starts to become that bit more clear!  (Or at least, it did with me!)


Code Rules, Quality Gates & Metrics

Having understood that most of the power of NDepend stems from the data provided by CQLinq queries, we can revisit the various metrics on the Dashboard.  Here, we can see that amongst the most obvious metrics such as lines of code and the number of assemblies, types and namespaces etc. there are more interesting metrics regarding Quality Gates, Rules and Issues.

If we click on the numbers next to the Quality Gates that have either failed, warned or passed, NDepend opens up a Queries and Rules explorer window and a Queries and Rules Edit window.

Within the Queries and Rules Explorer, we can see a treeview of a number of different rules, grouped together by category.  These are code quality rules that ship out of the box with NDepend, and include such categories as Code Smells, Architecture, Naming Conventions and many others.  The icons next to the various category groups show us whether our source code that has been analysed has failed (i.e. violates a rule), is considered a warning, or passes for each rule defined.  If we click on a rule category within the treeview, we see the individual rules from that category in the window to the right.  This shows us how much of our code has matched with the fail/warn/pass states.  Clicking on the individual rule name will cause the Queries and Rules Edit window to show the details for the chosen rule.  Note that, by default and for me at least, the Queries and Rules Explorer Window was docking along the bottom of the Visual Studio main window, whilst the Queries and Rules Edit Window was docking to the right of the main Visual Studio window (as a new tab along with my Solution Explorer).  It was initially confusing as I hadn't noticed one window being updated when interacting with the other window, but once you're aware of this, it becomes much easier to understand.  One other quirk of the Queries and Rules Explorer window is that there doesn't appear to be any way to search for a specific rule within the many rules contained in the various category groups and I found myself sometimes manually expanding each of the category groups in turn in order to find a rule I'd previously been examining.  It would be great if the ability to search within this window was introduced in a future version of NDepend.

The Queries and Rule Edit window is divided into a bottom section that shows the various parts of your source code (the assembly, type or method) grouped in a treeview that are matched by the chosen rule.  In the top section, we can see the rule name and a textual description of the rule itself.  For example, when looking at one of the Code Smells category rules, Avoid Types Too Big, we can see that the description states, "This rule matches types with more than 200 lines of code. Only lines of code in JustMyCode methods are taken account.  Types where NbLinesOfCode > 200 are extremely complex to develop and maintain.".

Hang on.  NbLinesOfCode?   We've seen that before.  Sure enough, clicking the "View Source Code" button at the top of the top section of the Queries and Rules Edit window changes the top section view into something else.  A CQLinq query!

Note that if the Queries and Rules Edit window is docked to the right in its default position, the overall window can appear a little cramped.  For me, this made it a little unclear exactly what I was looking at until I undocked the window and expanded its size.

Each and every one of the more than 200 rules that ship out of the box with NDepend are based upon a CQLinq query.  What's more is that each and every one of these queries can be edited.  And you can create your own category groups and rules by defining your own CQLinq queries.  Think about that for a moment as this is really what NDepend is providing you with.  Lots and a lots of pre-defined query and examination power right out of the box, but mostly the ability to have a full-featured, fluent and highly intuitive Linq-To-Code (for want of a better expression) provider that you can make full use of to perform all manner of examinations on your own source code.

Here's the pre-defined CQLinq query for determining which methods are in need of refactoring:

warnif count > 0 from m in JustMyCode.Methods where 
  m.NbLinesOfCode > 30 ||           
  m.CyclomaticComplexity > 20 ||    
  m.ILCyclomaticComplexity > 50 ||  
  m.ILNestingDepth > 5 ||           
  m.NbParameters > 5 ||             
  m.NbVariables > 8 ||              
  m.NbOverloads > 6                 

select new { m, m.NbLinesOfCode, m.NbILInstructions, m.CyclomaticComplexity, 
             m.ILCyclomaticComplexity, m.ILNestingDepth, 
             m.NbParameters, m.NbVariables, m.NbOverloads }

As can be seen above, the query makes extensive use of the built-in computed properties, all of which are very intuitively named, to gather a list of methods throughout the solution, and specifically methods that are from "JustMyCode" meaning that NDepend helpfully filters out all methods that belong to external frameworks and libraries, that fall afoul of the rules of this query and therefore are prime candidates for refactoring.  And, of course, if you don't like that NDepend only considers methods with more than 30 lines of code to be in need of refactoring, you can always edit the query and change that value to any number you like that better fits you and your team.

When running NDepend from the Visual Studio plug-in you get the ability to click on any of the results from the currently selected query/rules in the Queries and Rules Edit window and be taken directly to the type or method detected by the rule.   This is a really powerful and easy way to navigate through your code base to find all of specific areas of code that may be in need of attention.  I did, however, notice that not all matched code elements can be easily navigated to (see below for more detail).

NDepend also has extensive tooltip windows that can pop-up giving all manner of additional information about whatever you've selected and is currently in context.  For example, hovering over results in the Query and Rules Edit window shows a large yellow tooltip containing even more information about the item you're hovering over:

The "View Source Code" button within the Queries and Edit window acts as a toggle switch between viewing the source code of the CQLinq query and the textual description of the query.  The textual description often also includes text indicating "How To Fix" any of your code that violates the rule that the query relates to.  This can be a very handy starting point for identifying the first steps to take in refactoring and improving your code.

When editing CQLinq queries, the Queries and Edit window provides full intellisense whilst editing or writing query syntax, as well as even more helpful tooltip windows that give further help and documentation on the various CQLinq objects and properties used within the query.

NDepend allows individual rules to be designated as "Critical" rules and this means that any code found that falls afoul of a critical rule will result in a "failure" rather than a "warning".  These are the rules that you can define (and many are predefined for you, but you can always change them) which should not be violated and often such rules are included as part of NDepend's Quality Gates.

Quality Gates are the way that NDepend can determine an overall status of whether the code rules upon which the quality gate is based has passed or failed.  A Quality Gate is something that, like the code rules themselves, can used as-is out of the box, be modified from an existing one or be completely defined by yourself.  Quality gates can rely on one or more code rules in order to determine if the analysed source code is considered good enough to be released to production, so whereas the individual rules themselves will return a list of types, methods or other code artifacts that have matched the query, quality gates will return a pass, warn or fail status that can be used as part of an automated build process in order to fail the build, similar to how compilation errors will fail the compilation whereas warning won't.  Very powerful stuff.  Quality gates are defined using the same CQLinq syntax we've seen before:

// <QualityGate Name="Percentage Code Coverage" Unit="%" />
failif value < 70%
warnif value < 80%

Note that the first comment line actually defines this query as a quality gate and the first two lines of the query show the relative values from the query that should trigger the different return statuses of warning vs failure.

As well as code rules and quality gates, NDepend contains other metrics such as a SQALE Debt Ratio and other debt ratings.  The debt here is the technical debt that may exist within the code base.  Technical Debt is defined as the implied cost of rework/refactoring of parts of the solution in order to fix elements of code that are considered to be of lower quality.  The SQALE Ratio is a well defined, largely industry accepted mechanism for calculating the level of debt within a software project and is expressed as a percentage of the estimated technical-debt, compared to the estimated effort it would take to rewrite the code element from scratch.  NDepend uses the SQALE ratio to show a "Debt Rating", expressed as a letter from A (the least amount of debt) through to E (the most amount of debt), that is attached to the overall solution as well as individual elements of code such as types and methods and this debt rating can be seen on many of the tooltips such as the one shown when hovering over Query and Rules Edit window results as well as on the NDepend dashboard and Analysis Report.  There's a lot more metrics provided by NDepend including a very interesting Abstractness vs Instability metric that's shown on the front page of the Analysis Report.  Speaking of which...


Back to the beginning

So, having taken a diversion to understanding the core foundations of what makes NDepend tick, we can go back to that dashboard and analysis report that we saw at the beginning just after we created a new NDepend project and ran an analysis on our code for the very first time.

The dashboard and report contain many of the same metrics but whilst the dashboard is a window inside Visual Studio (or the separate Visual NDepend IDE), the report is an entirely standalone HTML report and so can be sent to colleagues who may not have the requisite development tools on their machines and be viewed by them.  This also means that the Analysis report is the perfect artifact to be produced as part of your automated build process.  Many teams will often have a dashboard of some description that shows statistics and metrics on the build process and the quality of the code that's been built.  NDepend's analysis report can be easily integrated into such a dashboard, or even hosted on a website domain as it's a static HTML file with some linked images.

The analysis report also contains static snapshot versions of the various graphical metrics representations that can be viewed inside NDepend such as the Dependency Graph, Dependency Matrix, Treemap Metric View and another interesting graph which plots Abstractness vs Instability.

This graph is an interesting one and can give a good idea of which areas of the analysed solution are in need of some care and attention but for very different reasons.

The graph plots each assembly within the solution against a diagonal baseline that runs from the top left corner of the graph to the bottom right.  Starting at the bottom right, as we move along the x-axis, towards the left, we get more "instable" and moving up the y-axis towards the top, we get more "abstract".  Abstractness is how much that project depends upon abstract classes or interfaces rather than concrete classes.  This indicates that the project is somewhat easier to change and is more malleable.  Instability is how easy such change can be made without breaking the code by analysing the various inter-dependencies within that code.  This is a combination of two other metrics, those of afferent coupling and efferent coupling.  Afferent coupling is the number of types that depend upon the given assembly whilst efferent coupling is the number of external types that this assembly depends upon.  We want our assemblies to remain inside the diagonal green area of the graph.  If an assembly's code is high in abstractness but also has a very low amount of either internal or external coupling, we're headed towards the "zone of uselessness".  This isn't a place we want to be, but correcting this may be reasonably easy since code being well within this area means that it doesn't do an awful lot and so could probably be fairly easily removed.  If, however, an assembly's  code is both overly rigid in its dependence on concrete types and also rigid in its instability (i.e has a high amount of internal and/or external coupling) we're headed towards the "zone of pain".  This is an area that we really don't want to be in since code being well within this area means that it's probably doing a lot - maybe even too much - and it's going to be very difficult to change it without breaking things.

Keeping an eye on each assembly's place within the Abstractness vs Instability graph can help to ensure your code stays at both the correct level of abstraction without being too abstract and also ensuring that the code remains fairly well decoupled allowing easier modification.  Metrics such as the abstractness vs instability graph and the treemap metric view are some of my favourite features of NDepend as they present their data in a very easily consumable format that can be viewed and understood "at a glance" whilst being based on some very complex underlying data.


But wait, it gets better

Similar to the Queries and Edit window that allows us to define our own code queries and rules as well as modifying existing ones, NDepend includes a very powerful Search function. 

This function uses the same kind of interface as the Query and Rules Edit window, but allows arbitrary queries to be performed against the analysed code with the matching results showing immediately within the window.  NDepend's search functionality can be optionally constrained to methods, types, assemblies or namespaces, but by default will search across all code elements.  Searches can be based on simple text matching, regular expressions or even full CQLinq queries.  This makes NDepend's search functionality even more powerful than something like the "Go to anything/everything" functionality that's been a part of ReSharper for a long time and is now also a part of Visual Studio itself as we can not only search for pieces of our code based on textual matches, but we can also use the highly expressive power of NDepend's CQLinq syntax, objects and properties to search for code based upon such things as method size, code lacking unit tests, code with no callers or too many callers.  As with the Queries and Rules edit window results, we can double-click on any result and be navigated directly to the matched line of code.  One quirk I did notice when using the Search functionality is that, when searching for matching elements inside of a method - i.e. fields, or other certain elements of code such as interfaces, you must click on the containing method or type that's shown as part of the search results and not the actual interface or field name itself.  Clicking on these will cause NDepend to display various errors as to why it can't navigate directly to that element.  I suspect this is due to the fact that NDepend's analysis works on the compiled intermediate language and not directly against the source code, but since this is a code search function, it would be nice if such navigation were possible.

As well as NDepend's ability to analyse the source code of your solution, it can leverage other tools' artifacts and data to improve its own analysis.  One such major piece of data that NDepend can utilise is code coverage data.  There are a number of tools that can calculate the code coverage of your solution.  This is the amount of code within your solution that is "covered" by unit tests, that is code that is executed as part of one or more unit tests.  Code coverage is built into Visual Studio itself at the highest SKU level, Enterprise, but is also provided by a number of other tools such as JetBrains' dotCover, and NCover.  NDepend can import and work with the code coverage output data of all three of the aforementioned tools, although the files from the dotCover tool have to be provided to NDepend in a special format.  Once some code coverage data is added to an NDepend project, several additional metrics relating to code coverage become available within the complete quite of NDepend metrics, such as identifying code that should have a minimum level of coverage and code whose coverage percentage should never decrease over time as well as other interesting metrics such as the amusingly titled C.R.A.P. metric.  This is the "Change Risk Analyzer and Predictor" metric and gives methods a score based upon their level of code coverage versus their complexity with more complex methods requiring higher code coverage.  Having the ability to integrate code coverage into NDepend's metrics suite is another very nice touch and allows the full set of metrics for your solution to be centralised in a single location.  This also helps to keep automated builds fairly simple without too many moving parts whilst also providing an extensive amount of data on the build.

We've mentioned a few times how NDepend can be integrated into your automated build process.  There's nothing new in having such tools be able to be integrated with such a fundamental part of any software team's daily processes, but whereas some tools simply give you the ability to run them from a command line and then leave you to figure out the exact return values from the executable and other output artifacts and how they might be integrated, NDepend provides some very comprehensive online documentation (even with links to that documentation from the NDepend menus inside of Visual Studio) giving very precise instructions for integrating NDepend into a large number of modern Continuous Integration tools, such as Visual Studio Team Services, TeamCity, Jenkins, FinalBuilder & SonarQube.  This documentation not only includes extensive step-by-step guides to the integration, but also tutorial and walkthrough videos.  This is a very nice touch.

Finally, we've touched upon this above, but NDepend has the ability to perform its analysis in a temporal way.  This essentially means that NDepend can analyse the same solution at different times and then compare different sets of analysis results.  This allows the production of "trends" within the analysis data and is incredibly powerful and possibly one of the most useful features of the NDepend product.  We saw earlier how the Analysis report and the NDepend dashboard contain metrics for such things as Debt rating of the solution, but upon closer examination, we can see that the debt rating and percentage is shown as either an increase or a decrease since the last time that the solution's debt was calculated:

Having such data available to us is incredibly powerful.  We can now track these metrics over time to see if our solution is generally improving or decreasing in quality.  Something that is very important for any software development team who must maintain a solution over a long period of time.   NDepend allows the setting of a "baseline" set of metrics data against which new sets of recently analysed data can be compared.  This gives us a powerful ability to not only compare our most recent set of analysis results with the immediately prior set of results, which if NDepend's analysis is integrated into a continuous build process could be very recently indeed, but also being able to compare our most recent set of results with those of a day ago, a week or a month ago, or even this time last year.  With this we can see not only how our code changes over small time frames, but over larger units of time too.  This ability to view metrics over time frames is helpfully built into many of the interfaces and windows within NDepend, so for example, the Dashboard contains a very useful filter at the top of the window allowing us to set the time range for the charts shown within the dashboard.  NDepend includes the ability to generate a full set of metrics around trends within the solution, so we can for example, track such things as how many "issues" that an earlier NDepend analysis identified have been fixed as well as how many new issues have been introduced, and many of the built-in rules that NDepend defines are specifically based upon changes to the code over time.  For example, there is a complete category group, "Code Smells Regression" that contains many rules starting with the name "From now...".  These specific rules will be broken if code quality for the specific attribute measured falls over time, from one analysis run to the next.  This helps to ensure that code is not only improved in quality, but stays that way.  NDepend doesn't stop there with its ability to view changes over time, and the CQLinq query language includes a large amount of properties that specifically make use of the ability to examine such change in the code over time.  Objects/Domains such as Issues and IssuesInBaseline, and properties such as NewerVersion on a method can allow us to write arbitrary queries comparing parts of our code over time.  So, for example, the following query shows us which methods have increased in cyclomatic complexity:

from m in Methods
where m.NewerVersion().CyclomaticComplexity > m.OlderVersion().CyclomaticComplexity
select m

This is exceptionally powerful data to have on a software project developed and maintained by a commercial team as it allows very informed decisions to be made regarding how much effort within a given amount of work (say, an agile sprint) should be dedicated not to adding additional features and functionality but to maintenance of the existing code that's already there.  Moreover, not only do we know how much effort we should expend on code maintenance but also we know exactly where that effort should be directed to have maximum impact in improving code quality - a critical factor in the success of any software project.



So, we've looked at NDepend and examined some of its powerful static analysis features and we've seen how such a tool could be integrated into a broader software development life-cycle, but should you actually use NDepend for your static analysis needs?

NDepend is a commercial tool and its license comes in two flavours.  A Developer license is required for each developer that will use NDepend on their own development machine, and this license is currently priced at 399 euros.  If you want to use NDepend as part of your Continuous Integration process on a build machine, you'll need a separate license for that which is currently priced at 799 euros.  All licenses are for one year and can be renewed with a 40% discount on the initial purchase price.  All ongoing licenses are eligible for continued upgrades to the product.  All licenses are perpetual fallback licenses meaning that you can stop renewing and still keep using your current version, although you'll lose the ability to upgrade to newer versions of the product.

As a software tool, it's expensive but not exorbitantly so.  For any commercial software development team that cares about the quality of their code, and especially a team tasked with maintaining a software product over time, NDepend is not really expensive at all when compared to the cost of other tools that typical software development teams will invest in.  In such an environment, NDepend is well worth the money.  NDepend has the ability to fit in with most modern team's processes and workflows and has the power to be able to be configured and tweaked to any given team's requirements regarding what constitutes good vs bad code and the level of "quality" that the code has to adhere to before it's considered good enough for production release.  This gives the tool incredible power and flexibility.  That said, I think it's perhaps too expensive a tool for individual software developers who may wish to purchase just for themselves.  Also, to the best of my knowledge, there's no free version of NDepend available for open source projects, as there are of other tools such as ReSharper, OzCode, TeamCity, AppVeyor and many more.  For this reason, I'd love to see another SKU or edition of the NDepend product that is more affordable for individual developers.  Perhaps an "NDepend Lite" that ships only as a Visual Studio plug-in and removes some functionality such as the ability to edit CQLinq queries and the tracking of metrics over time and having a more affordable price for individual developers.

Setup of NDepend is very easy, being delivered as a simple .zip archive file.  The integration of the Visual Studio plugin is also very simple to install and use, and registering your software license is straight-forward and uncomplicated so you can be up and running with the NDepend product in a matter of minutes.  In a way, this is deceptive as once you're back in Visual Studio (or even the Visual NDepend stand alone IDE) it can be difficult to know exactly where to start.  Again, this is largely my own fault for trying to dive straight in without referring to the documentation.  And it must be said that the NDepend's documentation is exceptionally good.  Although newer versions of NDepend try to improve ease of accessibility into the software, the newly added "guide tool-tips" are often not necessarily easily discoverable until you stumble onto them resulting in a bit of a chicken-and-egg situation.  It's hard to really criticise the product for this, though, as by its very nature, it is a highly technical and complex product.  Having done things the hard way, my recommendation would be to spend some time reading the (very good) documentation first and only then diving into the product.

Although the initial learning curve can be quite steep, usage of the product is very easy, especially once you get familiar with the various functions, menu options and windows. And if you're stuck at any point, the excellent documentation and handy tool-tips are always only a mouse click (or hover) away.  Also, NDepend is quick.  Very quick.  Once analysis of a solution is completed - using my test solution, NDepend was able to perform its analysis in less than half the time it takes to fully compile and build the solution - the complete code-base can be queried and searched in near real-time.  And as well as providing a plethora of highly informative and profoundly useful metrics on the analysed code, NDepend also contains such functionality as a general-purpose "code-search-on-steroids" that give other tools offering similar functionality (i.e ReSharper) a run for their money.

NDepend isn't perfect, and I did discover a few quirks with the product as I used it.  One quirk seemed to be a bit of a false positive which was a broken built-in rule that stated that your own methods shouldn't be called "Dispose".  The section of code that was identified as breaking this rule did indeed have a Dispose() method, but it was necessary due to interface implementation.  In this case, it wasn't the IDisposable interface that the class was implementing, but rather the IHttpModule interface that forced me to implement the Dispose() method, yet NDepend didn't seem to like that and flagged that as a broken rule.  NDepend also has a rule about the names of methods not being too long, however, I was finding that my unit tests were frequently falling afoul of this rule as most of them had quite long method names.  This is quite normal for unit tests, and is often considered good or best-practice to name unit tests in such a way with a very expressive name.  You can always decide not to include the test assemblies in the NDepend analysis, but then you may miss out on other NDepend rules that would be very helpful, such as warning when a method has too many lines of code.  It would be nice to be able to tell NDepend to omit certain files or assemblies from certain rules or categories of rules similar to how you can tell ReSharper to ignore certain inspections via comments within your code.  You can omit code from queries at the moment, however, in order to achieve this you have to manually edit the CQLinq of the query, which isn't the best or simplest way.  On the plus side, NDepend have said that they will support a [SuppressWarning] attribute as well as custom views of code within queries, making ignoring certain code much easier in a future version of the product.

Another quirk I found was that navigating to certain code elements from either the search results or results of code rules in the Queries and Rules Edit window can sometimes be problematic.  One rule states "Avoid interfaces too big" and so the results shown in the window are all matched interfaces that the rule has deemed too big.  However, trying to navigate to the interface by double-clicking on the matching result gives an error with a reason of "N/A because interface", which is quite confusing and somewhat unintuitive.  It would be nice if navigation to code via double-clicking on query results was more universal, meaning that all results for all code elements would navigate to the corresponding section of source code when clicked.

An omission from NDepend's functionality that I'd love to see included out-of-the-box would be the ability to identify "code clones" - sections of code that are duplicated in various different parts of the solution.  Whilst this isn't a "metric" as such, it's certainly one of the first ports of call that I turn to when looking to refactor and improve an existing large code base.  In incumbent code-bases, there is frequently a fair amount of duplicated code that exists and removing as much duplication as you possibly can not only reduces the number of lines of code (so long as readability and functionality are maintained, less code is always better than more code) but it also helps in the refactoring of methods against other metrics such as cyclomatic complexity and other factors.  NDepend does include a "Search for Duplicate Code" powertool although in my experience of comparing its output to that of the "Analyze solution for Code Clones" option in Visual Studio Enterprise, I found that it didn't detect as many instances of duplicated code.

This is almost certainly due to the difference in how the two tools work - the NDepend feature looks for methods that make many calls to a "set" of other methods, whilst the Visual Studio function will look for duplicates of actual code itself.  The NDepend feature works in the way it does no doubt due to examining the compiled code rather than the raw source code.  However, in the test solution that I was using, which I knew contained many sections of identically duplicated code inline within many different methods,  NDepend failed to identify a lot of the duplicated code (not to mention taking significantly longer to perform the analysis - a whopping 42 minutes for a 38KLOC solution), whilst the Visual Studio feature detected them all in less than 5 minutes.  Also, as the NDepend functionality is only provided from the command line invoked Powertools, it's not easy to immediately navigate to the offending lines of code by double-clicking as we can with features that are part of the Visual Studio plug-in.  The good news, though, is that in speaking to Patrick, the NDepend lead developer, he tells me that a first-class in-built function for detecting code duplicates is on the roadmap for a future version of the NDepend product.  Hopefully, this functionality shouldn't be too long in coming as NDepend is a very actively developed product, receiving frequent updates.

One other feature I'd love to see in a future version of NDepend would be some dashboard metrics or query rules around identifying connascence within a software solution. Admittedly, some of the levels of connascence are dynamic and so can't really be identified without running the actual code, however, the first five types of connascence can definitely be identified via static analysis and it'd be great if NDepend could include identification of these out-of-the-box.

Overall, I would definitely recommend NDepend if code quality is important to you.  For any commercial software team charged with maintaining a code-base over a period of time, quality of that code-base is of critical importance to the overall success of the software and the ability and ease of continued development.  A tool like NDepend provides great power and insight into the quality of the code-base both right now as well as how that quality fluctuates over time.  NDepend's ability to both aggregate quality metrics into a single, simple debt percentage or rating as well as its ability to identify the many, individual specific issues that the code-base suffers from and how those issues are introduced or resolved over time is a huge help for any team in knowing not only exactly when to focus effort on quality improvement but where to focus that effort.

DDD North 2017 In Review

IMG_20171014_085217On Saturday, 14th October 2017, the 7th annual DDD North event took place.  This time taking place in the University of Bradford.

IMG_20171015_171513One nice element of the DDD North conferences (as opposed to the various other DDD conferences around the UK) is that I'm able to drive to the conference on the morning of the event and drive home again after the event has finished.  This time, the journey was merely 1 hour 20 minutes by car, so I didn't have to get up too early in order to make the journey.  On the Saturday morning, after having a quick cup of coffee and some toast at home, I headed off towards Bradford for the DDD North event.

IMG_20171014_085900After arriving at the venue and parking my car in one of the ample free car parks available, I headed to the Richmond Building reception area where the conference attendees were gathering.  After registering my attendance and collecting my conference badge, I headed into the main foyer area to grab some coffee and breakfast.  The catering has always been particularly good at the DDD North conferences and this time round was no exception.  Being a vegetarian nowadays, I can no longer avail myself of a sausage or bacon roll, both of which were available, however on this occasion there was also veggie sausage breakfast rolls available too.   A very nice and thoughtful touch!  And delicious, too!

After a some lovely breakfast and a couple of cups of coffee, it was soon time to head off the the first session of the day.  This one was to be Colin Mackay's User Story Mapping For Beginners.

IMG_20171014_093309Colin informs us that his talk will be very hands-on, and so he hands out some sticky notes and markers to some of the session attendees, but unfortunately, runs out of stock of them before being able to supply everyone.

Colin tells us about story mapping and shares a quote from Martin Fowler:

"Story mapping is a technique that provides the big picture that a pile of stories so often misses"

Story mapping is essentially a way of arranging our user stories, written out on sticky notes, into meaningful "groups" of stories, tasks, and sections of application or business functionality.  Colin tells us that it's a very helpful technique for driving out a "ubiquitous language" and shares an example of how he was able to understand a sales person's usage of the phrase "closing off a customer" to mean closing a sale, rather than the assuming it to mean that customer no longer had a relationship with the supplier.  Colin also states that a document cannot always tell you the whole story.  He shares a picture from his own wedding which was taken in a back alley from the wedding venue.  He says how the picture appears unusual for a wedding photo, but the photo doesn't explain that there'd been a fire alarm in the building and all the wedding guests had to gather in the back alley at this time and so they decided to take a photo of the event!  He also tells us how User Story Mapping is great for sparking conversations and helps to improve prioritisation of software development tasks.

Colin then gets the attendees that have the sticky notes and the markers to actually write out some user story tasks based upon a person's morning routine.  He states that this is an exercise from the book User Story Mapping by Jeff Patton.  Everyone is given around 5 minutes to do this and afterwards, Colin collects the sticky notes and starts to stick them onto a whiteboard.  Whilst he's doing this, he tells us that there's 3 level of tasks with User Story Mapping.  At the very top level, there's "Sea" level.  These are the user goals and each task within is atomic - i.e. you can't stop in the middle of it and do something else.  Next is Summary Level which is often represented by a cloud or a kite and this level shows greater context and is made up of many different user goals.  Finally, we have the Sub-functions, represented by a fish or a clam.  These are the individual tasks that go to make up a user goal. So an example might have a user goal (sea level) of "Take a Shower" and the individual tasks could be "Turn on shower", "Set temperature", "Get in shower", "Wash body", "Shampoo hair" etc.

After an initial arrangement of sticky notes, we have our initial User Story Map for a morning routine.  Colin then says we can start to look for alternatives.  The body of the map is filled with notes representing details and individual granular tasks, there's also variations and exceptions here and we'll need to re-organise the map as new details are discovered so that the complete map makes sense.  In a software system, the map becomes the "narrative flow" and is not necessarily in a strict order as some tasks can run in parallel.  Colin suggests using additional sticker or symbols that can be added to the sticky note to represent which teams will work on which parts of the map.

Colin says it's good to anthropomorphise the back-end systems within the overall software architecture as this helps with conversations and allows non-technical people to better understand how the component parts of the system work together.  So, instead of saying that the web server will communicate with the database server, we could say that Fred will communicate with Bob or that Luke communicates with Leia. Giving the systems names greater helps.

We now start to look at the map's "backbone".  These are the high level groups that many individual tasks will fit into.  So, for our morning routine map, we can group tasks such as "Turn off alarm" and "Get out of bed" as a grouping called "Waking up".  We also talk about scope creep.  Colin tells us that, traditionally, more sticky notes being added to a board even once the software has started to be built is usually referred to as scope creep, however, when using techniques such as User Story Mapping, it often just means that your understanding of the overall system that's required is getting better and more refined.

IMG_20171014_101304Once we've built our initial User Story Map, it's easy to move individual tasks within a group of tasks in a goal below a horizontal line which was can draw across the whiteboard.  These tasks can the represent a good minimum viable product and we simply move those tasks in a group that we deem to be more valuable, and thus required for the MVP, whilst leaving the "nice to have" tasks in the group on the other side of the line.  In doing this, it's perfectly acceptable to replace a task with a simpler task as a temporary measure, which would then be removed and replaced with the original "proper" task for work beyond MVP.  After deciding upon our MVP tasks, we can simply rinse and repeat the process, taking individual tasks from within groups and allocating them to the next version of the product whilst leaving the less valuable tasks for a future iteration. 

Colin says how this process results in producing something called "now maps" as the represent what we have, or where we're at currently, whereas what we'll often produce is "later maps", these are the maps that represent some aspect of where we want to be in the future.  Now maps are usually produced when you're first trying to understand the existing business processes that will be modelled into software.  From here, you can produce Later maps showing the iterations of the software as will be produced and delivered in the future.  Colin also mentions that we should always be questioning all of the elements of our maps, asking question such as "Why does X happen?", "What are the pain points around this process?", "What's good about the process?" and "What would make this process better?".  It's by continually asking such questions, refining the actual tasks on the map, and continually reorganising the map that we can ultimately create great software that really adds business value.

Finally, Colin shares some additional resources where we can learn more about User Story Mapping and related processes in general.  He mentions the User Story Mapping book by Jeff Patton along with The Goal by Eli Goldratt, The Phoenix Project by Gene Kim, Kevin Behr and George Spafford and finally, Rolling Rocks Downhill by Clarke Ching.

After Colin's session is over, it's time for a quick coffee break before the next session.   The individual rooms are a little distance away from the main foyer area where the coffee is served, and I realised by next session was in the same room as I was already sat!  Therefore, I decided I'm simply stay in my seat and await the next session.  This one was to be David Whitney's How Stuff Works...In C# - Metaprogramming 101.

IMG_20171014_104223David's talk is going to be all about how some of the fundamental frameworks that we use as .NET developers everyday work and how they're full of "metaprogramming".  Throughout his talk, he's going to decompose an MVC (Model View Controller) framework, a unit testing framework and a IoC (Inversion of Control) container framework to show they work and specifically to examine how they operate on the code that we write that uses and consumes these frameworks.

To start, David explains what "MetaProgramming" is.  He shares the Wikipedia definition, which in typical Wikipedia fashion, is somewhat obtuse.  However the first statement does sum it up:

"Metaprogramming is a programming technique in which computer programs have the ability to treat programs as their data."

This simply means that meta programs are programs that operate on other source code, and Meta programming is essentially about writing code that looks at, inspects and works with your own software's source code.

David says that in C#, meta programming is mostly done by using class within the System.Reflection namespace and making heavy use of things such as the Type class therein, which allows us to get all kinds of information about the types, methods and variables that we're going to be working with.  David shows a first trivial example of a meta program, which enumerates the list of types by using a call to the Assembly.GetTypes() method:

public class BasicReflector
	public Type[] ListAllTypesFromSamples()
		return GetType().Assembly.GetTypes();

	public MethodInfo[] ListMethodsOn<T>()
		return typeof(T).GetMethods();

He asks why you want to do this?  Well, it's because many of the frameworks we use (MVC, Unit Testing etc.) are essentially based on this ability to perform introspection on the code that you write in order to use them.  We often make extensive use of the Type class in our code, even when we're not necessarily aware that we're doing meta-programming but the Type class is just one part of a rich "meta-model" for performing reflection and introspection over code.  A Meta-Model is essentially a "model of your model".  The majority of methods within the System.Reflection namespace that provide this Metamodel usually end with "Info" in the method name, so methods such as TypeInfo, MethodInfo, MemberInfo and ConstructorInfo can all be used to give us highly detailed information and data about our code.

As an example, a unit testing framework at it's core is actually trivially simple.  It essentially just finds code and runs it.  It examines your code for classes decorated with a specific attribute (i.e. [TestFixture]) and invokes methods that are decorated with a specific attribute(s)(i.e. [Test]).  David says that one of his favourite coding katas is to write a basic unit testing framework in less than an hour as this is a very good exercise for "Meta Programming 101".

We look at some code for a very simple Unit Testing Framework, and there's really not a lot to it.  Of course, real world unit testing frameworks contain many more "bells-and-whistles", but the basic code shown below performs the core functionality of a simple test runner:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;

namespace ConsoleApp1
    public class Program
        public static void Main(string[] args)
            var testFinder = new TestFinder(args);
            var testExecutor = new TestExecutor();
            var testReporter = new TestReporter();
            var allTests = testFinder.FindTests();
            foreach (var test in allTests)
                TestResult result = testExecutor.ExecuteSafely(test);

    public class TestFinder
        private readonly Assembly _testDll;

        public TestFinder(string[] args)
            var assemblyname = AssemblyName.GetAssemblyName(args[0]);
            _testDll = AppDomain.CurrentDomain.Load(assemblyname);

        public List<MethodInfo> FindTests()
            var fixtures = _testDll.GetTypes()
                .Where(x => x.GetCustomAttributes()
                    .Any(c => c.GetType()
            var allMethods = fixtures.SelectMany(f => 
                f.GetMethods(BindingFlags.Public | BindingFlags.Instance));
            return allMethods.Where(x => x.GetCustomAttributes()
                .Any(m => m.GetType().Name.StartsWith("Test")))

    public class TestExecutor
        public TestResult ExecuteSafely(MethodInfo test)
                var instance = Activator.CreateInstance(test.DeclaringType);
                test.Invoke(instance, null);
                return TestResult.Pass(test);
            catch (Exception ex)
                return TestResult.Fail(test, ex);

    public class TestReporter
        public void Report(TestResult result)
            Console.Write(result.Exception == null ? "." : "x");

    public class TestResult
        private Exception _exception = null;
        public Exception Exception { get => _exception;
            set => _exception = value;

        public static TestResult Pass(MethodInfo test)
            return new TestResult { Exception = null };

        public static TestResult Fail(MethodInfo test, Exception ex)
            return new TestResult { Exception = ex };

David then talks about the ASP.NET MVC framework.  He says that it is a framework that, in essence, just finds and runs user code, which sounds oddly familiar to a unit testing framework!  Sure, there's additional functionality within the framework, but at a basic level, the framework simply accepts a HTTP request, finds the user code for the requested URL/Route and runs that code (this is the controller action method).  Part of running that code might be the invoking of a ViewEngine (i.e. Razor) to render some HTML which is sent back to the client at the end of the action method.  Therefore, MVC is merely meta-programming which is bound to HTTP.  This is a lot like an ASP.NET HttpHandler and, in fact, the very first version of ASP.NET MVC was little more than one of these.

David asks if we know why MVC was so successful.  It was successful because of Rails.  And why was Rails successful?  Well, because it had sensible defaults.  This approach is the foundation of the often used "convention over configuration" paradigm.  This allows users of the framework to easily "fall into the pit of success" rather than the "pit of failure" and therefore makes learning and working with the framework a pleasurable experience.  David shows some more code here, which is his super simple MVC framework.  Again, it's largely based on using reflection to find and invoke appropriate user code, and is really not at all dissimilar to the Unit Testing code we looked at earlier.  We have a ProcessRequest method:

public void ProcessRequest(HttpContext context)
	var controller = PickController(context);
	var method = PickMethod(context, controller);
	var instance = Activator.CreateInstance(controller);
	var response = method.Invoke(instance, null);

This is the method that orchestrates the entire HTTP request/response cycle of MVC.  And the other methods called by the ProcessRequest method use the reflective meta-programming and are very similar to what we've already seen.  Here's the PickController method, which we can see tries to find types whose names both start with a value from the route/URL and also end with "Controller".  We can also see that we use a sensible default of "HomeController" when a suitable controller can't be found:

private Type PickController(HttpContext context)
	var url = context.Request.Url;
	Type controller = null;
	var types = AppDomain.CurrentDomain.GetAssemblies()
					.SelectMany(x => x.GetTypes()).ToList();
	controller = types.FirstOrDefault(x => x.Name.EndsWith("Controller")
					&& url.PathAndQuery.StartsWith(x.Name)) 
					?? types.Single(x => x.Name.StartsWith("HomeController"));
	return controller;

Next, we move on to the deconstruction of an IoC Container Framework.  An IoC container framework is again a simple framework that works due to meta-programming and reflection.  At their core, they simply stores a dictionary of mappings of interfaces to types, and they expose a method to register this mapping, as well as a method to create an instance of a type based on a given interface.  This creation is simply a recursive call ensuring that all objects down the object hierarchy are constructed by the IoC Container using the same logic to find each object's dependencies (if any).   David shows us his own IoC container framework on one of his slides and it's only around 70 lines of code.  It almost fits on a single screen.  Of course, this is a very basic container and doesn't have all the real-world required features such as object lifetime management and scoping, but it does work and performs the basic functionality.  I haven't shown the code here as it's very similar to the other meta-programming code we've already looked at, but there's a number of examples of simple IoC containers out there on the internet, some written in only 15 lines of code!

After the demos, David talks about how we can actually use the reflection and meta-programming we've seen demonstrated in our own code as we're unlikely to re-write our MVC, Unit Testing or IoC frameworks.  Well, there's a number of ways in which such introspective code can be used.  One example is based upon some functionality for sending emails, a common enough requirement for many applications.  We look at some all too frequently found code that has to branch based upon the type of email that we're going to be sending:

public string SendEmails(string emailType)
	var emailMerger = new EmailMerger();
	if (emailType == "Nightly")
		var nightlyExtract = new NightlyEmail();
		var templatePath = "\\Templates\\NightlyTemplate.html";
		return emailMerger.Merge(templatePath, nightlyExtract);
	if (emailType == "Daily")
		var dailyExtract = new DailyEmail();
		var templatePath = "\\Templates\\DailyTemplate.html";
		return emailMerger.Merge(templatePath, dailyExtract);
	throw new NotImplementedException();

We can see we're branching conditionally based upon a string that represents the type of email we'll be processing, either a daily email or a nightly one.  However, by using reflective meta-programming, we can change the above code to something much more sophisticated:

public string SendEmails(string emailType)
	var strategies = new Email[] {new NightlyEmail(), new DailyEmail()};
	var selected = strategies.First(x => x.GetType().Name.StartsWith(emailType));
	var templatePath = "\\Templates\\" + selected.GetType().Name + ".html";
	return new EmailMerger().Merge(templatePath, selected);

IMG_20171014_112710Another way of using meta-programming within our own code is to perform automatic registrations for our DI/IoC Containers.  We often have hundreds or thousands of lines of manual registration, such as container.Register<IFoo, Foo>(); and we can simplify this by simply enumerating over all of the interfaces within our assemblies and looking for classes that implement that interface and possibly are called by the same name prefix and automatically registering the interface and type with the IoC container.  Of course, care must be taken here as such an approach may actually hide intent and is somewhat less explicit.  In this regard, David says that with the great power available to us via meta-programming comes great responsibility, so we should take care to only use it to "make the obvious thing work, not make the right thing totally un-obvious".

Finally, perhaps one of the best uses of meta-programming in this way is to help protect code quality.  We can do this by using meta-programming within our unit tests to enforce some attribute to our code that we care about.  One great example of this is to ensure that all classes within a given namespace have a specific suffix to their name.  Here's a very simple unit test that ensures that all classes in a Factories namespace have the word "Factory" at the end of the class name:

public void MakeSureFactoriesHaveTheRightNamingConventions()
	var types = AppDomain.CurrentDomain
		.SelectMany(a => a.GetTypes())
		.Where(x => x.Namespace == "MyApp.Factories");

	foreach (var type in types)

After David's session was over it was time for another quick coffee break.  As I had to change rooms this time, I decided to head back to the main foyer and grab a quick cup of coffee before immediately heading off to find the room for my next session.  This session was James Murphy's A Gentle Introduction To Elm.

IMG_20171014_120146James starts by introducing the Elm language.  Elm calls itself a "delightful language for reliable web apps".  It's a purely functional language that transpiles to JavaScript and is a domain-specific language designed for developing web applications.  Being a purely functional language allows Elm to make a very bold claim.  No run-time exceptions!

James asks "Why use Elm?".  Well, for one thing, it's not JavaScript!  It's also functional, giving it all of the benefits of other functional languages such as immutability and pure functions with no side effects.  Also, as it's a domain-specific language, it's quite small and is therefore relatively easy to pick up and learn.  As it boasts no run-time exceptions, this means that if your Elm code compiles, it'll run and run correctly.

James talks about the Elm architecture and the basic pattern of implementation, which is Model-Update-View.  The Model is the state of your application and it's data.  The Update is the mechanism by which the state is updated, and the View is how the state is represented as HTML.  It's this pattern that provides reliability and simplicity to Elm programs.  It's a popular, modern approach to front-end architecture, and the Redux JavaScript framework was directly inspired by the Elm architecture.  A number of companies are already using Elm in production, such as Pivotal, NoRedInk, Prezi and many others.

Here's a simple example Elm file showing the structure using the Model-Update-View pattern.  The pattern should be understandable even if you don't know the Elm syntax:

import Html exposing (Html, button, div, text)
import Html.Events exposing (onClick)

main =
  Html.beginnerProgram { model = 0, view = view, update = update }

type Msg = Increment | Decrement

update msg model =
  case msg of
    Increment ->
      model + 1

    Decrement ->
      model - 1

view model =
  div []
    [ button [ onClick Decrement ] [ text "-" ]
    , div [] [ text (toString model) ]
    , button [ onClick Increment ] [ text "+" ]

Note that the Elm code is generating the HTML that will be rendered by the browser.  This is very similar to the React framework and how it also performs the rendering for the actual page's markup.  This provides for a strongly-typed code representation of the HTML/web page, thus allowing far greater control and reasoning around the ultimate web page's markup.

You can get started with Elm by visiting the projects home page at elm-lang.org.  Elm can be installed either directly from the website, or via the Node Package Manager (NPM).  After installation, you'll have elm-repl - a REPL for Elm, elm-make which is the Elm compiler, elm-package - the Elm package manager and elm-reactor - the Elm development web server.  One interesting thing to note is that Elm has strong opinions about cleanliness and maintainability, so with that in mind, Elm enforces semantic versioning on all of it's packages!

James shows us some sample Elm statements in the Elm REPL. We see can use all the standard and expected language elements, numbers, strings, defining functions etc.  We can also use partial application, pipe-lining and lists/maps, which are common constructs within functional languages.  We then look at the code for a very simple "Hello World" web page, using the Model-Update-View pattern that Elm programs follow.  James is using Visual Studio Code to as his code editor here, and he informs us that there's mature tooling available to support Elm within Visual Studio Code.

We expand the "Hello World" page to allow user input via a textbox on the page, printing "Hello" and then the user's input.  Due to the continuous Model-Update-View loop, the resulting page is updated with every key press in the textbox, and this is controlled by the client-side JavaScript that has been transpiled from the Elm functions.  James shows this code running through the Elm Reactor development web server.  On very nice feature of Elm Reactor is that is contains built-in "time-travel" debugging, meaning that we can enumerate through each and every "event" that happens within our webpage. In this case, we can see the events that populate the "Hello <user>" text character-by-character.  Of course, it's possible to only update the Hello display text when the user has finished entering their text and presses the Enter key in the textbox, however, since this involves maintaining state, we have to perform some interesting work in our Elm code to achieve it.

James shows us how Elm can respond to events from the outside world.  He writes a simply function that will respond to system tick events to show an ever updating current time display on the web page.  James shows how we can work with remote data by defining specific types (unions) that represent the data we'll be consuming and these types are then added to the Elm model that forms the state/data for the web page.  One important thing to note here is that we need to be able to not only represent the data but also the absence of any data with specific types that represent the lack of data.  This is, of course, due to Elm being a purely functional language that does not support the concept of null.

IMG_20171014_121610The crux of Elm's processing is taking some input (in the form of a model and a message), performing the processing and responding with both a model and a message.  Each Elm file has an "init" section that deals with the input data.  The message that is included in that data can be a function, and could be something that would access a remote endpoint to gather data from a remote source.  This newly acquired data can then be processed in the "Update" section of the processing loop, ultimately for returning as part of the View's model/message output.  James demonstrates this by showing us a very simple API that he's written implementing a simply To-Do list.  The API endpoint exposes a JSON response containing a list of to-do items.  We then see how this API endpoint can be called from the Elm code by using a custom defined message that queries the API endpoint and pulls in the various to-do items, processes them and writes that data into the Elm output model which is ultimately nicely rendered on the web page.

Elm contains a number of rich packages out-of-the-box, such as a HTTP module.  This allows us to perform HTTP requests and responses using most of the available HTTP verbs with ease:

import Json.Decode (list, string)

items : Task Error (List String)
items =
    get (list string) "http://example.com/to-do-items.json"


corsPost : Request
corsPost =
    { verb = "POST"
    , headers =
        [ ("Origin", "http://elm-lang.org")
        , ("Access-Control-Request-Method", "POST")
        , ("Access-Control-Request-Headers", "X-Custom-Header")
    , url = "http://example.com/hats"
    , body = empty

It's important to note, however, that not all HTTP verbs are available out-of-the-box and some verbs, such as PATCH, will need to be manually implemented.

James wraps up his session by talking about the further eco-system around the Elm language.  He mentions that Elm has it's own testing framework, ElmTest, and that you can very easily achieve a very high amount of code coverage when testing in Elm due to it being a purely functional language.  Also, adoption of Elm doesn't have to be an all-or-nothing proposition.  Since Elm transpiles to JavaScript, it can play very well with existing JavaScript applications.  This means that Elm can be adopted in a piece meal fashion, with only small sections of a larger JavaScript application being replaced by their Elm equivalent, perhaps to ensure high code coverage or to benefit from improved robustness and reduced possibility of errors.

Finally, James talks about how to deploy Elm application when using Elm in a real-world production application.  Most often, Elm deployment is performed using WebPack, a JavaScript module bundler.  This often takes the form of shipping a single small HTML file containing the necessary script inclusions for it to bootstrap the main application.

IMG_20171014_131053After James' session was over, it was time for lunch.  All the attendees made there way back to the main foyer area where a delicious lunch of a selection of sandwiches, fruit, crisps and chocolate was available to us.  As is customary at the various DDD events, there were to be a number of grok talks taking place over the lunch period.  As I'd missed the grok talks at the last few DDD events I'd attended, I decided that I'd make sure I aught a few of the talks this time around.

I missed the first few talks as the queue for lunch was quite long and it took a little while to get all attendees served, however, after consuming my lunch in the sunny outdoors, I headed back inside to the large lecture theatre where the grok talks were being held.  I walked in just to catch the last minute of Phil Pursglove's talk on Azure's CosmosDB, which is Microsoft's globally distributed, multi-model database.  Unfortunately, I didn't catch much more than that, so you'll have to follow the link to find out more. (Update:  Phil has kindly provided a link to a video recording of his talk!)

The next grok talk was Robin Minto's OWASP ZAP FTW talk.  Robin introduces us to OWASP, which is the Open Web Application Security Project and exists to help create a safer, more secure web.  Robin then mentions ZAP, which is a security testing tool produced by OWASP.  ZAP is the Zed Attack Proxy and is a vulnerability scanner and intercepting proxy to help detect vulnerabilities in your web application.  Robin shows us a demo application he's built containing deliberate flaws, Bob's Discount Diamonds.  This is running on his local machine.  He then shows us a demo of the OWASP ZAP tool and how it can intercept all of the requests and responses made between the web browser and the web server, analysing those responses for vulnerabilities and weaknesses.  Finally, Robin shows us that the OWASP ZAP software contains a handy "fuzzer" capability which allows it to replay requests using lists of known data or random data - i.e. can replay sending login requests with different usernames/passwords etc.

The next grok talk was an introduction to the GDPR by John Price.  John introduces the GDPR, which is the new EU wide General Data Protection Regulations and effectively replaced the older Data Protection Act in the UK.  GDPR, in a nutshell, means that users of data (mostly companies who collect a person's data) need to ask permission from the data owner (the person to whom that data belongs) for the data and for what purpose they'll use that data.  Data users have to be able to prove that they have a right to use the data that they've collected.  John tells us that adherence to the GDPR in the UK is not affected by Brexit as it's already enshrined in UK law and has been since April 2016, although it's not really been enforced  up to this point.  It will start to be strictly enforced from May 2018 onwards.  We're told that, unlike the previous Data Protection Act, violations of the regulations carry very heavy penalties, usually starting at 20 million Euros or 4% of a company's turnover.  There will be some exceptions to the regulations, such as police and military but also exception for private companies too, such as a mobile phone network provider giving up a person's data due to "immediate threat to life".  Some consent can be implied, so for example, entering your car's registration number into a web site for the purposes of getting an insurance quote is implied permission to use the registration number that you've provided, but the restriction is that the data can only be used for the specific purpose for which it was supplied.  GDPR will force companies to declare if data is sent to third parties.  If this happens, the company initially taking the data and each and every third-party that receives that data have to inform the owner of the data that they are in possession of the data.  GDPR is regulated by the Information Commissioners Office in the UK.  Finally, John says that the GDPR may make certain businesses redundant.  He gives an example industry of credit reference agencies.  Their whole business model is built on non-consentual usage of data, so it will be interesting to see how GDPR affects industries like these.

After John's talk, there was a final grok talk however, I needed a quick restroom break before the main sessions of the afternoon, so headed off for my restroom break before making my way back to the room for the first of the afternoon's sessions.  This was Matt Ellis's How To Parse A File.

IMG_20171014_142833Matt starts his session by stating that his talk is all about parsing files, but he immediately says, "But, Don't do it!"  He tells us that it's a solved problem and we really shouldn't be writing code to parsing files by hand for ourselves and should just use one of the many excellent libraries out there instead.  Matt does discuss why you decide you really needed to parse files for yourself.  Perhaps you need better speed and efficiency or maybe it's to reduce dependencies or to parse highly specific custom formats.  It could even be parsing for things that aren't even files such as HTTP headers, standard output etc.  From here, Matt mentions that he works for JetBrains and that the introduction of simply parsing a file is a good segue into talking about some of the features that can be found inside many of JetBrains' products.

Matt starts by looking at the architecture of many of JetBrains' IDE's and developer tools such as ReSharper.  They're build with a similar architecture and they all rely on a layer that they call the PSI layer.  The PSI layer is responsible for parsing, lexing and understanding the user code that the tool is working on.  Matt says that he's going to use the Unity framework to show some examples throughout his session and that he's going to attempt to build up a syntax tree for his Unity code.  We first look at a hand-rolled parser, this one is attempting to understand the code by observing each character at a time.  It's a very laborious approach and prone to error, so this is an approach to parsing that we shouldn't use.  Matt tells us that the best approach, which has been "solved" many time in the past is ti employ the services of a lexer.  This is a processor that turns the raw code into meaningful tokens based upon the words and vocabulary of the underlying code or language and gives structure to those tokens.  It's from the output of the lexer that we can more easily and robustly perform the parsing.  Lexers are another solved problem, and many lexers already exist for popular programming languages such as lex, CsLex, FsLex, flex, JFlex and many more.

Lexers generate source code, but it's not human readable code.  It's similar to how .NET language code (C# or VB.NET) is first compiled to Intermediate Language prior to being JIT'ed at runtime.  The code output from the lexer is read by the parser and from there the parser can try to understand the grammar and structure of the underlying code via syntactical analysis.  This often involved the use of Regular Expressions in order to match specific tokens or sets of tokens.  This works particularly well as Regular Expressions can be translated into a state machine and from there, translated into a transition table.  Parsers understand the underlying code that they're designed to work on, so for example, a parser for C# would know that in a class declaration, there would be the class name which would be preceded by a token indicating the scope of the class (public, private etc).  Parsing is not a completely solved problem. It's more subjective, so although solutions exist, they're more disparate and specific to the code or language that they're used for and therefore, they're not a generic solution.

Matt tells us how parsing can be done either top-down or bottom-up. Top down parsing starts at highest level construct of the language, for example at a namespace or class level in C#, and it then works it's way down to the lower level constructs from there - through methods and the code and locally scoped variables in those methods.  Bottom up parsing works the opposite way around, starting with the lower level constructs of the language and working back up to the class or namespace.  Bottom up parsers can be beneficial over top-down ones as they have the ability to utilise shift-reduce algorithms to simplify code as it's being parsed.  Parsers can even be "parser combinators". These are parsers built from other, simpler, parsers where the input the next parser in the chain is the output from the previous parser in the chain, more formally known as recursive-descendant parsing.  .NET's LINQ acts in a similar way to this.  Matt tells us about an F# parser combinator called FParsec and a C# parser is sort of like this. FParsec is a parser combinator for F# along with a C# parser combinator called Sprache, itself relying heavily on LINQ:

Parser<string> identifier =
    from leading in Parse.WhiteSpace.Many()
    from first in Parse.Letter.Once()
    from rest in Parse.LetterOrDigit.Many()
    from trailing in Parse.WhiteSpace.Many()
    select new string(first.Concat(rest).ToArray());

var id = identifier.Parse(" abc123  ");

Assert.AreEqual("abc123", id);

Matt continues by asking us to consider how parsers will deal with whitespace in a language.  This is not always as easy as it sounds as some languages, such as F# or Python, use whitespace to give semantic meaning to their code, whilst other languages such as C# use whitespace purely for aesthetic purposes.  In dealing with whitespace, we often make use of a filtering lexer.  This is a simple lexer that specific detects and removes whitespace prior to parsing.  The difficulty then is that, for languages where  whitespace is significant, we need to replace the removed whitespace after parsing.  This, again, can be tricky as the parsing may alter the actual code (i.e. in the case of a refactoring) so we must again be able to understand the grammar of the language in order to re-insert whitespace into the correct places.  This is often accomplished by building something known as a Concrete Parse Tree as opposed to the more normal Abstract Syntax Tree.  Concrete Parse Tree's work in a similar way to a C# Expression Tree, breaking down code into a hierarchical graph of individual code elements.

IMG_20171014_144725Matt tells us about other uses for Lexers such as the ability to determine specific declarations in the language.  For example, in F# typing 2. would represent a floating point number, where as typing [2..0] would represent a range.  When the user is only halfway through typing, how can we know if they require a floating point number or a range?  Also such things as comments within comments, for example: /* This /* is */ valid */  This is something that lexers can be good at, at such matching is difficult to impossible with regular expressions.

The programs that use lexers and parsers can often have very different requirements, too.  Compilers using them will generally want to compile the code, and so they'll work on the basis that the program code that they're lexing/parsing is assumed correct, whilst IDE's will take the exact opposite approach.  After all, most of the time whilst we're typing, our code is in an invalid state.  For those programs that assume the code is in an invalid state most of the time, they often use techniques such as error detection and recovery.  This is, for example, to prevent your entire C# class from being highlighted as invalid within the IDE just because the closing brace character is missing from the class declaration.  They perform error detection on the missing closing brace, but halt highlighting of the error at the first "valid" block of code immediately after the matching opening brace.  This is how the Visual Studio IDE is able to only highlight the missing closing brace as invalid and not the entire file full of otherwise valid code.  In order for this to be performant, lexers in such programs will make heavy use of caching to prevent having to continually lex the entire file with every keystroke.

Finally, Matt talks about how JetBrains often need to also deal with "composable languages".  These are things like ASP.NET MVC's Razor files, which are predominantly comprised of HTML mark-up, but which can also contain "islands" of C# code.  For this, we take a similar approach to dealing with whitespace in that the file is lexed for both languages, HTML and C# and the HTML is temporarily removed whilst the C# code is parsed and possibly altered.  The lexed tokens from both the C# and the preserved HTML are then re-combined after the parsing to re-create the file.

After Matt's session, there was one final break before the last session of the day.  Since there was, unfortunately, no coffee served at this final break, I made my way directly to the room for my next and final session, Joe Stead's .NET Core In The Real World.

IMG_20171014_154513Joe starts his talk by announcing that there's no code or demos in his talk, and that his talk will really just be about his own personal experience of attempting to migrate a legacy application to .NET Core.  He says that he contemplated doing a simple "Hello World" style demo for getting started with .NET Core, but that it would give a false sense of .NET Core being simple.  In the real-world, and when migrating an older application, it's a bit more complicated than that.

Joe mentions the .NET Standard and reminds us that it's a different thing than .NET Core.  .NET Core does adhere to the .NET Standard and Joe tells us that .NET Standard is really just akin to Portable Class Libraries Version 2.0.

Joe introduces the project that he currently works on at his place of employment.  It's a system that started life in 2002 and was originally built with a combination of Windows Forms applications and ASP.NET Web Forms web pages, sprinkled with Microsoft AJAX JavaScript.   The system was in need of being upgraded in terms of the technologies used, and so in 2012, they migrated to KnockoutJS for the front-end websites, and in 2013, to further aid with the transition to KnockoutJS, they adopted the NancyFX framework to handle the web requests.  Improvements in the system continued and by 2014 they had started to support the Mono Framework and had moved from Microsoft SQL Server to a PostgreSQL database.  This last lot of technology adoptions was to support their growing demand for a Linux version of their application from their user base.  The adoptions didn't come without issues, however, and by late 2014, they started to experience serious segfaults in their application.  After some diagnosis after which they never did fully get to the bottom of the root cause of the segfaults, they decided to adopt Docker in 2015 as a means of mitigating the segfault problem.  If one container started to display problems associated with segfaults, they could kill the container instance and create a new one.  By this point, they were in 2015 and decided that they'd start to now look into .NET Core.  It was only in Beta at this time, but were looking for a better platform that Mono that might provide some much needed stability and consistency across operating systems.  And since they were on such a roll with changing their technology stacks, they decided to move to Angular2 on the front-end, replacing KnockoutJS in 2016 as well!

By 2017, they'd adopted .NET Core v1.1 along with RabbitMQ and Kubernetes.  Joe states that the reason for .NET Core adoption was to move away from Mono.  By this point, they were not only targeting Mono, but a custom build of Mono that they'd had to fork in order to try to fix their segfault issues.  They needed much more flexible deployments such as the ability to package and deploy multiple versions of their application using multiple different versions of the underlying .NET platform on the same machine.  This was problematic in Mono, as it can be in the "full" .NET Framework, but one of the benefits of .NET Core is the ability to package the run-time with your application, allowing true side-by-side versions of the run-time to exist for different applications on the same machine.

Joe talks about some of the issues encountered when adopting and migrating to .NET Core.  The first issue was missing API's.  .NET Core 1.0 and 1.1 were built against .NET Standard 1.x and so many API's and namespace were completely missing.  Joe also found that many NuGet packages that his solution was dependent upon were not yet ported across to .NET Core.  Joe recalls that testing of the .NET Core version of the solution was a particular challenge as few other people had adopted the platform and the general response from Microsoft themselves was that "it's coming in version 2.0!".  What really helped save the day for Joe and his team was that .NET Core itself and many of the NuGet packages were open source.  This allowed them to fork many of the projects that the NuGet packages were derived from and help with transitioning them to support .NET Core.  Joe's company even employed a third party to work full time on helped to port NancyFX to .NET Core. 

Joe now talks about the tooling around .NET Core in the early days of the project.  We examine how Microsoft introduced a whole new project file structure, moving away from XML representation in the .csproj files, and moving to a JSON representation with project.json.  Joe explains how they had to move their build script and build tooling to the FAKE build tool as a result of the introduction of project.json.  There were also legal issues around using the .NET Core debugger assemblies in tools other than Microsoft's own IDE's, something that the JetBrain's Rider IDE struggled with.  We then look at tooling in the modern world of .NET Core and project.json has gone away, and reverted back to the .csproj files although they're much more simplified and improved.  This allows the use of MSBuild again, however, FAKE itself now has native support for .NET Core.  The dotnetCLI tool has improved greatly and the legal issues around the use of the .NET Core debugging assemblies has been resolved, allowing third-party IDE's such as JetBrain's Rider to use them again. 

IMG_20171014_161612Joe also mentions how .NET Core now, with the introduction of version 2.0, is much better than the Mono Framework when it comes to targeting multiple run-times.  He also mentions issues that plagued their use of libcurl on the Mac platform when using .NET Core 1.x, but that these have now been resolved in .NET Core 2.0 as .NET Core 2.0 now uses the native macOS implementation rather than trying to abstract that and use it's own implementation.

Joe moves on to discuss something that's not really specific to .NET Core, but is a concern when developing code to be run on multiple platforms.  He shows us the following two lines of code:

TimeZoneInfo.FindSystemTimeZoneById("Eastern Standard Time");

He asks which is the "correct" one to use.   Well, it turns out that they're both correct.  And possibly incorrect!   The top line works on Windows, and only on Windows, whilst the bottom line works on Linux, and not on Windows.  It's therefore incredibly important to understand such differences when targeting multiple platforms with your application.  Joe also says how, as a result of discrepancies such as the timezone issue, the tooling can often lie.  He recalls a debugging session where one debugging window would show the value of a variable with one particular date time value, and another debugging window - in the exact same debug session - would interpret and display the same variable with an entirely different date time value.  Luckily, most of these issues are now largely resolved with the stability that's come from recent versions of .NET Core and the tooling around it.

In wrapping up, Joe says that, despite the issues they encountered, moving to .NET Core was the right thing for him and his company.  He does say, though, that for other organisations, such a migration may not be the right decision. Each company, and each application, needs to be evaluated for migration to .NET Core on it's own merits.  For Joe's company, the move to .NET Core allowed them to focus attention elsewhere after migration.  They've since been able to adopt Kubernetes. They've been able to improve and refactor code to implement better testing and many more long overdue improvements.  In the last month, they've migrated again from .NET Core 1.1 to .NET Core 2.0 which was a relatively easy task after the initial .NET Core migration.  This one only involved the upgrading of a few NuGet packages and that was it.  The move to .NET Core 2.0 also allowed them to re-instate lots of code and functionality that had been temporarily removed thanks to the new, vastly increased API surface area of .NET Core 2.0 (really, .NET Standard 2.0).

IMG_20171014_165613After Joe's session, it was time for all the attendees to gather in the main foyer area of the university building for the final wrap-up and prize draws.  After thanking to sponsors, the venue, and the organisers and volunteers, without whom of course, events such as DDD simply wouldn't be able to take place, we moved onto the prize draw.  Unfortunately, I wasn't a winner, however, the day had been brilliant.

IMG_20171014_132318Another wonderful DDD event had been and gone but a great day was had by all.  We were told that the next DDD Event was to be a DDD Dublin, held sometime around March 2018.  So there's always that to look forward to.

Analyze Solution For Code Clones Missing In Visual Studio 2017

I was watching a video on Microsoft’s Channel 9 website regarding design patterns and during the video the presenter showed a feature of Visual Studio that analysed the currently loaded solution for “code clones” – sections of code that appear to be clones or duplicates in different methods.

This seemed like a very useful feature and something that I thought I could benefit from on a number of codebases that I’m currently working with.  So off I went to explore this handy feature for myself.

First thing to note this that, although this feature has been part of Visual Studio since Visual Studio 2012, it’s only available in the Enterprise SKU of the product (or the Premium or Ultimate versions for VS 2012/2013).

That was ok, as I’m using Visual Studio 2017 Enterprise, however, upon expanding the Analyze menu option, I was greeted with a distinct absence of the Analyze Solution For Code Clones option:


Hmm..  I was curious why the option was missing so went off doing my best googling to try to find out why.   And it turns out that there’s very little information out there on the internet regarding this feature, and even less information regarding troubleshooting why the option might not be appearing on the Analyze tools menu.

After some amount of aimlessly clicking around on the spurious search results, it suddenly dawned on me.   Perhaps it’s an “optional extra” component that you need to install?   Turns out I was right.

When I’d originally installed Visual Studio 2017, I’d selected one of the pre-configured “Workloads” to install.  In my case it was the “ASP.NET And Web Development” workload as that’s the sort of development work I do.  Turns out that, whilst this workload installs almost everything you might want or need for ASP.NET And Web Development, there’s a few crucial things missing, one of which is the “Architecture And Analysis Tools” component – which in turn includes the “Code Clone” tool:


In order to get the Code Clone tool installed, you can either select just that tool from the “Individual components” tab of the Visual Studio installer, or you can select the “Architecture and analysis tools” optional component (as seen on the right hand side in the above picture) which will select the Code Clone, Code Map and Live Dependency Validation tools.

Once this is done, restarting Visual Studio will give you the missing “Analyze Solution for Code Clones” menu option:


Easy when you know how, eh?