DDD Southwest 2017 In Review
This past Saturday 6th May 2017, the seventh iteration of the DDD South West conference was held in Bristol, at the Redcliffe Sixth Form Centre.
This was my second DDD South West conference that I’d attended, having attended the prior DDD South West conference in 2015 (they’d had a year off in 2016), so returning the 2017, the DDD South West team had put on another fine DDD conference.
Since the conference is in Bristol, I’d stayed over the evening before in Gloucester in order to break up the long journey down to the south west. After an early breakfast on the Saturday morning, I made the relatively short drive from Gloucester to Bristol in order to attend the conference.
Arriving at the Redcliffe centre, we were signed in and made our way to the main hall where we were treated to teas, coffees and danish pastries. I’d remembered these from the previous DDDSW conference, and they were excellent and very much enjoyed by the arriving conference attendees.
After a short while in which I had another cup of coffee and the remaining attendees arrived at the conference, we had the brief introduction from the organisers, running through the list of sponsors who help to fund the event and some other house-keeping information. Once this was done it was time for us to head off to the relevant room to attend our first session. DDDSW 7 had 4 separate tracks, however, for 2 of the event slots they also had a 5th track which was the Sponsor track – this is where one of the technical people from their primary sponsor gives a talk on some of the tech that they’re using and these are often a great way to gain insight into how other people and companies do things.
I heading off down the narrow corridor to my first session. This was Ian Cooper’s 12-Factor Apps In .NET.
Ian starts off my introducing the 12 factors, these are 12 common best practices for building web based applications, especially in the modern cloud-first world. There’s a website for the 12 factors that details them. These 12 factors and the website itself were first assembled by the team behind the Heroku cloud application platform as the set of best practices that most of the most successful applications running on their platform had.
Ian states how the 12 factors cover 3 broad categories: Design, Build & Release and Management. We first look at getting setup with a new project when you’re a new developer on a team. Setup automation should be declarative. It should ideally take no more than 10 minutes for a new developer to go from a clean development machine to a working build of the project. This is frequently not the case for most teams, but we should work to reduce the friction and the time it takes to get developers up and running with the codebase. Ian also state that a single application, although it may consist of many different individual projects and “moving parts” should remain together in a single source code repository. Although you’re keeping multiple distinct parts of the application together, you can still easily build and deploy the individual parts independently based upon individual .csproj files.
Next, we talk about application servers. We should try to avoid application servers if possible. What is an application server? Well, it can be many things, but the most obvious one in the world of .NET web applications is Internet Information Server itself. This is an application server and we often require this for our code to run within it. IIS has a lot of configuration of it’s own and so this means that our code is then tied to and dependent upon that specifically configured application server. An alternative to this is have our code self-host, which is entirely possible with ASP.NET Web API and ASP.NET Core. By self-hosting and running our endpoint of a specific port to differentiate it from other services that may also potentially run on the same server, we ensure our code is far more portable, and platform agnostic.
Ian then talks about backing services. These are the concerns such as databases, file systems etc. that our application will inevitably need to talk to. But we need to ensure that we treat them as the potentially ephemeral systems that they are, and therefore accept that such a system may not even have a fixed location. Using a service discovery service, such as Consul.io, is a good way to remove our application’s dependence on a a required service being in a specific place.
Ian mentions the port and adapters architecture (aka hexagonal architecture) for organising our codebase itself. He likes this architecture as it’s not only a very clean way to separate concerns, and keep the domain at the heart of the application model, it also works well in the context of a “12-factor” compliant application as the terminology (specifically around the use of “ports”) is similar and consistent. We then look at performance of our overall application. Client requests to our front-end website should be responded to in around 200-300 milliseconds. If, as part of that request, there’s some long-running process that needs to be performed, we should offload that processing to some background task or external service, which can update the web site “out-of-band” when the processing is complete, allowing the front-end website to render the initial page very quickly.
This leads on to talking about ensuring our services can start up very quickly, ideally no more than a second or two, and should shutdown gracefully also. If we have slow startup, it’s usually because of the need to built complex state, so like the web front-end itself, we should offload that state building to a background task. If our service absolutely needs that state before it can process incoming requests, we can buffer or queue early request that the service receives immediately after startup until our background initialization is complete. As an aside, Ian mentions supervisorD as a handy tool to keep our services and processes alive. One great thing about keeping our startup fast and lean, and our shutdown graceful, we essentially have elastic scaling with that service!
Ian now starts to show us some demo code that demonstrates many of the 12 best practices within the “12-factors”. He uses the ToDoBackend website as a repository of sample code that frequently follows the 12-factor best practices. Ian specifically points out the C# ASP.NET Core implementation of ToDoBackend as it was contributed by one of his colleagues. The entire ToDoBackend website is a great place where many backend frameworks from many different languages and platforms can be compared and contrasted.
Ian talks about the ToDoBackend ASP.NET Core implementation and how it is built to use his own libraries Brighter and Darker. These are open-source libraries implementing the command dispatcher/processor pattern which allow building applications compliant with the CQRS principle, effectively allowing a more decomposed and decoupled application. MediatR is a similar library that can be used for the same purposes. The Brighter library wraps the Polly library which provides essential functionality within a highly decomposed application around improved resilience and transient fault handling, so such things as retries, the circuit breaker pattern, timeouts and other patterns are implemented allowing you to ensure your application, whilst decomposed, remains robust to transient faults and errors – such as services becoming unavailable.
Looking back at the source code, we should ensure that we explicitly isolate and declare our application’s dependencies. This means not relying on pre-installed frameworks or other library code that already exists on a given machine. For .NET applications, this primarily means depending on only locally installed NuGet packages and specifically avoiding referencing anything installed in the GAC. Other dependencies, such as application configuration, and especially configuration that differs from environment to environment – i.e. database connection strings for development, QA and production databases, should be kept within the environment itself rather than existing within the source code. We often do this with web.config transformations although it’s better if our web.config merely references external environment variables with those variables being defined and maintained elsewhere. Ian mentions HashiCorp’s Vault project and the Spring Boot projects as ways in which you can maintain environment variables and other application “secrets”. An added bonus of using such a setup is that your application secrets are never added to your source code meaning that, if your code is open source and available on something like GitHub, you won’t be exposing sensitive information to the world!
Finally, we turn to application logging. Ian states how we should treat all of our application’s logs as event streams. This means we can view our logs as a stream of aggregated, time-ordered events from the application and should flow continuously so long as the application is running. Logs from different environments should be kept as similar as possible. If we do both of these things we can store our logs in a system such as Splunk, AWS CloudWatch Logs, LogEntries or some other log storage engine/database. From here, we can easily manipulate and visualize our application’s behaviour from this log data using something like the ELK stack.
After a few quick Q&A’s, Ian’s session was over and it was time to head back to the main hall where refreshments awaited us. After a nice break with some more coffee and a few nice biscuits, it was quickly time to head back through the corridors to the rooms where the sessions were held for the next session. For this next session, I decided to attend
the sponsors track, which was Ed Courtenay’s “Growing Configurable Software With Expression
Ed’s session was a code heavy session which used a single simple application based around filtering a set of geographic data against a number of filters. The session looked at how configurability of the filtering as well as performance could be improved with each successive iteration refactoring the application to using an ever more “expressive” implementation. Ed starts by asking what we define as configurable software. We all agree that software that can change it’s behaviour at runtime is a good definition, and Ed says that we can also think of configurable software as a way to build software from individual building blocks too. We’re then introduced to the data that we’ll be using within the application, which is the geographic data that’s freely available from geonames.org.
From here, we dive straight into some code. Ed shows us the IFilter interface that exposes a single method which provide a filter function:
using System;
namespace ExpressionDemo.Common
{
public interface IFilter
{
Func<IGeoDataLocation, bool> GetFilterFunction();
}
}
The implementation of the IFilter interface is fairly simple and combines a number of actual filter functions that filter the geographic data by various properties, country, classification, minimum population etc.
public Func<IGeoDataLocation, bool> GetFilterFunction()
{
return location => FilterByCountryCode(location) && FilterByClassification(location)
&& FilterByPopulation(location) && FilterByModificationDate(location);
}
Each of the actual filtering functions is initially implemented as a simple loop, iterating over the set of allowed data (i.e. the allowed country codes) and testing the currently processed geographic data row against the allowed values to determine if the data should be kept (returned true from the filter function) or discarded (returning false):
private bool FilterByCountryCode(IGeoDataLocation location)
{
if (!_configuration.CountryCodes.Any())
return true;
foreach (string countryCode in _configuration.CountryCodes) {
if (location.CountryCode.Equals(countryCode, StringComparison.InvariantCultureIgnoreCase))
return true;
}
return false;
}
Ed shows us his application with all of his filters implemented in such a way and we see the application running over a reduced geographic data set of approximately 10000 records. We process all of the data in around 280ms, however Ed tells us that it's really a very naive implementation and with only a small improvement in the filter implementations, we can improve this. From here, we look at the same filters, this time implemented as a Func<T, bool>:
private Func<IGeoDataLocation, bool> CountryCodesExpression()
{
IEnumerable<string> countryCodes = _configuration.CountryCodes;
string[] enumerable = countryCodes as string[] ?? countryCodes.ToArray();
if (enumerable.Any())
return location => enumerable
.Any(s => location.CountryCode.Equals(s, StringComparison.InvariantCultureIgnoreCase));
return location => true;
}
We're doing the exact same logic, but instead of iterating over the allow country codes list, we're being more declarative and simply returning a Func<> which performs the election/filter for us. All of the other filters are re-implemented this way, and Ed re-runs his application. This time, we process the exact same data in around 250ms. We're starting to see an improvement. But we can do more. Instead of returning Func<>'s we can go even further and return an Expression.
We pause looking at the code to discuss Expressions for a moment. These are really "code-as-data". This means that we can decompose our algorithm that performs some specific type of filtering and can then express that algorithm as a data structure or tree. This is a really powerful way or expressing our algorithm and allows our application simply functionally "apply" our input data to the expression rather than having to iterate or loop over lists as we had in the very first implementation. What this means for our algorithms is increased flexibility in the construction of the algorithm, but also increase performance. Ed shows us the implementation filter using an Expression:
private Expression<Func<IGeoDataLocation, bool>> CountryCodesExpression()
{
var countryCodes = _configuration.CountryCodes;
var enumerable = countryCodes as string[] ?? countryCodes.ToArray();
if (enumerable.Any())
return enumerable.Select(CountryCodeExpression).OrElse();
return location => true;
}
private static Expression<Func<IGeoDataLocation, bool>> CountryCodeExpression(string code)
{
return location => location.CountryCode.Equals(code, StringComparison.InvariantCultureIgnoreCase);
}
Here, we've simply taken the Func<> implementation and effectively "wrapped" the Func<> in an Expression. So long as our Func<> is simply a single expression statement, rather than a multi-statement function wrapped in curly brackets, we can trivially turn the Func<> into an Expression:
public class Foo
{
public string Name
{
get { return this.GetType().Name; }
}
}
// Func<> implementation
Func<Foo,string> foofunc = foo => foo.Name;
Console.WriteLine(foofunc(new Foo()));
// Same Func<> as an Expression
Expression<Func<Foo,string>> fooexpr = foo => foo.Name;
var func = fooexpr.Compile();
Console.WriteLine(func(new Foo()));
Once again, Ed refactors all of his filters to use Expression functions and we again run the application. And once again, we see an improvement in performance, this time processing the data in around 240ms. But, of course, we can do even better!
The next iteration of the application has us again using Expressions, however, instead of merely wrapping the previously defined Func<> in an Expression, this time we're going to write the Expressions from scratch. At this point, the code does admittedly become less readable as a result of this, however, as an academic exercise in how to construct our filtering using ever more lower-level algorithm implementations, it's very interesting. Here's the country filter as a hand-rolled Expression:
private Expression CountryCodeExpression(ParameterExpression location, string code)
{
return StringEqualsExpression(Expression.Property(location, "CountryCode"), Expression.Constant(code));
}
private Expression StringEqualsExpression(Expression expr1, Expression expr2)
{
return Expression.Call(typeof (string).GetMethod("Equals",
new[] { typeof (string), typeof (string), typeof (StringComparison) }), expr1, expr2,
Expression.Constant(StringComparison.InvariantCultureIgnoreCase));
}
Here, we're calling in the Expression API and manually building up the expression tree with our algorithm broken down into operators and operands. It's far less readable code, and it's highly unlikely you'd actually implement your algorithm in this way, however if you're optimizing for performance, you just might do this as once again we run the application and observe the results. It is indeed slightly faster than the previous Expression function implementation, taking around 235ms. Not a huge improvement over the previously implementation, but an improvement nonetheless.
Finally, Ed shows us the final implementation of the filters. This is the Filter Implementation Bridge. The code for this is quite complex and hairy so I've not reproduced it here. It is, however, available in Ed's Github repository which contains the entire code base for the session.
The Filter Implementation Bridge involves building up our filters as Expressions once again, but this time we go one step further and write the resulting code out to a pre-compiled assembly. This is a separate DLL file, written to disk, which contains a library of all of our filter implementations. Our application code is then refactored to load the filter functions from the external DLL rather than expecting to find them within the same project. Because the assembly is pre-compiled and JIT'ed, when we invoke the assembly's functions, we should see another performance improvement. Sure enough, Ed runs the application after making the necessary changes to implement this and we do indeed see an improvement in performance. This time processing the data set in around 210ms.
Ed says that although we've looked at performance and the improvements in performance with each successive refactor, the same refactorings have also introduced flexibility in how our filtering is implemented and composed. By using Expressions, we can easily change the implementation at run-time. If we have these expressions exported into external libraries/DLL's, we could fairly trivially decide to load a different DLL containing a different function implementation. Ed uses the example of needing to calculate VAT and the differing ways in which that calculation might need to be implemented depending upon country. By adhering to a common interface and implementing the algorithm as a generic Expression, we can gain both flexibility and performance in our application code.
After Ed’s session it time for another refreshment break. We heading back through the corridors to the main hall for more teas, coffees and biscuits. Again, with only enough time for a quick cup of coffee, it was time again to trek back to the session rooms for the 3rd and final session before lunch. This one was Sandeep Singh’s “JavaScript Services: Building Single-Page Applications With ASP.NET Core”.
Sandeep starts his talk with an overview of where Single Page Applications (SPA’s) are currently. There’s been an explosion of frameworks in the last 5+ years that SPA’s have been around, so there’s an overwhelming choice of JavaScript libraries to choose from for both the main framework (i.e. Angular, Aurelia, React, VueJS etc.) and for supporting JavaScript libraries and build tools. Due to this, it can be difficult to get an initial project setup and having a good cohesion between the back-end server side code and the front-end client-side code can be challenging. JavaScriptServices is a part of the ASP.NET Core project and aims to simplify and harmonize development for ASP.NET Core developers building SPA’s. The project was started by Steve Sanderson, who was the initial creator of the KnockoutJS framework.
Sandeep talks about some of the difficulties with current SPA applications. There’s often the slow initial site load whilst a lot of JavaScript is sent to the client browser for processing and eventual display of content, and due to the nature of the application being a single page and using hash-based routing for each page’s URL, and each page being loaded because of running some JavaScript, SEO (Search Engine Optimization) can be difficult. JavaScriptServices attempts to improve this by combining a set of SPA templates along with some SPA Services, which contains WebPack middleware, server-side pre-rendering of client-side code as well as routing helpers and the ability to “hot swap” modules on the fly.
WebPack is the current major module bundler for JavaScript, allowing many disparate client side asset files, such as JavaScript, images, CSS/LeSS,Sass etc. to be re-compiled/transpiled and combined in a bundle for efficient delivery to the client. The WebPack middleware also contains tooling similar to dotnet watch, which is effectively a file system watcher and can rebuild, reload and re-render your web site at development time in response to real-time file changes. The ability to “hot swap” entire sets of assets, which is effectively a whole part of your application, is a very handy development-time feature. Sandeep quickly shows us a demo to explain the concept better. We have a simple SPA page with a button, that when clicked will increment a counter by 1 and display that incrementing count on the page. If we edit our Angular code which is called in response to the button click, we can change the “increment by 1” code to say “increment by 10” instead. Normally, you’d have to reload your entire Angular application before this code change was available to the client browser and the built-in debugging tools there. Now, however, using JavaScript services hot swappable functionality, we can edit the Angular code and see it in action immediately, even continuing the incrementing counter without it resetting back to it’s initial value of 0!
Sandeep continues by discussing another part of JavaScriptServices which is “Universal JavaScript”, also known as “Isomorphic JavaScript”. This is JavaScript which is shared between the client and the server. How does JavaScriptServices deal with JavaScript on the server-side? It uses a .NET library called Microsoft.AspNetCore.NodeServices which is a managed code wrapper around NodeJS. Using this functionality, the server can effectively run the JavaScript that is part of the client-side code, and send pre-rendered pages to the client. This is a major factor in being able to improve the initial slow load time of traditional SPA applications. Another amazing benefit of this functionality is the ability to run an SPA application in the browser even with JavaScript disabled! Another part of the JavaScriptServices technology that enables this functionality is RoutingServices, which provides handling for static files, and a “fallback” route for when explicit routes aren’t declared. This means that a request to something like http://myapp.com/orders/ would be rendered on the client-side via the client JavaScript framework code (i.e. Angular) after an AJAX request to retrieve the mark-up/data from the server, however, if client-side routing is unavailable to process such a route (in the case that JavaScript is disabled in the browser, for example) then the request is sent to the server so that the server may render the same page (perhaps requiring client-side JavaScript which is now processed on the server!) on the server before sending the result back to the client via the means of a standard HTTP(s) request/response cycle.
I must admit, I didn’t quite believe this, however, Sandeep quickly setup some more demo code and showed us the application working fine with a JavaScript enabled browser, whereby each page would indeed be rendered by the JavaScript of the client-side SPA. He then disabled JavaScript in his browser and re-loaded the “home” page of the application and was able to demonstrate still being able to navigate around the pages of the demo application just as smoothly (due to the page’s having been rendered on the server and sent to the client without needing further JavaScript processing to render them). This was truly mind blowing stuff. Since the server-side NodeServices library wraps NodeJS on the server, we can write server-side C# code and call out to invoke functionality within any NodeJS package that might be installed. This means that if have a NodeJS package that, for example, renders some HTML markup and converts it to a PDF, we can generate that PDF with the Node package from C# code, sending the resulting PDF file to the client browser via a standard ASP.NET mechanism. Sandeep wraps up his talk by sharing some links to find out more about JavaScriptServices and he also provides us with a link to his GitHub repository for the demo code he’s used in his session.
After Sandeep’s talk, it was time to head back to the main conference area as it was time for lunch. The food at DDDSW is usually something special, and this year was no exception. Being in the South West of England, it’s obligatory to have some quintessentially South West food – Pasties! A choice of Steak or Cheese and Onion pasty along with a packet of crisps, a chocolate bar and a piece of fruit and our lunches were quite a sizable portion of food. After the attendees queued for the pasties and other treats, we all found a place to sit in the large communal area and ate our delicious lunch.
The lunch break at DDDSW was an hour and a half and usually has some lightning or grok talks that take place within one of the session rooms over the lunch break. Due to the sixth form centre not being the largest of venues (some of the session rooms were very full during some sessions and could get quite hot) and the weather outside turning into being a rather nice day and keeping us all very warm indeed, I decided that it would be nice to take a walk around outside to grab some fresh air and also to help work off a bit of the ample lunch I’d just enjoyed.
I decided a brief walk around the grounds of the St. Mary Redcliffe church would be quite pleasant and so off I went, capturing a few photos along the way. After walking around the grounds, I still had some time to kill so decided to pop down to the river front to wander along there. After a few more minutes, I stumbled across a nice old fashioned pub and so decided that a cheeky half pint of ale would go down quite nice right about now! I ordered my ale and sat down in the beer garden overlooking the river watching the world go by as I quaffed! A lovely afternoon lunch break it was turning out to be.
After a little while longer it was time to head back to the Redcliffe sixth form centre for the afternoon’s sessions. I made my way back up the hill, past the church and back to the sixth form centre. I grabbed a quick coffee in the main conference hall area before heading off down the corridor to the session room for the first afternoon session. This one was Joel Hammond-Turner’s “Service Discovery With Consul.io for the .NET Developer”
Joel’s session is about the Consul.io software used for service discovery. Consul.io is a distributed application that allows applications running on a given machine, to query consul for registered services and find out where those services are located either on a local network or on the wider internet. Consul is fully open source and cross-platform. Joel tells us the Consul is a service registry – applications can call Consul’s API to register themselves as an available service at a known location – and it’s also service discovery – applications can run a local copy of Consul which will synchronize with other instances of Consul on other machines in the same network to ensure the registry is up-to-date and can query Consul for a given services’ location. Consul also includes a built-in DNS Server, a Key/Value store and a distributed event engine. Since Consul requires all of these features itself in order to operate, they are exposed to the users of Consul for their own use. Joel mentions how his own company, Landmark, is using Consul as part of a large migration project aimed at migrating a 30 year old C++ legacy system to a new event-driven distributed system written in C# and using ASP.NET Core.
Joel talks about distributed systems and they’re often architected from a hardware or infrastructure perspective. Often, due to each “layer” of the solution requiring multiple servers for failover and redundancy, you’ll need a load balancer to direct requests to this layer to a given server. If you’ve got web, application and perhaps other layers, this often means load balancers at every layer. This is not only potentially expensive – especially if your load balancers are separate physical hardware – it also complicates your infrastructure. Using Consul can help to replace the majority of these load balancers since Consul itself acts as a way to discover a given service and can, when querying Consul’s built-in DNS server, randomize the order in which multiple instances of a given service are reported to a client. In this way, requests can be routed and balanced between multiple instances of the required service. This means that load balancing is reduced to a single load balancer at the “border” or entry point of the entire network infrastructure.
Joel proceeds by showing us some demo code. He starts by downloading Consul. Consul runs from the command line. It has a handy built-in developer mode where nothing is persisted to disk, allowing for quick and easy evaluation of the software. Joel say how Consul command line API is very similar to that of the source control software, Git, where the consul command acts as an introducer to other commands allowing consul to perform some specific function (i.e. consul agent –dev starts the consul agent service running in developer mode).
The great thing with Consul is that you’ll never need to know where on your network the consul service is itself. It’s always running on localhost! The idea is that you have Consul running on every machine on localhost. This way, any application can always access Consul by looking on localhost on the specific Consul port (8300 by default). Since Consul is distributed, every copy of Consul will synchronize with all others on the network ensuring that the data is shared and registered services are known by each and every running instance of Consul. The built in DNS Server will provide your consuming app with the ability to get the dynamic IP of the service my using a DNS name of something like [myservicename].services.consul. If you've got multiple servers for that services (i.e. 3 different IP's that can provide the service) registered, the built-in DNS service will automagically randomize the order in which those IP addresses are returned to DNS queries - automatic load-balancing for the service!
Consuming applications, perhaps written in C#, can use some simple code to both register as a service with Consul and also to query Consul in order to discover services:
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory, IApplicationLifetime lifetime)
{
loggerFactory.AddConsole(Configuration.GetSection("Logging"));
loggerFactory.AddDebug();
app.UseMvc();
var serverAddressFeature = (IServerAddressesFeature)app.ServerFeatures.FirstOrDefault(f => f.Key ==typeof(IServerAddressesFeature)).Value;
var serverAddress = new Uri(serverAddressFeature.Addresses.First());
// Register service with consul
var registration = new AgentServiceRegistration()
{
ID = $"webapi-{serverAddress.Port}",
Name = "webapi",
Address = $"{serverAddress.Scheme}://{serverAddress.Host}",
Port = serverAddress.Port,
Tags = new[] { "Flibble", "Wotsit", "Aardvark" },
Checks = new AgentServiceCheck[] {new AgentCheckRegistration()
{
HTTP = $"{serverAddress.Scheme}://{serverAddress.Host}:{serverAddress.Port}/api/health/status",
Notes = "Checks /health/status on localhost",
Timeout = TimeSpan.FromSeconds(3),
Interval = TimeSpan.FromSeconds(10)
}}
};
var consulClient = app.ApplicationServices.GetRequiredService<IConsulClient>();
consulClient.Agent.ServiceDeregister(registration.ID).Wait();
consulClient.Agent.ServiceRegister(registration).Wait();
lifetime.ApplicationStopping.Register(() =>
{
consulClient.Agent.ServiceDeregister(registration.ID).Wait();
});
}
Consul’s services can be queried based upon an exact service name or can be queries based upon a tag (registered services can assign multiple arbitrary tags upon registration to themselves to aid discovery).
As well as service registration and discovery, Consul also provides service monitoring. This means that Consul itself will monitor the health of your service so that should one of the instances of a given service become unhealthy or unavailable, Consul will prevent that service’s IP Address from being served up to consuming clients when queried.
Joel now shared with us some tips for using Consul from within a .NET application. He says to be careful of finding Consul registered services with .NET’s built-in DNS resolver. The reason for this is that .NET’s DNS Resolver is very heavily cached, and it may serve up stale DNS data that may not be up to date with the data inside Consul’s own DNS service. Another thing to be aware of is that Consul’s Key/Value store will always store values as byte arrays. This can sometimes be slightly awkward if we’re storing mostly strings inside, however, it’s trivial to write a wrapper to always convert from a byte array when querying the Key/Value store. Finally, Joel tells us about a more advanced feature of Consul which is the “watches”. Theses are effectively events that Consul will fire when Consul’s own data changes. A good use for this would be to have some code that runs in response to one of these events that can re-write the border load balancer rules to provide you with a fully automatic means of keeping your network infrastructure up-to-date and discoverable.
In wrapping up, Joel shares a few links to his GitHub repositories for both his demo code used during his session and the slides.
After Joel’s talk it was time for another refreshment break. This one being the only one of the afternoon was accompanied by another DDD South West tradition – cream teas! These went down a storm with the conference attendees and there was so many of them that many people were able to have seconds. There was also some additional fruit, crisps and chocolate bars left over from lunch time, which made this particular break quite gourmand.
After a few cups of coffee which were required to wash down the excellent cream teas and other snack treats, it was time to head back down the corridor to the session rooms for the final session of the day. This one was Naeem Sarfraz’s “Layers, Abstractions & Spaghetti Code: Revisiting the Onion Architecture”.
Naeem’s talk was a retrospective of the last 10+ years of .NET software development. Not only a retrospective of his own career, but he posits that it’s quite probably a retrospective of the last 10 years of many people’s careers. Naeem starts by talking about standards. He asks us to cast our minds back to when we started new jobs earlier in our career and that perhaps one of the first things that we had to do in starting in the job was to read page after page of “standards” documents for the new team that we’d joined. Coding standards, architecture standards etc. How times have changed nowadays where the focus is less on onerous documentation but more of expressive code that is easier to understand as well as such practices as pair programming allowing new developers to get up to speed with a new codebase quickly without the need to read large volumes of documentation.
Naeem then moves on to show us some of the code that he himself wrote when he started his career around 10 years ago. This is typical of a lot of code that many of us write when we first started developing, and has methods which are called in response to some UI event that has business logic, database access and web code (redirections etc.) all combined in the same method!
Naeem says he quickly realised that this code was not great and that he needed to start separating concerns. At this time, it seemed that N-Tier became the new buzzword and so newer code written was compliant with an N-Tier architecture. A UI layer that is decoupled from all other layers and only talks to the Business layer which in turn only talks to the Data layer. However, this architecture was eventually found to be lacking too.
Naeem stops and asks us about how we decompose our applications. He says that most of the time, we end up with technical designs taking priority over everything else. We also often find that we can start to model our software by starting at the persistence (database) layer first, this frequently ends up bleeding into the other layers and affects the design of them. This is great for us as techies, but it’s not the best approach for the business. What we need to do is start to model our software on the business domain and leave technical designs for a later part of the modelling process.
Naeem shows us some more code from his past. This one is slightly more separated, but still has user interface elements specifically designed so that rows of values from a grid of columns are designed in such a way that they can be directly mapped to a DataTable type, allowing easy interaction with the database. This is another example of a data-centric approach to the entire application design.
We then move on to look at another way of designing our application architecture. Naeem tells us how he discovered the 4+1 approach, which has us examining the software we’re building by looking at it from different perspectives. This helps to provide a better, more balanced view of what we’re seeking to achieve with it’s development.
The next logical step along the architecture path is that of Domain-Driven Design. This takes the approach of placing the core business domain, described with a ubiquitous language – which is always in the language of the business domain, not the language of any technical implementation – at the very heart of the entire software model. One popular way of using domain-driven design (DDD) is to adopt an architecture called “Onion architecture”. This is also known as “Hexagonal architecture” or “Ports and adapters architecture”.
We look at some more code, this time code that is compliant with DDD. However, on closer inspection, it really isn’t. He says how this code adopts all the right names for a DDD compliant implementation, however, it’s only the DDD vernacular that’s been adopted and the code still has a very database-centric, table-per-class type of approach. The danger here is that we can appear to be following DDD patterns, but we’re really just doing things like implementing excessive interfaces on all repository classes or implementing generic repositories without really separating and abstracting our code from the design of the underlying database.
Finally, we look at some more code, this time written by Greg Young and adopting a CQRS based approach. The demo code is called SimplestPossibleThing and is available at GitHub. This code demonstrates a much cleaner and abstracted approach to modelling the code. By using a CQRS approach, reads and writes of data are separated also, with the code implementing Commands - which are responsible for writing data and Queries – which are responsible for reading data. Finally, Naeem points us to a talk given by Jimmy Bogard which talks about architecting application in “slices not layers”. Each feature is developed in isolation from others and the feature development includes a “full stack” of development (domain/business layer, persistence/database layer and user interface layers) isolated to that feature.
After Naeem’s session was over, it was time for all the attendees to gather back in the main conference hall for the final wrap-up by the organisers and the prize draw. After thanking the various people involved in making the conference what it is (sponsors, volunteers, organisers etc.) it was time for the prize draw. There were some good prizes up for grabs, but alas, I wasn’t to be a winner on this occasion.
Once again, the DDDSW conference proved to be a huge success. I’d had a wonderful day and now just had the long drive home to look forward to. I didn’t mind, however, as it had been well worth the trip. Here’s hoping the organisers do it all all over again next year!
