Thursday 28 October 2010

How not to namespace (Part 1)

This is always something that bugs me. Consider the following Visual Studio projects:

  • Acme.PetShop.Common
  • Acme.PetShop.Data
  • Acme.PetShop.Services

What is the “Common” suffix for? I don’t see the point (at all). Namespaces are logical groupings, the “common” namespace is “Acme.PetShop”. The Visual Studio projects should be as follows:

  • Acme.PetShop
  • Acme.PetShop.Data
  • Acme.PetShop.Services

“Acme.PetShop” is common to “Acme.PetShop.Data” and “Acme.PetShop.Services”.

There is additional reasoning. For any supporter of the “Acme.PetShop.Common” namespace, I would ask you what is the distinction or rule for putting types in an “Acme.PetShop.Common” namespace and putting types in an “Acme.PetShop” namespace? I am willing to bet you cannot come up with such a distinction or rule. The “Acme.PetShop” namespace always exists by virtue that the “Acme.PetShop.Data” and “Acme.PetShop.Services” namespace exist.

Furthermore, you won’t be able to find a “Common” namespace anywhere in the .NET Framework Base Class Libraries (I have checked).


Monday 25 October 2010

Timeouts to the subscription database when starting NServiceBus

When using durable subscription storage (using a database), when starting NServiceBus you may see the following errors logged:

NHibernate.Util.ADOExceptionReporter {log4net:HostName=MyHost} - NHibernate.ADOException: While preparing SELECT this_.SubscriberEndpoint as y0_ FROM [Subscription] this_ WHERE this_.MessageType in (@p0) an error occurred ---> System.InvalidOperationException: Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.

NHibernate.Util.ADOExceptionReporter {log4net:HostName=MyHost} - While preparing SELECT this_.SubscriberEndpoint as y0_ FROM [Subscription] this_ WHERE this_.MessageType in (@p0) an error occurred

NHibernate.Util.ADOExceptionReporter {log4net:HostName=MyHost} - System.InvalidOperationException: Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.

NHibernate.Util.ADOExceptionReporter {log4net:HostName=MyHost} - Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.

NServiceBus.Unicast.UnicastBus {log4net:HostName=MyHost} - MyHostFailed handling message.
System.InvalidOperationException: Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.

NServiceBus.Unicast.Transport.Msmq.MsmqTransport {log4net:HostName=MyHost} - Failed raising 'transport message received' event for message with ID=00000000-0000-0000-0000-000000000000\000000
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> NHibernate.ADOException: While preparing SELECT this_.SubscriberEndpoint as y0_ FROM [Subscription] this_ WHERE this_.MessageType in (@p0) an error occurred ---> System.InvalidOperationException: Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.

NServiceBus.Unicast.Transport.Msmq.MsmqTransport {log4net:HostName=MyHost} - Message has failed the maximum number of times allowed, ID=00000000-0000-0000-0000-000000000000\000000.

The problem here is to do with having a lot of logical messages in a single physical message and that physical message is the first one that is received by NServiceBus after starting up.

In NServiceBus, you can send multiple message instances at once. You can do this by calling “IBus.Publish(IMessage[])”. These multiple message instances are wrapped up into a single physical message. There does not appear to be any hard limit on the number of logical messages that can be put into a single physical message. You can, therefore, send a lot of logical messages in a single physical message.

When NServiceBus starts up and receives the first physical message, if you are using durable subscription storage (using a database), NServiceBus will look up the subscriptions from the subscriptions database (so that it knows where to send the message). Once the subscriptions list is initialised, it is stored in memory so there are no subsequent database look-ups. Note that new subscriptions are received via messages, so the subscription list in memory will be kept up to date as subscriptions are added (when a new subscription is received it is added to the in-memory list and written to the subscription database).

NServiceBus will perform the subscription database look-up for every logical message in the first physical message it receives, opening a connection to the subscription database for each logical message. It appears that NServiceBus will not close the connection to the subscription database until it has finished processing the message. All logical messages must be processed within the same operation when they are contained in a single physical message. Should you have more logical messages than the maximum database connection pool size, the operation will not complete (it will timeout).

For example, let’s say your connection string for the subscription database allows 100 connections in the connection pool but you have 150 logical messages in the first physical message that is received. As the message is the first received, NServiceBus needs to look up the subscriptions. So NServiceBus try to will open 150 connections to the subscription database, it will successfully open 100. When processing the 101st logical message, it will not be able to establish a connection because there are 100 open connections and that is the maximum allowed (the first 100 connections will not be closed until the whole operation on the physical message is completed). The operation to establish the 101st connection will timeout. The whole operation on the physical message will be considered to have failed, any transactions will be rolled back and the physical message will be re-tried the configured number of times (with each re-try failing for the same reason). Eventually the physical message will be sent to the error queue and the next physical message will be processed.

Should a subsequent physical message be processed successfully (i.e. one with less logical messages than allowed database connections), you can re-try your physical message with a large number of logical messages and it will also be processed successfully (because the subscriptions list will already be initialised in memory at that point).

The short-term work around is to have less logical messages than you have allowed database connections (for that first message). The default connection pool size is 100, so that is the limit. I am looking for a long term solution, I think this is a bug in NServiceBus.


Monday 18 October 2010

Visual Studio 2008 fails to create BizTalk 2009 project

I got bitten by this (again) on my new machine, putting this note here for my own reference as much as anything.

When attempting to create a BizTalk 2009 project in Visual Studio 2008…

  • File –> New Project… –> [Any BizTalk Project Template]

or

  • (From Solution) Add –> New Project… –> [Any BizTalk Project Template]

…you might get presented with the following error message:

  • Creating project ‘[Project name]’… project creation failed.

When you install Service Pack 1 for VS2008 it seems to overwrite a Registry key used by the BizTalk 2009 SDK. Putting the Registry key back to its original value fixes the problem.

The safest way to achieve this is to repair the BizTalk 2009 installation after installing Service Pack 1 for VS2008. The repair only takes a few moment to run (or at least it did for me). Some on-line resources suggest fixing the Registry key manually but I have trust issues with that and so long as you have the BizTalk 2009 ISO on hand, the repair operation takes about the same amount of time as opening RegEdit.exe and finding the right key (making sure you get the correct 32/64-bit Registry key).


Friday 15 October 2010

What is coupling?

Coupling is a measure of dependencies. Where Component A depends on Component B, coupling exists.

The direction of coupling matters, there are two kinds:

  • Afferent – who depends on you.
  • Efferent – who you depend on.

There are characteristics and rules of thumb for different coupling “signatures”. For example:

  • An “Observer” class might have lots of efferent coupling but no afferent coupling i.e. the Observer will depend on lots of classes but few (or perhaps no) classes will depend on the observer. This is okay, it is probably expected.
  • Conversely, a framework component might have lots of afferent coupling but no efferent coupling i.e. lots of components might depend on the framework but the framework will depend on few (or perhaps no) components itself. Again, this is okay and is probably expected.

You may find coupling patterns within your application, depending on the specifics of your business domain.

When measured in this way, coupling becomes a useful metric. It enables you to ask questions like “If I change Component A, how many components will this impact?”. This information allows you to assess risk when making changes, therefore you are able to determine if you can change a component safely.

You should minimise both forms of coupling but having no coupling is not possible.


Thursday 14 October 2010

Silent, unattended installation of SQL Server 2008

This is something that’s not nearly as hard as it sounds. Silent, unattended installs for any Microsoft product hang on the .ini file, the usual problem is creating these. In the SQL Server 2008 world there is a neat trick to generating your .ini file. If you run through the install wizard, on the “Ready to Install” dialog near the end you will be shown a “Configuration file path” value at the bottom of the dialog box. This is the path to the .ini file representing the configuration you just selected (which components, accounts etc.).

image

In the above summary, I’ve selected most of the features aside from SQL Server Integration Services, Report Server (I just want the Database Engine for BizTalk Server 2009). I’ve also selected a named instance and a custom installation directory for the instance (because I want to run side-by-side with SQL Server 2008 R2).

Note the “Configuration file path” value and quit the installer.

Take a copy of the .ini file from the path shown and put it in a simple path. You need to make the following changes:

  • Change the “QUIET” value from “False” to “True” (to suppress dialogs)
  • Change the “INDICATEPROGRESS” value from “False” to “True” (to show output in the console)

Run the installer using the following command line:

  • C:\>[Path to SQL Server media]\setup.exe /CONFIGURATIONFILE="[some path]\ConfigurationFile.ini"

The console should present you with something like the following:

One or more affected files have operations pending.
You should restart your computer to complete this process.

Things to be aware of:

  • The accounts you chose for the Administrator roles are stored in the .ini file, so choose Domain Accounts/Groups or well known accounts/groups (like BUILTIN\Administrators). If you choose machine specific accounts you will have problems installing to other servers.
  • The path to the SQL Server 2008 installation media is also stored in the .ini file so store it in a common location like a network share (and run the install wizard from this location when generating the .ini file).

You can use this strategy to ensure consistency of SQL Server installations across a team of developers or for quicker building of virtual machines.

Note this technique also works with MS SQL Server 2008 R2 but you must do two things differently:

  • Comment out the “UIMODE” entry in the .ini file (put a “;” at the start of the line).
  • Add the “/IAcceptSQLServerLicenseTerms” switch when calling setup.exe (no need for a value with the switch).

Wednesday 13 October 2010

Installing BizTalk 2009 on Windows Server 2008 R2

Be aware you will likely hit some errors when running the configuration wizard. Typically something like:

Failed to connect to the SQL database 'SSODB' on SQL Server 'SQLServerName'
0x80131700 (Win32)

There is a known fix for this, details on Microsoft’s Knowledge Base here.

In short, you need to download Hotfix KB2252691 and run regasm.exe against “ssosql.dll”.

Remember to run the 64-bit version of regasm.exe on 64-bit systems.


Tuesday 12 October 2010

Package management in .NET

Looks like Microsoft are solving the horrendous situation around package management in the .NET world. The solution is called NuPack.

NuPack solves the annoying situation that when working with open source libraries you have to hunt around for the various versions, find which ones are compatible and manually add then to your solution. Manually adding open source frameworks to your solution can involve all sorts of things such as putting binaries in a “library” folder, modifying configuration files and jumping through various other hoops. Mileage varies according to framework.

NuPack address most, if not all of these problems. Taking inspiration from Ruby Gems, it manifests itself through a Visual Studio Add-In. It automates the entire process of adding an open source framework to your solution. Binaries will be downloaded from a centralised feed, your configuration files will be updated and what ever else needs to be done will be done. There’s also the neat trick that should you wish to use FrameworkX and it depends on FrameworkY, then NuPack will automatically retrieve (and configure) FrameworkY when you request FrameworkX (it will resolve a whole dependency tree for you).

It has the added benefit that it does not install anything or put items in the GAC. You can add a dependency to your solution with NuPack and commit your solution to source control as normal. Should another developer then get the solution from source control everything will be there that they need, they don’t need to run NuPack themselves or even have NuPack installed. NuPack also has a PowerShell interface.

It can be found on CodePlex and the usual suspects have introductory posts here (ScottGu), here (Hanselman) and here (Haack).

NuPack is not the only candidate solution to this problem though, OpenWrap is an alternative.


Friday 8 October 2010

Aiming for High Availability

I like this…

"My suspicion is the number-one cause of outages is human error and after that is badly tested apps, and everything else is way down. What we ought to be focus on in terms of high availability is probably not what we are focused on at all."

- Michael Stonebraker, http://www.theregister.co.uk/2010/09/13/michael_stonebraker_interview/


About Me