Load Testing with Visual Studio Team System

Visual Studio Team System offers a lot functionality for automated testing. While you can write unit tests with MSTest in “Visual Studio 2005/2008 Team Edition for Developers” you’ll need “Visual Studio 2005/2008 Team Edition for Testers” to author Web Tests and Load Tests.

Web Tests are essentially a script to execute against a web site, a Load Test can execute one or more Web Tests with a set of parameters. These parameters can be:

• User count (this can ramp up and down)
• Browser mix (IE 5.5/6.0/7.0/8.0, Firefox 2.0/3.0 and Netscape 6.0 plus Smart Phone and Pocket PC)
• Network mix (LAN, Cable, dial-up)

Once you have authored your Web and Load Tests you can run the Load Tests from “Visual Studio 2005/2008 Team Edition for Testers”. In this scenario you are limited in that you can only simulate a limited number of users (exact number depends on your machine). To scale out and simulate the load of many thousands of users you can use the “Visual Studio Team System 2005/2008 Test Load Agent”. This is available as a separate product, download a 90 day trial here:

http://www.microsoft.com/downloads/details.aspx?FamilyID=572e1e71-ae6b-4f92-960d-544cabe62162&displaylang=en

The “Visual Studio Team System 2005/2008 Test Load Agent” allows you to set up Test Agents and Test Controllers. Test Agents simulate requests from Users and Test Controllers orchestrate the Test Agents.

According to the documentation, a Test Agent on a server with a 2.6 GHz processor and 2GB RAM can simulate approximately 1000 virtual users. A single Test Controller installed on a similar specification server can orchestrate approximately 30 Test Controllers. These numbers are approximate as the specifics will depend on your web application. If your web application is graphics heavy you will not be able to simulate as many users to due to increase memory demands of the requests for larger pages.

These are very rough numbers, but if your average web page size is 300KB and you have 1000 virtual users simulated per Test Agent, the server running the Test Agent needs to deal with up 300MB worth of requested data at once. The load on each Test Agent is mitigated in that the Test Agent simulates user thinking times (times are configurable in the scripts). So each machine won’t be making 1000 requests constantly, more like 1000 requests every 4 or 5 seconds. So a server with 2GB RAM supporting 1000 virtual users seems a reasonable basis for estimates.

If you want to simulate 10,000 users you would need:

1 Test Agent per 1000 virtual users = 10 x Test Agents = 10 x servers

1 Test Controller per 30 Test Agents = 1 x Test Controller = 1 x servers

This means a total of 11 servers, each with 2.6GHz processor and 2GB RAM.

You can find more information on the metrics here: http://msdn.microsoft.com/en-us/library/ms253092.aspx

Test from Live Writer now I can install it to Windows Server 2008

With a bit of luck of I should be able to post some pretty pictures…

And that’s me off to my ivory tower…

April News

Last month Microsoft held its annual MIX conference in Las Vegas. The MIX event is similar to the MSDN conference but the content is targeted towards Microsoft’s web platforms. The event saw a lot of news about Silverlight, the highlight being the official announcement on Silverlight 3.0. The beta for this was made publicly available during the conference. The conference also saw the final release of Microsoft’s ASP.NET MVC framework and further announcements on Windows Azure.

Microsoft News

·         The widely expected announcement of Silverlight 3 Beta was confirmed at the MIX conference along with betas for the necessary tooling for Visual Studio and the Expression Blend designer suite. New features in Silverlight 3 include support for higher quality video and audio, 3D graphics, hardware acceleration, animation effects, improved accessibility for partially sighted people, Search Engine Optimisation (SEO) and a whole raft of changes to enable a richer user experience. Silverlight 3 also supports an “out of browser” experience meaning Silverlight application can be hosted outside of the browser frame and effectively become rich desktop applications delivered by the web. The Register has an analysis of Silverlight versus the rival Air/Flash combination from Adobe, details here, here and here.

·         Silverlight has also seen an additional large scale application in the form of Microsoft WorldWide Telescope.

·         Silverlight also sees further platform support with tooling for the popular Eclipse development environment. With Moonlight (the open source implementation of Silverlight supporting Linux platforms), Silverlight’s own compatibility with the Mac operating system and Firefox and Safari web browsers its and even more compelling cross platform solution, offering the benefits of a rich user experience and ease of deployment for web applications.

·         Microsoft’s next client operating system, Windows 7, which had a beta release back in February will likely reach Release Candidate in May.

·         The final of Internet Explorer 8 has been released.

·         Microsoft’s ASP.NET MVC framework has been released. Interestingly this is has been released under Microsoft Public License which is an OSI approved license. This could be the first steps of Microsoft embracing open source software.

·         Windows Azure development continues to gather momentum. Microsoft announced plans to deliver core relational database features as part of SQL Data Services (the data access service for Windows Azure). The is equivalent to SQL Server in the cloud and porting applications to this platform should be as simple as “changing your connection string”. Further analysis on SQL Data Services from The Register here. Microsoft will imminently roll out Windows Azure to multiple data centres in North America. A European data centre (most likely in Ireland) is in the pipeline.

·         Microsoft’s interactive table top display technology, Surface, is also slowly gaining traction. BMW have found another commercial application for the product.

·         Microsoft released more details of their Windows Marketplace for Mobile offering which allows developers to sell applications through a Microsoft portal. Developers will keep 70% of the sales revenue.

Community & Industry News

·         The UK Azure User Group had its inaugural meeting on 31^st March. Members of the Microsoft Azure team spoke and did a couple of demos. Some useful information was made available, the pricing and SLAs will be announced this summer and the Azure will go commercial in Q4 this year. The next event will be on the 14^th May, more information here.

·         IBM to buy Sun Microsystems?

Articles & Blogs

·         Scott Guthrie’s MIX wrap up.

·         Somasegar gives us a recap on the .NET tooling we have had post VS2008.

·         Microsoft are building a framework for composing web applications with Silverlight and ASP.NET using Domain Driven Design principals. The framework is at a very early stage and has been labelled .NET RIA Services. RIA meaning Rich Internet Applications. The framework offers a common approach to data access using Object Relational Mapping techniques and consistent validation rules on the client and server. The two MIX sessions are available here.

·         ASP.NET MVC and mobile web applications.

·         Scott Guthrie’s MIX Silverlight for Business Applications demo.

·         Channel 9 on Silverlight 3.0 for great business applications.

·         Windows Azure applications now run with Full Trust.

·         Visual Studio 2010 will see a deployment framework called MSDeploy. This will be used to seamlessly deploy web applications dealing with the all dependencies down to the IIS settings.

·         A list of Microsoft Product Teams on Twitter.

·         Udi Dahan looks at making better Domain models, “From CRUD to Domain Driven Fluency”.

·          Improving Session Factory Initialisation in NHibernate.

·         SQL Server SCOPE_IDENTITY() sometimes returns incorrect value.

·         In case anyone was wondering, the technology that was originally Microsoft BizTalk Services is now the .NET Services component of Windows Azure. This includes a Service Bus that can traverse firewalls (built with Windows Communication Foundation), Access Control (built with Microsoft’s CardSpace) and workflow (built with Windows Workflow Foundation).

Downloads & Tools

·         ASP.NET MVC 1.0 has been released.

·         Silverlight 3 SDK Beta 1 is available.

·         Windows Azure March CTP is available (this includes both the Tools for Visual Studio and the SDK).

·         PEX is a project from Microsoft Research that analyses your code and produces unit tests to test edge case scenarios.

·         Chess is another project from Microsoft Research that analyses multi-threaded code and produce unit tests to check for dead-locks and other problems with multi-threading.

·         ADO.NET Data Services (Microsoft’s REST framework) has a new CTP.

·         Web Service Studio is a tool that will allow you to invoke web services interactively, it supports WCF and REST.

·         Velocity is a true distributed cache from Microsoft, currently in beta. It’s final roadmap is yet to be defined but here is NHibernate support and Velocity will be included in ASP.NET 4.0. There is a Q & A available for Velocity.

·         The SQL Data Services component of Windows Azure has an SDK available.

·         The Live Framework (another component of Windows Azure) has had an SDK refresh with the April CTP.

·         The Windows Installer Clean-up Utility will help remove stubborn programs you might have difficulty un-installing.

·         ReSharper 4.5 Beta is available for download. Please note that version 4.5 is available as a free upgrade to people with a licence for version 4.1.

·         Accompanying the final release of ASP.NET MVC, is a version 1.0 release of the MVC Contrib project.

·         The session videos from the MIX conference are available for download here and here.

·         There’s an open source Database Versioning and Documentation Tool for Microsoft SQL Server available on CodePlex.

And finally...

·         Ever had a frustrating call to a tech support centre? Now there’s an un-official “handshake” to let them know you’re not a luddite.

·         Interesting news that Skype is the world’s biggest international telco, accounting for 8% of the world’s talk time.

Windows Azure costs for developers

I attended the first meeting of the Azure User Group UK tonight (http://ukazurenet.com/) and an interesting question popped up. Right now you can register for Azure for free to work with the CTP releases (http://go.microsoft.com/fwlink/?LinkID=129453). However, eventually Microsoft will monetise this service. SLAs and pricing should be published this summer and you will have to pay for services come Q4 2009. 

At this point what happens to developers who might be putting together proof of concepts or prototypes, or for projects still in the development stage? Well, it was suggested by a Microsoft employee that access to Windows Azure may be included in a Visual Studio license or MSDN Subscription. I think this is a pretty good solution to the problem.