Even though this is a part of my “warm-up” blog posts for GOTO Aarhus 2012 Conference, I’ll start out referring to a related event this week. At our latest meeting in the Danish TFS User Group, held at the Microsoft office in Hellerup, Rune Abrahamsson from BRFkredit had a presentation on how they are using http://cuite.codeplex.com/ for developing functional tests for one of their systems. If you are interested, Mads (our agile coach at ScanJour) has also blogged from the user group here.

Although the people attending the TFS Users Group meetings are mostly developers, we often have at least one testing-related topic on the agenda, and this time it was clear that many of the present developers does have experience with developing functional tests, often using a UI automation framework like Microsoft Coded UI. So even before attending the GOTO Aarhus 2012 Conference, it seems I can conclude that developers are writing functional tests, which pleases me, as I think the people writing the software, also are the best at writing automation for it. The testers can then do actual quality assurance, by ensuring that the customer requirements have been automated, as well as performing exploratory testing to exercise the software in new ways.

Sorry for the bad picture quality; the slide shows at the bottom which areas (including functional UI tests) are covered by developer tests, whereas e.g. exploratory tests are performed by their manual testers (which are actually domain experts, not full-time testers). I find it very positive to see a development team take quality seriously, by including test automation, when they are in a situation where they are lacking testers. I guess this is not an uncommon scenario in the industry, that you have to get test assistance from other teams/departments.

And it makes me wonder if manual testers should be forced to not do manual scripted test at all, but only do exploratory testing and quality assurance, since the developers will fill the gap themselves, by automating the scripted test cases?

To put this in the context of this years GOTO conference, I have been looking at the biographies of some of the speakers, and found a video by Steve Freeman on Sustainable Test-Driven Development, where he touches topics like:

• Too coupled production and test code, which makes refactoring difficult (beginning of video)
• Test code structure (similar to Given/When/Then pattern of e.g. SpecFlow)
• “You come back to the code after 6 months, and forgot why you did this” (approx. 13:00)
• Patterns for writing test code. As simple as good variable naming, using DSL syntax to have more more readable test code
• Prepare for your test to fail at some point, simply by having clear error messages when it happens (“Explain yourself”, “Describe yourself” and “Tracer Objects” around 23 mins into the video).
• Make your tests robust, e.g. “Only Enforce Order When It Matters” (around 40:00)
• “Tests Are Code Too” (43:35), which is the last slide and also seems to be the headline for this presentation

The last slide is shown here:

Although all four bullets here are important, I have myself faced the “If it’s hard to test, that’s a clue” problem, when developing unit tests, both for production code as well as test automation framework code. As an automation tester, I also regularly have the problem when writing automated functional tests against production code, e.g. missing id’s on UI controls, no clear interface for testing etc. When the development team is writing automated functional tests as part of definition of done, it should hopefully result in code better suited for automation. And then the general conclusion, that test automation should be treated like any other code activity.

This year Steve Freeman has a session on Raspberry Pi with extra toppings Monday at 13:20, which unfortunately is a timeslot where I have also some other sessions I would like to see (What is value and Mythbusting Remote Procedure Calls), but it might be that I have to change my mind; as it could also be fun to get an introduction to the Raspberry Pi.

After watching this video, I feel confident that I will meet developers at the GOTO Aarhus 2012 Conference this year with a quality/testing mindset, and I’m looking forward to talk with you about your view on test automation, and how I as an automation tester can bring value and increase the quality of our software products, even if it's no longer a dedicated test automation developer writing the functional tests.

Feel free to comment on this post below.

Today I went to one of the lunch sessions. A little strange name, as you cannot bring you lunch with you to these sessions, as they are held in the usual rooms only containing chairs. They are just placed in the lunch break timeslot, and seems to be a possibility for the exhibitors to indirectly demo their products.

The topic of this session was “A Live Demo Introduction to the Kinect API”. I had hoped that the session would show ways to actual use a Kinect, but was more a presentation of a layer on top of the Kinect API that the speaker had developed. I guess there must be possible usage scenarios for Kinect-enabled software (also in business applications), but this session didn’t show any.

Couldn’t find any testing related sessions in next timeslot, so went to DEV303 - ASP.NET Loves HTML5 on support for HTML5 in Visual Studio 2012. Also attended the workshop yesterday where we developed a Metro style application for Windows 8 using javascript, and surely it seems that use of HTML5/javascript is increasing. An example where use of HTML5 for making rounded corners, reduces 37 lines of CSS and HTML to 5 lines (left). On the right slide showing a new feature in Visual Studio to start up a different browser when debugging the web application.

Then the session showed improvements to ASP.NET Server controls, e.g. FileUpload that has been extended with support for multiple files (since this is possible in HTML5), improvements to CSS editor and intellisense/compile-time-check of javascript code. Certainly useful for javascript developers, and my impression is that javascript will become increasingly popular.

Last session of the day was DEV312 - Creating Robust, Maintainable Coded UI Tests with Visual Studio 2012 presented by Brian Keller, which I of course had to see

As you can see from the “Objectives” slide above, this is basically the same talk he gave at TechEd last year, now just “upgraded” to TFS 2012. According to Brian Keller, the equivalent session at TechEd North America 2012 contained some more in-depth features presented by a member of the dev team for CodedUI Framework, but apparently this meant that some of the audience was “lost”. My feedback to Brian will be that he should probably split this session into two, one introduction (as this session more or less was) and then an advanced for people that have worked with CodedUI tests.

Some of the new features of CodedUI in Visual Studio 2012/TFS 2012 are different types of testing projects (left), a new field for error message in case an assertion fails (middle) and the most interesting where screenshots are taken for each test step (right). I think this last feature is actually more valuable than a video recording, as you get a picture per step, instead of having to find where in a video recording a specific step has been performed:

Of course he showed the “action recording to CodedUI test” feature, where you transform a manual recorded action recording from Test Manager into C# code. I don’t think this will ever work, as it generates multiple UIMaps, and therefore I don’t think this can be categorized as “maintainable”…

The official part of TechEd today ended with a “Ask the experts” session where we had a change to talk with Microsoft employees about various issues and questions on our current TFS 2010 setup. Really valuable, and I really appreciate this kind of opportunities to speak directly with the people actually developing the products. Yet another fine day at TechEd.

After this the EURO 2012 semi-final match between Germany and Italy was shown in the "Theater" room, with burgers and beers for dinner, even though this wasn't planned originally. Again I'm impressed by the hospitality of Microsoft and the TechEd organizers.

First session today was DEV345 - The Accidential Team Foundation Server Admin, which didn’t quite stick to the subject, but was more about differences between TFS 2010 and TFS 2012 and how to upgrade, and not general information for people who has taken over responsibility of a TFS Server from someone else.

One of the most important notes from this session was to run Best Practices Analyzer Tool for Team Foundation Server, not only once or twice but on a monthly basis. Some other interesting slides on what is automatically enabled when upgrading from TFS 2010 to TFS 2012 (left) and what has to be manually enabled (right):

And some slides on permissions in TFS, which is actually split into TFS, SharePoint and SQL Server Reporting Service at the left. The middle slide (apologize for the bad quality) shows the different permission names in the different parts and finally a “pattern” for configuring one AD per role per team project:

This separation of permissions has not been changed for TFS 2012, so it’s still required to set permissions in TFS, SharePoint and SQL Server Reporting Services. http://tfsadmin.codeplex.com/ was presented which might can help setting permissions.

A new feature of TFS 2012 is to create “teams” within a Project, which will the get own product backlog, burndown charts etc. Hope this feature can reduce number of necessary projects in TFS, as projects introduce a lot of limitations on branching, lab environments (which cannot be deployed across TFS projects).

Next I went to DEV340 - Taking Your Application Lifecycle Management to the Cloud With the Team Foundation Service which showed

• Setting up a hosted TFS on http://tfspreview.com
• Creating a new project
• Create a simple MVC 4 solution, check in code and configuring build definition(s) for compiling code on TFS server
• Create website on Azure
• Configure deployment build definition, to deploy your website to Azure

All this was done within approx. 30 minutes, and is basically all it takes for a company to set up the infrastructure for source control, builds, work item tracking and deployment. If I should emphasize one thing from this TechEd conference, this is probably it.

Also learned from this session that the TFS team is working in 3 weeks sprints, and releasing to tfspreview.com after every release, which have given them a lot of really valuable feedback from customers on features to improve. Sounds like a really interesting success story on releasing often to production. And for Microsoft it’s also a big change compared to earlier versions of TFS, where they could only ship new feature with service packs. And that Microsoft have 4 people operating the tfspreview.com site, running thousands of accounts. Of course there is still work to do for the "on-premise" TFS admin, but certainly it sounds there is a potential optimization in costs for running TFS this way.

From what I have seen of TFS 2012 here at TechEd, I must say that the web UI is really impressive and in general it seems Microsoft have improved TFS on a lot of areas, and really hope to be able to upgrade as soon as possible when TFS 2012 has been released.

Here are slides on terms for using TFS Preview and currently supported features:

The worst session I have attended so far was OSP432 - Application Lifecycle Management: Automated Builds and Testing for SharePoint projects (if the speaker reads this, I’m sure he agrees ). First slides started out by showing which problems to solve. So far, so good, but note the right slide here mentioning use of PowerShell remoting for deploying the SharePoint solution to a test environment.

This is built into TFS as the build-deploy-test feature, but nevertheless he showed how to customize a build process template in order to execute a PowerShell script.

This is surely not the way to do it, and if you’re a SharePoint developer reading this, make sure to use the standard deployment build process template shipped with both TFS 2010 and TFS 2012. He also showed how to create a CodedUI test using the standard recorder tool shipped with Visual Studio. Nothing new for me there, except the fact that CodedUI tests can be used for automating SharePoint UI tests. And then he did the mistake of editing the UIMap designer.cs file. The designer.cs file is autogenerated by VS (as it says in the top of the file), and should of course never be modified. See the file name in upper left corner of image:

Breaking the post today into two parts, as it has become quite long already. Go to second part

In his talk Building Robust, Maintainable Coded UI Tests with Visual Studio 2010 at Belgium TechDays 2011, Brian Keller shows the concept of “using using" (slide 11 in his slide deck) in order to dispose application under test after test has finished.

Of course it’s useful to clean up after your tests, but as I learned the hard way this week (although I‘m certainly not the first one to blog about it…), remember to end your UI based tests, with verifying that your application actually can close without crashing. In our case, a javascript window was blocking execution, which my automated test simply ignored and passed…

My pattern for solving this, is the following code, that besides using an ApplicationUnderTest as suggested by Brian Keller, also tries to close the window (a browser instance in this case) before ending the test case.

/// <summary>
/// Creates and updates a case in Captia UI
/// </summary>
[TestMethod]
[TestCategory("BVT")]
public void TC17015_CreatingAndEditingACase()
{
    // Step: Log on to Captia and click the button “Create case”.
    ApplicationUnderTest browserWindow = CaptiaMainUIActions.OpenCaptia(TestConfig.Instance.CaptiaBaseUrl);

    try
    {
        // Execute test steps/actions...

        CloseCaptiaAndVerify(browserWindow);
    }
    finally
    {
        DisposeBrowserWindowIfNotClosed(browserWindow);
    }
}

private static void CloseCaptiaAndVerify(ApplicationUnderTest browserWindow)
{
    CaptiaMainUIActions.CloseCaptia();
    bool captiaClosed = WaitForm.Wait(
        () => ((browserWindow.Process == null) || (browserWindow.Process.HasExited)),
        TimeSpan.FromSeconds(10), "Waiting for Captia window to close.", 2);

    Assert.IsTrue(captiaClosed, "Expected Captia window to be closed.");
}

private static void DisposeBrowserWindowIfNotClosed(ApplicationUnderTest browserWindow)
{
    if ((browserWindow != null) && (browserWindow.Process != null) && (!browserWindow.Process.HasExited))
        browserWindow.Dispose();
}

The method CloseCaptiaAndVerify() uses my CodedUI UIMap to click the upper-right cross to close the window, and a custom-made WaitForm to wait up to ten seconds (polling every 2 seconds) to verify that window was actually closed. Otherwise test fails.