Rasmus Møller Selsmark

On software test and test automation

TechEd Europe 2012 – Thursday (part two)

clock June 28, 2012 22:40 by author rasmus

Today I went to one of the lunch sessions. A little strange name, as you cannot bring you lunch with you to these sessions, as they are held in the usual rooms only containing chairs. They are just placed in the lunch break timeslot, and seems to be a possibility for the exhibitors to indirectly demo their products.

The topic of this session was “A Live Demo Introduction to the Kinect API”. I had hoped that the session would show ways to actual use a Kinect, but was more a presentation of a layer on top of the Kinect API that the speaker had developed. I guess there must be possible usage scenarios for Kinect-enabled software (also in business applications), but this session didn’t show any.

IMG_1100

 

Couldn’t find any testing related sessions in next timeslot, so went to DEV303 - ASP.NET Loves HTML5 on support for HTML5 in Visual Studio 2012. Also attended the workshop yesterday where we developed a Metro style application for Windows 8 using javascript, and surely it seems that use of HTML5/javascript is increasing. An example where use of HTML5 for making rounded corners, reduces 37 lines of CSS and HTML to 5 lines (left). On the right slide showing a new feature in Visual Studio to start up a different browser when debugging the web application.

IMG_1104 IMG_1105

Then the session showed improvements to ASP.NET Server controls, e.g. FileUpload that has been extended with support for multiple files (since this is possible in HTML5), improvements to CSS editor and intellisense/compile-time-check of javascript code. Certainly useful for javascript developers, and my impression is that javascript will become increasingly popular.

 

Last session of the day was DEV312 - Creating Robust, Maintainable Coded UI Tests with Visual Studio 2012 presented by Brian Keller, which I of course had to see Smile

IMG_1114 IMG_1115

As you can see from the “Objectives” slide above, this is basically the same talk he gave at TechEd last year, now just “upgraded” to TFS 2012. According to Brian Keller, the equivalent session at TechEd North America 2012 contained some more in-depth features presented by a member of the dev team for CodedUI Framework, but apparently this meant that some of the audience was “lost”. My feedback to Brian will be that he should probably split this session into two, one introduction (as this session more or less was) and then an advanced for people that have worked with CodedUI tests.

Some of the new features of CodedUI in Visual Studio 2012/TFS 2012 are different types of testing projects (left), a new field for error message in case an assertion fails (middle) and the most interesting where screenshots are taken for each test step (right). I think this last feature is actually more valuable than a video recording, as you get a picture per step, instead of having to find where in a video recording a specific step has been performed:

IMG_1116 IMG_1117 IMG_1122

Of course he showed the “action recording to CodedUI test” feature, where you transform a manual recorded action recording from Test Manager into C# code. I don’t think this will ever work, as it generates multiple UIMaps, and therefore I don’t think this can be categorized as “maintainable”…

 

The official part of TechEd today ended with a “Ask the experts” session where we had a change to talk with Microsoft employees about various issues and questions on our current TFS 2010 setup. Really valuable, and I really appreciate this kind of opportunities to speak directly with the people actually developing the products. Yet another fine day at TechEd.

After this the EURO 2012 semi-final match between Germany and Italy was shown in the "Theater" room, with burgers and beers for dinner, even though this wasn't planned originally. Again I'm impressed by the hospitality of Microsoft and the TechEd organizers.




TechEd Europe 2012 – Thursday (part one)

clock June 28, 2012 22:01 by author rasmus

First session today was DEV345 - The Accidential Team Foundation Server Admin, which didn’t quite stick to the subject, but was more about differences between TFS 2010 and TFS 2012 and how to upgrade, and not general information for people who has taken over responsibility of a TFS Server from someone else.

One of the most important notes from this session was to run Best Practices Analyzer Tool for Team Foundation Server, not only once or twice but on a monthly basis. Some other interesting slides on what is automatically enabled when upgrading from TFS 2010 to TFS 2012 (left) and what has to be manually enabled (right):

IMG_1058 IMG_1060 

 

And some slides on permissions in TFS, which is actually split into TFS, SharePoint and SQL Server Reporting Service at the left. The middle slide (apologize for the bad quality) shows the different permission names in the different parts and finally a “pattern” for configuring one AD per role per team project:

IMG_1064 IMG_1065 IMG_1066

This separation of permissions has not been changed for TFS 2012, so it’s still required to set permissions in TFS, SharePoint and SQL Server Reporting Services. http://tfsadmin.codeplex.com/ was presented which might can help setting permissions.

 

A new feature of TFS 2012 is to create “teams” within a Project, which will the get own product backlog, burndown charts etc. Hope this feature can reduce number of necessary projects in TFS, as projects introduce a lot of limitations on branching, lab environments (which cannot be deployed across TFS projects).

IMG_1070

 

Next I went to DEV340 - Taking Your Application Lifecycle Management to the Cloud With the Team Foundation Service which showed

  • Setting up a hosted TFS on http://tfspreview.com
  • Creating a new project
  • Create a simple MVC 4 solution, check in code and configuring build definition(s) for compiling code on TFS server
  • Create website on Azure
  • Configure deployment build definition, to deploy your website to Azure

All this was done within approx. 30 minutes, and is basically all it takes for a company to set up the infrastructure for source control, builds, work item tracking and deployment. If I should emphasize one thing from this TechEd conference, this is probably it.

Also learned from this session that the TFS team is working in 3 weeks sprints, and releasing to tfspreview.com after every release, which have given them a lot of really valuable feedback from customers on features to improve. Sounds like a really interesting success story on releasing often to production. And for Microsoft it’s also a big change compared to earlier versions of TFS, where they could only ship new feature with service packs. And that Microsoft have 4 people operating the tfspreview.com site, running thousands of accounts. Of course there is still work to do for the "on-premise" TFS admin, but certainly it sounds there is a potential optimization in costs for running TFS this way.

From what I have seen of TFS 2012 here at TechEd, I must say that the web UI is really impressive and in general it seems Microsoft have improved TFS on a lot of areas, and really hope to be able to upgrade as soon as possible when TFS 2012 has been released.

Here are slides on terms for using TFS Preview and currently supported features:

IMG_1084 IMG_1085

 

The worst session I have attended so far was OSP432 - Application Lifecycle Management: Automated Builds and Testing for SharePoint projects (if the speaker reads this, I’m sure he agrees Smile). First slides started out by showing which problems to solve. So far, so good, but note the right slide here mentioning use of PowerShell remoting for deploying the SharePoint solution to a test environment.

IMG_1087 IMG_1088 IMG_1089

This is built into TFS as the build-deploy-test feature, but nevertheless he showed how to customize a build process template in order to execute a PowerShell script.

IMG_1095

This is surely not the way to do it, and if you’re a SharePoint developer reading this, make sure to use the standard deployment build process template shipped with both TFS 2010 and TFS 2012. He also showed how to create a CodedUI test using the standard recorder tool shipped with Visual Studio. Nothing new for me there, except the fact that CodedUI tests can be used for automating SharePoint UI tests. And then he did the mistake of editing the UIMap designer.cs file. The designer.cs file is autogenerated by VS (as it says in the top of the file), and should of course never be modified. See the file name in upper left corner of image:

IMG_1093

 

Breaking the post today into two parts, as it has become quite long already. Go to second part




TechEd Europe 2012 – Wednesday

clock June 28, 2012 08:33 by author rasmus

Second day here at TechEd started with a keynote on Windows 8, where some interesting features was presented, like:

  • Touch. Had the opportunity later today at a workshop to work for two hours with Windows 8 and a touch screen, and I feel sure that when we all have touch screens in a couple of years, this will be a feature that will be hard to work without. More on this later in this post
  • Run legacy versions of Windows using Hyper-V installed with Windows 8. Started amazingly fast, but it for some reason always do that on Microsoft presentations Smile
  • Remote FX, i.e. 3D graphics in Remote Desktop connections. Microsoft expect more people to remotely log into their machines, and then it’s of course nice to have same user experience as logged directly into the machine. Touch also works through Remote Desktop.
  • Windows-to-go, i.e. running Windows 8 from a USB drive. Looks fine on demos…

 

Two images taken during the Windows 8 keynote showing Hyper-V running a Windows 7 VM inside Windows 8 (left) and the tablet emulator available in Visual Studio 2012, that allows you to see how your metro app behaves on different screen resolutions and vertical/horizontal alignment:

IMG_1002 IMG_1005

Several Windows 8 compatible tablets and laptops/netbooks were shown, but surprisingly enough we haven’t seen a Surface yet.

 

Next we went to DEV337 on Software Testing with Microsoft Test Manager 2012 and Lab Management presented by Brian Keller. The new features available in TFS 2012 are:

  • Improved support for Exploratory Testing using MTM, where the steps are recorded during the test, and when filing a bug you can see the steps you have been doing and edit in these. Also being able to take screenshots/videos of the bug found.
  • Rich text fields in TFS Work Items, e.g. the Description field for a bug. Not a big feature, but will certainly make it easier to include small screenshots when creating a bug work item.
  • Being able to execute tests remotely. In his demo he had a person from the audience testing remotely on a Windows 8 tablet. When a bug was found, the action recording steps, system information etc. from the remote machine was sent to TFS. This is something I’m going to take a closer look at, as it seems it’s not required to install MTM inside lab environments then.

 

Images of an “exploratory bug” (note the rich text capabilities including a screen dump) and the feature matrix for test environments, depending on if you’re using SCVMM or standard environments (physical, VMware etc):

IMG_1013 IMG_1015

 

Next session was DEV370 Automating Server-Based Build, QA & Test with Visual Studio 2012, which I found more to be a 200-level introduction to the build features of TFS 2010 and 2012. We went through:

  • “What is quality? A system that meets quality expectations in one context, might not be acceptable in another context. An example could be different targets for code coverage in different scenarios (usually depending on customer expectations in an Agile project)
  • Definition of done. Must be measurable and automated, and can help keeping a constant level of quality by not pushing test related tasks to the end of project.
  • A checklist of five steps to get higher quality using the built-in TFS features (see image below)

 

According to his checklist (left), I must say that we at ScanJour are in pretty good shape then. The image on the right shows how to use a Layer Diagram to verify that architectural guidelines are satisfied by the code, and is probably something to take a closer look at. Unfortunately it requires Visual Studio Ultimate to set up, but can then be validated on the build server (by applying “/p:ValidateArchitecture=true” as MSBuild Argument)

IMG_1019 IMG_1027

I find the checklist to be a good minimum baseline for anyone involved in software development, but mostly focusing on “building it right” and not necessarily “building the right it”.

 

Channel9 is streaming live from the conference. Quite a number of people involved in a Channel9 production:

 

The I went to workshop WCL02-WRK on Building Your First Windows 8 Metro Style App where I had the opportunity to work with the Release Candidates (or “Release Preview” it seems to be called for some products) of Windows 8 and Visual Studio 2012 Express (yes, there was Express installed on the PC’s). We made a fine little cookbook application with support for Search and Share in Windows 8. “Share” seems to be the new way to drag/drop in Metro apps, now you cannot drag a picture from your Explorer Window into your Mail window.

IMG_1034

As mentioned earlier, the screens in the room were all touch-screens, and I must say it feels really natural to start dragging thing around on the screen using your fingers. Having both mouse, keyboard and touch seems to complement each other really well, depending on the task you are working with. I can surely imagine situations where touch is the preferred way to interact with the PC, especially when several persons are looking at same PC. Then it’s a lot easier to simply touch the screen instead of having to pass the mouse around. We just have to remember to clean our screens more often Smile

 

Last session of the day was DEV390 on IntelliTrace. The speaker started to compare IntelliTrace to the “black box” of airplanes, which gives you the opportunity to find out what went wrong, after something has happened. The second slide here shows what types of code you can expect to get IntelliTrace for:

IMG_1035 IMG_1037

Most of the session used existing features of Visual Studio 2010, which seems to work fine when executed from a developer PC (we never had success running from lab in ScanJour). Visual Studio 2012 introduces “IntelliTrace In Production” (or IntelliTrace Everywhere), where you can download PowerShell scripts that will enable IntelliTrace on any environment, which I think will be a very useful feature in order to repro and fix customer-reported issues:

IMG_1044

 

Kim noticed that all light bulbs (even on slides) are now low-energy Smile

image

 

We ended the day at the Amsterdam ArenA (home field of Ajax Amsterdam) watching the EURO 2012 semifinal between Spain and Portugal. Probably because it’s my first time at TechEd, but I’m really impressed by the work Microsoft puts into this conference.

IMG_1049




TechEd Europe 2012 – Tuesday

clock June 27, 2012 08:33 by author rasmus

I’m participating at TechEd in Amsterdam these days. Nice sunny weather and a fine hotel. Amsterdam seems to be a nice city.

But on to the important stuff, namely what’s happening at TechEd. My first day here (didn’t participate in workshops Monday) started with the usual Keynote session, covering

  • Cloud computing (this is a big topic this year)
  • Scalability of Windows Server 2012 and SCVMM 2012, where number of VM’s per Hyper-V host has increased. Looking forward to this.
  • Automation. According to speakers, every new Microsoft Server application now has PowerShell support. If a UI exists, it leverages the PowerShell cmdlets. PowerShell is definitely something to look closer at as SDET, since this can help you automate many tasks related to configuring system.
  • ScottGu presented Visual Studio 2012, where he created a small mobile app using MVC 4 and an iPhone emulator (installed as part of VS12?). Of course it was a Microsoft keynote demo (which always looks very convincing), but it actually seemed quite easy to develop this app, so MVC 4 is yet another technology to look closer at.
  • OData support out of the box in VS12. Something that will be useful for our customers, now we have our own OData provider for Captia

 

The following figure shows how Microsoft sees the application lifecycle, where applications are continuously developed, improved and deployed based on feedback from customers (although it seems to require a fairly homogenous environment with Windows Server 2012, SCVMM 2012/Azure, Visual Studio 2012 etc. Not convinced yet that it will work behind firewalls, with older Windows versions etc.)

IMG_0965

An example of this was being able to replay a Web Test “recorded” in Visual Studio on a production environment, and using IntelliTrace for debugging any issues found. There certainly are some interesting ideas in this.

Finally the keynote touched the BYOD (bring-your-own-device) challenge that many IT departments are facing. The point here was that it’s not important who owns the device, but rather who is in control if the device. The speaker presented tools that e.g. only would allow a mobile device to connect to company resource, if the device had enabled encryption and password. Another feature was to be able to remotely wipe devices, e.g. in case it’s stolen.

All in all an interesting keynote, but not sure I will watch it again when video is ready.

 

Next session was by FDN02 on Application Lifecycle Management (ALM) by Brian Keller, where he gave an overview of the new features in TFS 2012 and Visual Studio 2012. It certainly seems that agile planning and estimation has become a lot easier in TFS 2012, e.g. by simply drag/dropping work items from backlog into an iteration, and with the following task board build into TFS (left image):

IMG_0967   IMG_0969

The second slide shown here is actually the same as the one from the keynote, but here “decorated” with more specific technologies to be used in the different application lifecycle states. What I found especially interesting here was:

  • Mockups using PowerPoint (new Ribbon tab in PowerPoint). I think this has always been done, but now it’s actually supported by Microsoft Smile
  • Code reviews directly in Visual Studio 2012
  • Code clone analysis for detecting similar code blocks throughout the application
  • IntelliTrace initiated on production environment using PowerShell cmdlets (another example that PowerShell usage is increasing)
  • PreEmptive Analysis which provides functionality similar to Microsoft's own “Dr. Watson” and “Customer Improvement Program” features
  • And finally the hosted TFS available at http://tfspreview.com. When they get Lab Management support here, this will certainly be interesting for us to take a closer look at.

Good session, which is worth looking at when the videos becomes available.

 

Moving on to session AAP305 on Cloud Patterns for Application Architects. Not my primary area, but nevertheless interesting to hear about considerations for developing cloud-based applications. Of course Azure was used in the examples, but the session generally covered patterns learned from existing large systems like Amadeus. The overall architectural pattern is to protect the database from unnecessary load, by implementing stateless servers, so that users can freely move to other servers, and servers can be taken out without problems.

IMG_0978

The “worker process servers” on the right side of the slide, are performing asynchronous jobs, e.g. processing the actual buy, but on different servers than the front-end. An actual “real-life” example of how to minimize database load is Amazon that displays a text like “This item is usually available within 48 hours”, instead of querying for exact number of books in stock. Again this can save database round-trips, which can have a big performance impact on a large scale site.

 

Then went into SIA301 on Techniques for Hiding and Detecting Traces, where the speaker both showed some “bad guy” tricks for getting binaries into a system. Required either physical access to the machine or using social engineering. Either way it was shown how to get admin access (by e.g. just waiting for another admin to log on to the machine), and I can certainly imagine that this is actually a real issue. Various SysInternals tools was used to show how you can detect if something suspicious is running on a machine and also how to hide a process from being displayed in Windows Task Manager.

 

Final session Tuesday was DEV414 - Realizing the LINQ to Everything Dream. Having implemented a LINQ provider for Captia using our “web-enabled query layer” (SOM.ASP for people who knows Captia), it was really interesting to hear how it should have been implemented more correctly Smile. The speaker talked about a toolbox that Microsoft is working on internally that will make it easier to implement custom LINQ providers.

The session started out by talking about Monads, which is a Haskell construct developed years before LINQ, and serves as the theoretical foundation for LINQ.

IMG_0983

An implementation of a LINQ provider for searching movies at IMDB was presented (although rather quickly) and also an example on getting tweets using LINQ (it seems I need to get a Twitter-capable phone soon…)

Another example shown here is how to query WMI for new notepad.exe processes:

IMG_0989

Note the use of AsObservable() that will return an IObservable<> collection, meaning you subscribe to results when they become available, instead of getting the full result-set at once. Especially useful when querying big datasources like Twitter.

In the last part of the presentation, we were shown how to solve simple mathematical/logical tasks and even a Sudoku using LINQ:

IMG_0995

This certainly was “LINQ for Everything” and a good 400-session, that I will recommend others to watch.

 

From my colleague Kim, I hear that the WCL309 – What’s new in Microsoft Deployment Toolkit 2012 session presented interesting new features of this tool, that we might be able to use for automating SCVMM template creation for our TFS Lab Management. From DEV316 – ALM Tools for C++ in VS2012 it sounds that a some really useful features like static code analysis, unit testing and code coverage is coming up with the new version of Visual Studio 2012.

That’s it for today. Tomorrow will be about testing using Lab Management, IntelliTrace and automation. Stay tuned…