Rasmus Møller Selsmark

On software test and test automation

GOTO Aarhus 2012 at a glance

clock October 4, 2012 23:32 by author rasmus

The GOTO Aarhus 2012  conference is over, and I can look back at some really good and inspiring days where I had the pleasure of hearing speakers like Martin Fowler talking about NoSQL databases.


As the title says, it was an introduction to NoSQL databases, but with references back to relational databases and even network databases, which was the predecessor to the relational databases. I’m old enough to have been taught about network databases in university, but nevertheless really interesting to see how a computer science field like databases is evolving. As with many of the other speakers at the conference, Martin Fowler did a very good job of keeping the right balance between theory and practice.

Scott Hanselman was talking about how the browser and the Web has evolved so it’s now possible to even run a Commodore 64 or Linux emulator using JavaScript. As he said, “JavaScript is The Assembler Language of the Web”:


This was followed up Anders Hejlsberg and some of his team members from Microsoft, when they presented TypeScript, which is a new type-safe language with classes that compiles into JavaScript. Of course the TypeScript compiler is written in JavaScript Smile (Which is quite useful, as it then easily can run on different operating systems)



Michael T. Nygaard and several others had some really good sessions on Continuous Deployment. Michael talked about the concept of “deployment army”. Think we all know of situations where a deployment have required special skills.


One of the purposes of Continuous Deployment is to eliminate the need for big releases, by building up confidence around the software product, so it really always is in a releasable state. Then you can decide to release it or not, depending on your type of software and customers. And this is only a few of all the speakers I’ve heard in the last three days (actually the ones where I had some decent pictures...)


As said, it has been some fantastic days at the GOTO conference. The conference has been able to find just about the right balance between theory and practice in the sessions. Only very few sessions have been either too theoretical or practical oriented. Also when comparing this conference to Microsoft TechEd, which I attended earlier this year, I find the GOTO conference to have a much higher level, both when it comes to speakers and topics.

So if you are considering only one conference for the next year (and GOTO is able to keep the high professional level of speakers and topics) I will definitively recommend that you consider attending the GOTO conference. At least I certainly hope that I can be part of it again next year.

GOTO Aarhus 2012 - Day 2: What, no testers?!

clock October 2, 2012 22:35 by author rasmus

The good thing, for me as a tester, by attending a developer conference like GOTO, is that I get a different perspective on my profession, which I probably wouldn’t have gotten by attending a usual test conference. Today I saw a couple of sessions on Continuous Deployment, most remarkably a presentation by http://www.etsy.com/ in which it was shown how they do 30 deployments to production daily.

A common anti-pattern in relation to deployment, is the concept of a “deployment army” as presented by Michael T. Nygard in his presentation, where as a result of having few deployments, each release is so large that you need an “army” of people to deploy it. This gives a high cost on deployment, so you cannot afford to increase the number of deployments, and the result is a kind of dead-locked situation that you need to get out of. Continuous Deployment may solve this, as soon as you have accepted that you have a problem.

The benefits of doing Continuous Deployment is that the software development team gets almost immediate feedback on deployed changes, without having to wait for some time before releasing into production. Below is an example from Etsy, where they after a release noticed an increase in the number of warnings (the vertical red line at approx. 16:05). Within 10-15 minutes they reacted to the situation and was able to release a new version which fixed most of it, and 30 minutes after the incident, the issue was solved completely. Of course we want to avoid such issues, but when they happen, it’s important that the organization can react fast. So the benefits are basically that we get really fast and relevant feedback to changes to code. I’ll get back to what implications this has to an organization.

As stated by several speakers today, this of course requires the architecture to be prepared for this way of deployment, especially not having one big monolithic system, but being able to deploy individual components of the system. For the people being skeptical to how Continuous Deployment can actually work, Jez Humble mentioned in his presentation that when Flickr was acquired by Yahoo in 2005, they compared Flickr (which were deploying continuously to production) against all other Yahoo services, and it turned out that Flickr had the highest uptime. So Continuous Deployment certainly can work for high uptime sites, and after hearing the presentations today, it is certainly the case.

But lets take a look at a deployment process in general:

(NFR Test: Non-functional requirements testing)

This looks very much similar to a deployment cycle that I’m used to, although having some of the steps being manual or partly manual in my situation. In Continuous Deployment this must be performed several times per day, which leaves very limited time (actually none) for any manual task involved. So moving towards Continuous Deployment means that all of the processes, including test, must be automated. As it’s the developers that are monitoring the production site (performance, errors etc.) there are even no dedicated system operators involved. The DevOps (developer/operations) role is a part of the individual development team.

To follow up on the title of this blog post, I got a couple of minutes to talk with Michael T. Nygard (independent consultant) and Mike Brittain (Etsy), about the role of test automation. The very interesting answer from them was that they don’t have dedicated test automation engineers in Etsy and Michael said that he doesn’t see this role in new organizations, but rather see a role of SRE (software reliability engineer) around occasionally. Amazon is also present at the conference with a booth, which basically seems to be a recruitment campaign among Danish developers. I took a look at their job postings, and it turns out that they have no test automation engineer jobs either. I heard from others that Netflix (a TV/video streaming site) in their presentation also mentioned that they have a very limited number of testers.

Conclusion on state of test automation in the industry

I started out my series of GOTO blog posts at http://rasmus.selsmark.dk/post/2012/08/27/GOTO-Aarhus-2012-Is-it-time-for-developers-to-move-beyond-unit-tests.aspx by asking the question

“Is it common to have automation testers in the industry (like my current job) or is automation a part of usual software development activities?”

After today, I must say that the conclusion is that it’s not common to have dedicated testers (at least not in newer companies) and in order to do Continuous Deployment fully, dedicated testers becomes a bottleneck, and therefore this role does not exists in such an organization. And in the case of Etsy, they even have very few (<5) people doing manual exploratory testing out of a total of 350 employees. So the game is certainly changing for testers, if you move towards more often releases. I don’t feel concerned as such, as I’m sure any good tester has valuable domain knowledge, which can be used elsewhere in the organization, but we should be aware that the world is changing for us.

GOTO Aarhus 2012 - Day 1: New infrastructure creates new types of companies

clock October 1, 2012 23:12 by author rasmus

Mondays keynote at the GOTO Aarhus conference this year was held by Rick Falkvinge who is leader of the Swedish Pirate Party, a political party working for free information and a free Internet. Title of the keynote was “Red flags on the internet”, which refers to the “Red Flag Act” of 1865, where a law was introduced in United Kingdom, which required that a person should walk in front of every car waving with a red flag, to warn pedestrians.


It turned out that this law was lobbied by the British Railways in order to secure their interests. But the result was that Germany this way got 20 years advantage for their automobile industry, so it ended by hurting Britain more than it helped. Rick went on to other examples from the history, ending with Kodak, who actually invented the digital camera back in 1976, but because their income depended heavily on their analog film products, they didn’t develop this new digital technology further and eventually went bankrupt in January 2012. So the point here is that it doesn’t help protecting/hiding information and inventions, as it will at at some point emerge anyways.

Even though the Swedish Pirate Party is currently mostly a protest party, he drew lines back to the Green politics 40 years ago, which is now a common part of the official program for all political parties. He didn't give a good answer to how we can actually still get paid if all information is freely available, but I still find it interesting to see how history has shown that information and new technologies cannot be hidden away.

New infrastructure introduces new types of companies

With the shift from e.g. water mills to electricity, it was no longer necessary to build factories close to the energy source, but could rather be built where the actual need was. Unfortunately I didn’t get an image of the actual slide, but I do find it thoughtful this similarity with how we are still constrained today, e.g. of where companies can be built, so I think we need an image of a good old watermill-powered factory from Wikipedia. I’ll get back to this shortly.


RubyMotion - a company of the Internet age

This brings me on to Laurent Sansonetti, founder of RubyMotion which is a platform for developing Ruby-based applications on e.g. iPhone. I was so fortunate to have a talk with Laurent about his company.


Laurent have worked as developer for several large companies, including 7 years at Apple, before he decided to work on his own open-source project RubyMotion. Moving from a secure job at Apple and starting his own, did result in a couple of very insecure months with different challenges, and sometimes fear that the product would never made it. But it went well, and they now have several large customers and are making money. Congratulations, as it’s good to see that there is always room for new innovative products.

The structure of the company is what Laurent calls a “distributed startup”, meaning that the current three employees are spread across the entire world (Belgium, United States and Japan) and don’t have any office yet. They work from where they decide, more or less at the time they decide and communicate using CampFire, which is a group-based chat system, where they can have ongoing discussions, and see what has been talked about while they were gone. Because being spread across the world this way, means that the company is more or less “open” always. Then they meet every third month for a week physically, in order to actually meet and have face-to-face meetings.

Other advantages of being distributed this way, is that you actually get easier access to talent, as they don’t need to relocate in order to work for you, and perhaps otherwise wouldn’t be able to take the job. As Laurent said, it’s hard to find good programmers and even harder to have them relocate.

I think this is a good example of a company that leverages the possibilities of the internet age, to build a startup company that a few years ago would have been difficult to imagine. Especially it’s worth noting that with only 3 people they are able to keep the company running most of the day. Like back when factories had to built close to an energy source, I still think that (IT) companies of today are constrained to some extend of having access to talents or other kind of infrastructure. Although I actually like going to the office (working zone) every day, I think it’s worth paying attention to the possibilities that are available for starting up a new company with very little “deadweight” like physical building, network, servers etc., when all of this can be distributed/hosted on the net.