Thoughts on “The ‘A’ Word”

Alan Page is known as one of the authors of “How We Test Software at Microsoft”. But there is another good book and it’s called “The ‘A’ Word”. You can buy it on LeanPub.

The book is about automation in testing, but not how to do it — it’s about how to think about it. It’s short, just 58 pages, but very dense with ideas and Alan’s opinions.

As I am not qualified to give an opinion on automation topics (because I don’t have much experience with it), I’ve just gathered some notes for future referencing. Sections are divided by chapters.


Sometimes, testers use programming skills to help their testing. Sometimes, that code automates some application functionality. That’s it.

Testing: Failing to Succeed

There is a very famous concept called “Orders of Ignorance” introduced by Phillip Glen Armour (more here). Chapter’s idea is that mostly tests are done on 0OI level, but we should never forget about 2OI test. 0OI is a lack of ignorance (I know) and 2OI is a lack of awareness (I don’t know what I don’t know).

0OI tests are knowledge-proving tests, while 2OI tests are knowledge-acquiring tests.

The Robots are Taking Over

Humans fail when they don’t use automation to solve problems impossible or impractical for manual efforts.

Automation fails when it tries to do or verify something that’s more suited for a human evaluation.

To Automate …?

Good testers test first — or at the very least they think of tests first.

Great testers first think about how they’re going to approach a testing problem, then figure out what’s suitable for automation, and what’s not suitable.

You should automate 100% of the tests that should be automated

Alan’s heuristic when to automate: “I’m Bored”

The Coding Tester

Summary:

GUI Shmooey

For 95% of all software applications, directly automating the GUI is a waste of time.

For the record, I typed 99% above first, then chickened out.

Design for GUI Automation

Alan’s main points for disliking GUI automation:

I love GUI automation that can automatically explore variations of a GUI based task flow.

I like GUI automation is in stress or performance issues.

It’s (probably) a Design Problem

In the Middle

Alan’s brainstorming technique: first spend a reasonable amount of time focusing on the extremes — because often, some great ideas for “the middle” comes out of that brainstorming.

Test Design for Automation

The first step — and most important — is to think how you’re going to test.

From that initial test design effort, you can deduce what aspects of testing could be accomplished more efficiently with automation (and without).

Orchestrating Test Automation

Designing good tests is one of the hardest tasks in software development.

LOLUR AUTOMASHUN SUCKZ!

Your tests don’t suck:

Musings on Test Design

Some tests can only be run via some sort of test automation.

Some tests can only be done via human interaction.

You can’t effectively think about automated testing separately from human testing.

In my world, there are no such things as automated testing, exploratory testing, manual testing, etc.

There is only testing

Beyond Regression Tests & Testing with Code

Useful tests are tests that provide new information.

An automation strategy that only performs regression testing is short-sighted and incomplete.

How to make test useful:

More on Test Design

Test Design ideas are endless.

To be a good test designer (and tester), you need a lot of testing ideas, and you need to know how and when to apply them.