Why API's are like take-away menus

I recently had to give a presentation to my test team at work with about testing APIs. The team covers a wide range of experience levels and with varying technical knowledge so I wanted to try describe API's in a way that everyone would understand. The presentation I gave was well received so I wanted to share an excerpt of it here.

I've seen quite a few non-technical people look worried just at the mention of the word API. Possibly due to not understanding what and API actually is.

API's aren't scary. They are just made up of questions and replies.

API stands for Application Programming Interface but this doesn't tend to mean a lot to non-technical people.

At a very basic level, the term API is used to describe both sides of computer systems that talk to each other using language which is pre-agreed and understood. Software can make requests using APIs in order to receive responses and this is why API's are like take-away menus

Imagine the menu for your favourite take-away restaurant......

That menu is like an API between a customer and the restaurant. It is something that lets both side communicate with each other.

When a customer phones the take-away and places an order from the menu, imagine they are software that is making a request to an API. When the restaurant delivers the food, imagine this is like the customer receiving a response from the API.

The takeaway menu does not include the recipes and instructions on how to cook the food. The customer does not need to know how to make each dish (because the restaurant takes care of this). The menu simply lets the customer select the food they want to eat from the choices provided by the restaurant.

The recipes and instructions on how to cook the food would be like source code. APIs make it possible for software to share information and interact with each other without having to share all their code.

If the restaurant changes their recipes or ingredients, the menu will still work and the customer will still be able to order.

So if the source code changes for a piece of software, the API will still work.

If each dish on the menu is numbered (like at a Chinese take-away) and the restaurant changes the numbers, a customer with an old menu ordering using the old numbers will receive the wrong meal!

Changing the numbers on the menu is like changing an API which is already in use. This should be avoided if at all possible.

The Name, Shame and Flame API Vulnerability game.

Its now 2015 and we're all living in the future! Our world has become a place where invisible intangible things (like APIs) have become rather important in our day-to-day lives.

This morning while staring at my PC with sleepy eyes, details of a security vulnerability at www.moonpig.com popped up in one of my social media feeds linking to this article. The post itself was fairly negative towards Moonpig, along the lines of "I will never use their service again because they fail at security. How dare they fail to protect my details!".

However, reading the article I became especially interested in the story for two reasons. Firstly, this was an API flaw and I do a lot of integration testing involving APIs in my day job. We all know making APIs and using APIs is hard. Secondly, the article claimed Moonpig allegedly hadn't bothered to fix their API yet!

So I did what any good software tester would do, I attempted to recreate the bug.

Currently, my weapon of choice for querying APIs is Google's Postman client plugin for Chrome. I like Postman because it can query both SOAP and Restful APIs. It's also really easy to save and share API collections. Authenticating with APIs is fairly straight forward in Postman too, but from reading the article I wouldn't have to do that.

I still couldn't believe Moonpig's API had absolutely no authentication on it whatsoever!?! I decided to press on regardless. If I could experience a dodgy API first hand it would make it easier for me to identify similar dodgy APIs in the future.

Fortunately for Moonpig (and more importantly Moonpig's customers) by the time I was able to try a query, around 11 hours after the details of the vulnerability had been disclosed to the internet, their API was completely and utterly dead. It was no longer spitting out customer's details. It was totally unresponsive and returning status code of 0. RIP api.moonpig.com

I honestly don't think Moonpig would have fixed this problem had details of the vulnerability hadn't been shared online. The author of the original blog post states he told Moonpig about this problem back in 2013! As harsh as it may sound, in these circumstances I think sharing the vulnerability was definitely the right thing to do as it has forced Moonpig to take corrective action - even if that corrective action appears to be pulling the plug on the API rather than fixing it. Companies like Moonpig have a responsibility to look after our data!

And now the damage is done with major newspapers picking up on the story and Moonpig's reputation torn to shreds. I think the saddest part of the story is Moonpig ignoring such a bad bug for such a long time.

Selenium adventures - How to automate finding pictures of cute cats.

Blah blah automation. Blah blah Selenium. Apparently automated testing is what all the cool kids are doing these days. I'm not naive enough to believe that automated testing is some kind of magic spell that when cast the software will test itself and suddenly reveal the location of all the bugs. But, having experienced first hand the pain of long drawn out manual regression testing, if ANYTHING helps ease even a small amount of that pain - I want to know about it!

A few months ago I started wondering what Selenium is and how can I find out more about this automated testing magic. After a bit of searching I realised I wasn't the only one looking for answers. A lot of other people were already asking similar questions. I started my search on Quora. I like Quora, its familiar. I frequently read Quora on the train while commuting to pass the time.

Quora directed me to the Selenium Webdriver documentation so I switched from reading Quora on my daily commute to reading Selenium documentation instead. It introduced a lot of new concepts like the difference between asserting, verifying and crazy things called Xpaths. I was still pretty baffled even after reading the documentation. I needed tutorials.

I found some videos on Youtube of an Indian sounding guy explaining Selenium using Java. I tried to copy what he was doing as best I could. Eclipse (the IDE he was using on his videos) was horrid. I spent so much time working out how to "set up" Eclipse to get to the point where it would just open a web browser on its own. Eclipse forced me to learn what environment variables are and how to set them in Windows.

I persevered though the frustration and was eventually able to write this.

My first ever Selenium script. It opened a browser, went to Google and searched for cute cats. I like cats.

By this point I was aware that Selenium Webdriver could be powered by lots of different programming languages. Now the thing is, I'm not a programmer. Selenium needs a language and I wanted to be sure that I chose wisely before filling my head with copious amounts of this kind of nonsense.

The next language I decided to investigate was PHP. I chose to look at PHP because I test software written in PHP for my day job. The first obstacle I encountered was that PHP is a server side scripting language and I didn't have a server. Luckily I share the bus to the station every day with two PHP developers. They enlightened me and told me all about XAMPP. After a bit of faffing around I had a server. After a bit more faffing around I had written this.

General observations between PHP and Java were that the PHP was faster to create than Java due to typing it into notepad++ and not having to fight against Eclipse. With Java the output from the test was shown in real time, with PHP the output was only shown once the test had finished. PHP needed a server to run, Java didn't. Java compile errors might as well have been written in alien. Getting Java to compile was especially painful. PHP was ever so slightly easier to understand why things were broken.

Python was next on my list but as I have discovered so far, one does not simply start writing selenium scripts. I had to get my head around Python first. I had managed to download something called Python and install it. It was on my computer somewhere but had no idea how to even start using it. So I started reading this book called Invent your own computer games with Python. My general interest levels in games and gaming are much higher than non-gaming related topics. I think this really helped hold my attention. I would start copying Python from the book to make hangman or noughts and crosses. Then messing around and adding my own extra bits to see what they did.

At one point I wrote code that had a bug in it. I couldn't see the bug in the code as I didn't really know what I was looking at. But I could sure as hell see it when I tested what I'd written. Was able to eventually work out that four white spaces were in the wrong place which had made all the logic go wonky. I hadn't really realised until that point just how fragile code can be and how easy it is to accidently write bugs.

It took a bit more Googling to work out how to install the Selenium bindings for Python. Having never really had any need to type things into a command line before that took a bit of working out too. But by trial, error and possibly a small amount of luck. I was able to tell Python "python -m ensurepip --upgrade" and "python -m pip install selenium" About an hour later after staring at various error messages and pleading with Google for help. I managed to write my first ever Selenium script in Python and get it to run.

I still have a long way to go on my Selenium adventure but I'm in a better place now than I was a few months ago.

I think subconsciously, I've already decided that I want to continue learning Python. It held my attention much longer than Java or PHP. I especially like the lack of semi-colons at the end of every line. I think the more Python I learn, the more Selenium will fall into place. I just have to keep chipping away slowly at online tutorials until it starts to look like an Elephant.

When software goes bad, it can go very, very, very bad

I saw this article recently about how a website owned by a company called eDreams nearly charged a lady £23 billion for a return flight.

Closer inspection revealed that it wasn't the flight that caused the problem, it was actually the return baggage check-in cost.

How could such a massive error make it unspotted onto the eDreams website?

Maybe they didn't have any automated tests that could verify the return baggage cost was far to large.

Maybe the bug was an edge case that only occurred under very specific circumstances, which this customer accidently stumbled upon.

Maybe the testers were outsourced and just didn't care.

Maybe the testers didn't have control of their staging environment and were trying to test a moving target as their developers kept updating it every 20 minutes.

Maybe they didn't have a staging environment at all.

Maybe the testers were not given enough time to test.

Maybe their testing department found the bug, but didn't have the power to stop the build from being released.

Maybe the data the testers tested with was different to live data.

Maybe the customer followed a different path through the software to the testers.

Maybe none of the testers were looking at the return baggage check in cost, because that area had not changed recently.

Maybe the website simply wasn't tested at all.

There are potentially hundreds of reasons this bug could have been missed.

What damage has the bug caused? I would say massive damage to their company reputation. The £23 billion ticket made headline news! As a customer I would be wary of buying anything from their website. As a software tester, this does not sound like a company I would EVER want to work for.

About a month after the appearance of this bug on their website, British Airways and Iberia decided to withdraw their fares from three of their websites. This, in turn, wiped 59% off the e-dreams share value. How long now until this company folds?

When software goes bad, it can go very, very, very bad. Some people don't realise just how bad this can be. It's doubtful that there will be a happy ending for eDreams. Their story is a true software testing nightmare.

"Why the problems occurred?" See if you can guess!

This article was shared with me today. I especially loved the addition of the speech bubbles to stock clip-art. I found it very special and utterly hilarious. It really made me giggle!

See if you can guess the answer to this one....

Tips for staying happy and sane while testing software - Tip #2

Don't lie, ever. Not even a little lie, not even once. Software does not lie. Software will not cover up for you. Testers that lie, ALWAYS get bitten in the ass.

When I worked with small teams of testers sometimes a tester would get through their work for the day far too easily, far too efficiently, ask no questions and all of their tests would mysteriously pass. As a team lead, situations like this scared me a lot. It was my head on the block if tests were skipped and bugs missed, not theirs. I used to protect myself by covertly stealth testing random samples of suspicious "too good to be true" looking work. Liars frequently didn't get offered any more work.

As a tester I feel that it is essential to build a reputation for speaking the truth, the whole truth and nothing but the truth about the software in test. By telling the truth all the time about the software's behaviour a large team can start to spot patterns in the big picture together. I've seen really good things happen when large test teams start sharing honest observations between themselves. Tester says something like "I think I just saw something strange. I'm not sure why and I can't force it to happen again. I'm certain I saw it though." and then another tester pipes up "Yeah, I saw something similar on Wednesday. What were you doing when it happened?" and before you know it they have worked together to totally nail the bug and now they both can reproduce the issue on demand.

I once saw a guy sacked for passing a large set of test cases over the space of a week, all of which required a flat bed scanner. He was sacked because the company didn't own a flat bed scanner.

Speak the truth, don't lie.

Why new features are a bit like rainbows (and how to find solid gold bugs at the end of them).

I've been doing a fair bit of thinking recently about testing new features in software. New features are special. They are all shiny and new which means its very unlikely they will have been tested before. It can be insanely difficult to measure something new, especially if there is nothing similar to which it can be measured to against. Over the years I have seen many new ideas and designs translated into software and many test teams trying their best to test them. I would say that the number one cause of friction between departments, rivers of tester tears and silly quantities of overtime is adding new features to software.

So why is it so hard testing something new? Lets try understand the testing process for a new feature. Testing a new feature generally involves some kind of document that describes the feature and tells everyone what it should do. At a very basic level most people that don't test (and even some newbie testers) will imagine testing a new feature as something like this.

The picture above makes testing a new feature look easy. Compare the software against the description of what it should do. Look at the good bits and raise bugs for everything which is bad. But something incredibly important is missing from this picture. As testers, we must keep in mind that the feature description is written by human(s). Writing a feature description that is all encompassing that explains every microscopic detail and is 100% flawless is just as impossible as testing software in an all encompassing way covering infinite amount of paths in microscopic detail. It can't be done.

So, as testers, we need to give consideration to the designers intention. Once we start thinking about the designers intentions as well as what feature description says and what the software actually does. The picture now starts to looks a bit like this...

Seven distinct possibilities have been identified when testing the new feature. I've numbered these outcomes on the picture above.

Lets imagine we have a new feature. This feature is a clickable button in the software labelled 'P'. When the tester starts testing they end up in one of the coloured blobs on the picture. This is where things can start to get a little confusing so please forgive me but I'm going to colour my text to match the blobs in the picture.

1) The designer does not intend this button to print when clicked. The feature description says the button should print when clicked. When the tester clicks the button, it doesn't print.

2) The designer does not intend this button to print when clicked. The feature description says nothing about the button printing. When the tester clicks the button, it prints.

3) The designer intends the button to print, but doesn't say so in the feature description. When the tester clicks the button, it doesn't print.

4) The designer does not intend the button to print, but the feature description says it should print. When the tester clicks the button, it prints.

5) The designer intends the button to print, says so in the feature description but when the tester clicks the button it does not print.

6) The designer intends the button to print but does not say so in the feature description. When the tester clicks the button, it prints.

7) The designer intends the button to print. The feature description says the button should print when it's clicked. The tester clicks the button and it prints.

Straight away, we've identified there is a lot more going on in the testing process than just comparing software to a feature description.

So what can happen in all these different circumstances? Hmmmm, my guess would be something like this..

1) Tester raises bug, the designer may see the bug and close it straight away. (chance of tester tears as tester thinks "the designers are closing my bugs for no reason")

2) Tester must question the designer "is this button supposed to print?" some lesser testers may fail to ask this question (chance of friction as designer thinks "did the tester not read the feature description properly?" regardless of whether or not feature description contained the answer)

3) Tester doesn't see a problem, nothing in the description indicates a problem (chance of friction with additional tester tears as only the designer can see the bug. The designer may change the description in the middle of testing, or start raising their own bugs. When designers raise bugs sometimes they forget to mention critical information and testers don't take ownership of them. They can slip through the normal process net and slowly rot in a bug the database as no-one really knows what to do with them.)

4) Unless the test team has developed psychic powers, only the designer will see the bug. (chance of friction and tester tears. The worst scenario is that someone blames the test team for missing a bug they had no chance of ever seeing)

5) Bug raised, no further questions required. Sadness is deflected from the test team on to the programmers instead.

6) Bug raised that says the software does not match feature description (chance of friction when designer tells tester their bug is "as designed")

7) Everyone is happy!

When you look at the big picture and see that only 1 of 7 possible outcomes results in happiness for testers, designers and programmers. You start to realise that there are a lot of things that can make the team sad. Maybe if more people were aware of what was actually happening they would be able to avoid some of the associated problems.

I know when I test a new feature, the testing I do takes everything mentioned above into consideration then something special happens. The special bit happens in my head when I start applying my imagination, creativity, logic and common-sense. I suddenly start seeing bugs that no-one else can see! If you were to ask most people to describe a rainbow, they would say that a rainbow is red, orange, yellow, green, blue, indigo and violet. They describe what they can see with their eyes. However a rainbow is more than just visible light. It has infra-red at one end and ultra-violet at the other and you can't see these bits of the rainbow with just eyes. Our picture of testing a new feature now looks something like this...

Let me give you some examples of some things that happen when I test.

I'm looking at something right in the middle (Number 7 - designer intends it to happen, feature description describes it happening and it happens in the software) however this "feature" is making the software flash in such a way it's making my eyes hurt and I think we could be sued for inducing epileptic seizures.

I see the button is green text on a red background and think this won't be a fun experience for someone with colour blindness.

I see the Japanese version of the software contains an image of a Hong Kong flag and think people in Japan won't like this. Just as a confederate flag would be bad in the American version or a Swastika bad in the European version.

I see a pink elephant's trunk in a children's game that is HIGHLY inappropriate and think no way will this get past PEGI/ESRB or any other age ratings board of classification.

In a browser based game I think I might be able to skip from level 1 to level 100 if I start changing numbers in a the URL query string.

And this my friends is the secret to finding solid gold bugs at the end of a new feature rainbow. If you can learn to think independently about the thing that you're testing, you will start seeing the infra-red and ultra-violet bugs that no-one else can see or imagine.