The outside-in API testing approach
I’d like to start and say that this was an experiment to see if the process of documenting API’s and actually implementing it to that specification could be made simpler. It is still in progress and highly opinionated for my use case. So I decided to write about it ;)
As a backend and mobile developer, there always seems be a mismatch between the API specification and the ‘agreed-upon’ and versioned set of endpoints that is currently in production/dev and which are new features and how they should work etc etc etc.
I had the opportunity to work alongside another backend developer on a new project and we decided to challenge our understanding of specification tests and at any given point in time be able to safely say that the API is beautifully in sync with the mobile app or web client.
Obviously this sounds kinda complicated and something that does not really exist yet so we made some tooling around it to make our lives easier.
After watching a video from Adam Wathan on TDD Laravel, which completely blew my mind, it challenged us to think about these things in a different way. Instead of trying to keep the test suites per project in sync, try and write up some specs in a centralised place and sync automatically down to the projects.
This seemed like a super cool idea, mainly because developers could sit in the same room, talk through the feature/enhancement/epic and the resources it would require and when all the white-boarding was done, these endpoints could be documented and everyone would get an automated set of test specifications to code towards.
Sounds a bit ambitious but we went ahead and tried it anyway.
Enter Api Blueprint
Api Blueprint is a open specification to define API resources. From their site:
API Blueprint is simple and accessible to everybody involved in the API lifecycle. Its syntax is concise yet expressive. With API Blueprint you can quickly design and prototype APIs to be created or document and test already deployed mission-critical APIs.
What it doesn’t say there is that this specification is technology agnostic. This allows the Android team to sit with the PHP team and talk about the same things without having to write a single line of code.
The format is markdown and it writes beautifully. Here is an example endpoint:
# GET /message + Response 200 (text/plain) Hello World!
Apiary to the rescue
Apiary is a application that implements the Api Blueprint specification and provides the tools to easily write and visualise your API and endpoint resources side by side. Here is what the test tutorial looks like:
What drew us in immediately was the ease of generating the relevant content that would in the end be our full API specification. Also the document generating power so that even the product team could read what is available.
We have found our centralised spec tool and we went on to spend a couple of days talking and white-boarding the needed endpoints, what they would return and also the relevant RESTful error codes and their json data.
The thing about this tool is that it plugs into GitHub and pushes the changes to the
.apib file directly to your repo so that it’s versioned.
This means that without even writing a line of code, the developers will have a full specification to code against with minimal planning and documentation effort from anyone.
Now that we had our specs drawn up, still in markdown format, we needed a way to auto generate a bunch of tests that keeps itself in sync with the latest api blueprint so that an automated build can fail if specifications exist that is not implemented yet.
My next couple of posts will be about a project I call Outline and how it plugs into this workflow of documenting first and then having tests indicate that the API is safe to consume from the client without breaking things. The same applies the other way around, where the client expects a certain version of the API structure be specific.