The best tests are the worst. I mean that in a good way - they are intelligent, bring together scenarios that produce unexpected behavior or may even break things that were previously thought to be fully available/resilient.
In this day and age of Cloud and DevOps, applying software development best practices to create versioned, reusable, maintainable, extensible, and testable applications is a must. Using the best tools to get your job done makes sense – including IDEs, languages, package managers, and build/test tooling.
In this post, I will focus on the testing portion while addressing it from a creative angle.
Many of us are aware that there are various types of testing we want our application (and infrastructure) to leverage at different stages in its lifecycle - static analysis, unit-testing, integration/end-to-end functional tests, performance, security testing and chaos engineering. Shooting for all or most of the above tests to be automated, is ideal.
In this post I’ll explore and encourage a fun and creative test-driven culture, by sharing with you a unique perspective. Thoughts shared here are especially geared towards creating higher order tests (anything other than unit-tests which usually are written by the developer at the time of code creation).
My challenge to you is to see the application, and the business it supports, in a heightened or skewed way, while being careful not to get too carried away.
A certain amount of divergent thinking is required in understanding and formulating good tests. Even the most bizarre and seemingly unrelated combinations can end up creating the best tests. You might need to draw together things that are typically considered far apart.
Here are a few foundational considerations (I will be giving some examples later in the article):
- Attention is crucial – Take time to notice the weird things in an app. You might have accidental, or serendipitous discoveries, so pay close attention to the systems (layers, microservices, partner services etc.) around you. Maybe you can use them for other purposes than just integration testing, or even combine tests in some unexpected way.
- In any test, you don’t need to explain it. It should be self-explanatory. Make good comments. A good test needs the whole team’s perspective to complete it. The ‘tester’ just begins the work and may need support from some other team member.
- Have some fun in your test lives and lighten the load a little bit - make a smart test to poke fun at your developer, and circumvent architects with some subversive tests.
- The more you know your team and app, the better you can help with these unique tests. When juggling, you are not paying attention to one individual ball. The same goes with testing; the focus should be on all the associations. When you find your pattern, make your test for it – and maybe even break it.
- Every so often have a test competition. Pick a theme for your team. Go around the group – everyone must come up with a test. Making tests in a playful and slightly competitive space gets your team going and could be a lighthearted way to lift the veil of more routine integration/DR tests and make it fun. Consider a scoring system where the more outlandish the test the better. There is a ‘performative’ aspect to doing this as a team that might get the creative juices flowing in everyone.
Here are pointers on some unique/radical/chaos tests you might consider at the infrastructure and application layers:
- Infrastructure layer – all we see at this level are streams of packets with IP addresses and ports. So, maybe you can try to:
- Starve your resources (CPU, memory etc.)
- Change your system state
- Induce traffic delays
- Application layer
- Leverage metadata from the requests and use it to construct context-aware tests.
- Knowing unique identifiers of the customer, device, country etc., you can now use them to create a small, well-defined test, which in turn, allows for much faster feedback and you might discover latent problems sooner.
- If you don’t have access to system level resources like in serverless environments (AWS Lambda, Azure functions), then you will need to have a library that will help you inject tests, or faults, at the code level. Then you can:
- Target an individual user (could be your test user account) and look for signs of problems when logged in, while not disrupting other users.
- Determine that some endpoints may be down while others are fine. In order to simulate such a scenario, you can create a test targeting some endpoints only and then analyze the outcomes.
- Think of limiting a test to set of devices that you control and then have them running all the time. This type of “always-on” testing allows you to run tests against those devices on a regular basis and evaluate how the user experience works when the system is degraded.
All that creativity can only flourish if you have a solid foundation to begin with. Here are a few things to address upfront:
- Getting your test environment set up doesn’t just mean installing a few libraries and you are good to go. Ensure that it can be reproduced and deployed easily in test environments other than your laptop.
- Next, put some thought into the tooling and framework (libraries, suite set up and tear down, project structure etc.) to set you up for success.
Here, I’ve shared a more creative angle to the traditional testing portfolio, by encouraging you to consider and appreciate ideas that are not normally prioritized and to persuade you to combine tests that don’t normally go together. In doing so, you can create more robust test suites and rekindle that simple satisfaction of getting a test right. Consequently, your application and business become more resilient.
There may be all kinds of ‘radical’ tests that you are thinking now. Please feel free to share them with your team and in the comments below.
Cheers!
