Testing your (software) solution is of the utmost importance in any trade, but in the financial sector we truly strive for correctness. I assume you’re familiar with unit tests or writing code that has tests. Still to get correct working software, what can you do more? Do I strive for a code coverage of one hundred percent?
In this article, I will show you two methods of testing I use beside unit testing. This doesn’t concern code only. Testing can be also done at an earlier stage. As a bonus, I will try to explain why trying to reach coverage of 100% is a dud.
Gherkin is a language developed for describing scenarios of your system. This can be made functional in Specflow (if you use .NET) or Cucumber (if you use frontend). But what does it bring to the ‘testing’ table?
Gherkin is a structure for describing your soon to be software. Before you start writing any code, sit with your product owner and some colleagues to describe scenarios your code must be able to pass. Let’s say there is a coffee machine which has to brew coffee and tea. It can be describe like this:
Scenario: The customer wants a cup of tea Given The machine has warm water And the machine has tea powder When tea is selected Then a cup is placed And powder is put in cup And warm water is poured Scenario: The customer wants a cup of coffee Given The machine has warm water And the machine has coffee beans When coffee is selected Then a cup is placed And coffee is grinded And warm water is poured
Both scenarios must be handled by the same system and each individual line is a step which calls your actual code. The steps can be shared between scenario’s (for example: the cup is placed and warm water is poured). This makes ‘re-mixing’ steps to new scenarios great. If you want to extend the machine to serve just hot water, you only have to create one step ‘When warm water is selected’.
This way of development is far more behavioral than result based. This makes you think about how the software must behave instead of focusing on the outcome.
Bonus is that by writing Gherkin, also non-technical personnel can read and suggest new scenarios or think about the behavior without being distracted by implementation. This does not only test your code, but also the requirements and assumptions you have.
In the end, I incorporate these tests when doing a lot of Blackbox work with a lot of logical steps. Because Gherkin is testing both code and specification, I even discovered faults in specification. Together with my team I can let Gherkin files be reviewed by not only developers. For instance my colleague Paul is really avid in reading Gherkin from a business perspective and correcting specification faults.
At the other spectrum of Gherkin scripts, are end-to-end tests. This sits on top of all the other tests like unit tests and integration tests. End-to-End, or E2E for short, is used to automatically test your application as it was a real user. This can stem from the scenarios I just discussed but it always goes from the end-user perspective. The environment for E2E tests is always a fully deployed environment like a test server or a containerized version of all your backend, frontend, and infrastructure.
cy.visit('/instruments'); // Act cy.contains('Add instrument').click(); cy.get('#securityId').type('Something'); cy.get('form').submit();
In the above snippet you see that I instruct Cypress to:
- visit /instruments
- When there is something that contains the text ‘Add instrument’, click it
- Find element securityId and start typing the word ‘Something’
- Finish of with submitting the form
This is an easy example but shows a few things. Cypress works in order, but can wait between steps (without implicit calls) to find something on the page (incase it needs loading). Also, I can access the DOM as you would with frontend code. This is not something I would advise, but it is possible. The better solution is to just describe a label and navigate from there.
Here cypress looks for the text and assume the next element of the text label is the input field. This is in my opinion quite elegant and human like, because as a human I also look for a input next to word ‘Security id’ to be the input. Still this can be brittle because UI might change and the next thing might not be a input. So the more technical robust way would be getting it by a DOM element.
E2E is powerful and helps spotting regression when you’ve set it up. It does require a lot of effort and can break easily.
Read more on cypress.io
How to use test coverage
Test coverage is something every manager who knows of loves, because it’s looking like a key-performance-indicator.. but it isn’t
Why? Because not all code is equal. A rough indication:
- Business logic
- Infrastructure code for communicating with outside data
- Factories, mappers and other internal transfer code
- Setup to work with a runtime/framework
- Generated code or external libraries
Some parts are more beneficial to be tested than others. The order I’ve put them is for me the order of importance, so business logic should be tested and covered very well. This can be done with Gherkin. On the bottom is generated code, this is something I almost always excluded because it’s out of our control.
What would you do with coverage results? Well, if you can turn them into a map or word cloud like the one below with SonarQube, it’s a great tool to discover what to test next. The quadrant on top right is where I tend to look, with a mouse over SonarQube shows me the file as a circle with least coverage and highest cyclomatic complexity.
Testing isn’t only about unit tests and mindless upping coverage. Testing approach should vary, for various parts of your code and purpose. I like doing all of the discussed items in various parts of development. Because when my code is tested, I feel more relaxed at the end of the day.