K.I.S.S.

 Keep It Simple Silly

I started working on a project that had been under development for 6 months. There was unit testing in the code base but there was also a framework they had created which was independent of the production code base.

The idea of the independent framework was that the subject matter experts (SMEs) would generate data to test the application. They would think about all the different scenarios they would encounter in their job, before the software existed.

The independent test framework was great at confirming things were working. However when it failed it was very difficult to debug. The inability to maintaining a framework is the number one reason automation fails. So when things started failing, someone had to step through the framework, figure out the input data and reproduce the scenario in the production code. This way the developer could step through the failing scenario to understand what was going wrong. Without using the integrated development environment (IDE), it was impossible for the developer to figure out what was going wrong.

The scenario the developer created in the IDE was to create the data as setup variables then run the data through the application in debug mode. There was a huge amount of data that was very complex and difficult to set up. Occasionally the setup was wrong because the developer was not a SME. So there was a lot of back and forth with the SME to track down what was really going on.

My thought was, we have the data from the SME in JSON form and reference data was in spreadsheets. Rather than pulling all the data out of the JSON and into the unit testing framework, couldn't we just read the JSON and spreadsheets into a unit test?

The short answer was yes. It took a little while to create the new framework but once we did this for one scenario, doing it for subsequent scenarios was just dropping the SME data into the test section of the application. If it failed, the developer had the test data with the application code in the IDE and debugging it was a lot easier.

Bottom line, keep it simple, silly.