Climbing the Testing Pyramid

Added to Process

This interview is in Spanish. However, you can read a summary in English below.

I had the great pleasure of interviewing Natalia Lehmann, an Argentine software developer who has been working for Grupo Esfera for more than a decade. Natalia is passionate about the technical side of Agility, especially the automated tests that lead to developing higher quality software. Precisely she chose to share her practical experiences on how she has been doing development guided by acceptance tests.

Natalia began by talking about the tension that exists between the automation of acceptance tests and the technical difficulties to implement them on the graphical interface and make them run in a reasonable time. She also commented that she and her co-workers are defenders of the value of automating acceptance tests, since they have seen its benefit since these are fundamental to guarantee that progress can be made without breaking previous functionality and improving the quality of a system.

Natalia mentioned that testing manually is a repetitive process that invites mistakes at some point. Automatic tests help to establish agreements with users as they are defined in a readable text language that both developers and stakeholders understand. Acceptance tests under this format are executable and constitute part of the living documentation of a system. 

Doing ATDD (Acceptance Test Driven Development), she said, helps to focus on what is really important in addition to helping to identify what the main flow is. In contrast, putting aside the writing of automated tests is causing the system to be left with points that cannot be regressed automatically and therefore the need for manual tests is introduced. 

Natalia showed the Testing Pyramid which was introduced by authors such as Martin Fowler and Mike Cohn. In short, the pyramid states that we should have many unit tests, many service or integration tests, and some graphical user interface tests. This pyramid proves from the bottom up that integration actually occurs. One other detail is that the unit tests run very fast compared to the graphical interface. Unit tests run on allied components, better identify faults, and are repeatable; therefore, they are very valuable, but they do not detect integration problems. 

The tests on the graphical interface are very slow and not very reliable since they sometimes give false positives. The other problem they bring with them is the effort that must be devoted to their maintainability. An important factor is that these tests are much faster than a real user interacting with the system. Their level of precision is also questionable since they cover complete workflows, and it is often difficult to locate the error. 

Natalia went over alternatives that they used (mock objects) in order to speed up the run of graphical interface tests. Having repeatable results was the other challenge they faced, testing against APIs (Application Programming Interfaces) or underlying models were alternatives that were also tried. 

Natalia said that the composition of the teams she worked with are developers, without professionals who do manual tests; therefore, the profile of the members of their team has contributed greatly to automate the vast majority of the tests of the systems they build. 

To close Natalia suggested looking at tools like Cucumber that help with acceptance testing. And on a final note, she said that despite the technical problems it may cause, test-driven development is a path worth walking.

About the Author

Juan es un capacitador, expositor y pensador alternativo. Desde que Juan se expuso a Scrum a principios del 2007 se comprometió a continuar aprendiendo y aplicando Scrum en los equipos y organizaciones donde trabajo. Su camino lo ha puesto en los roles de ScrumMaster, Scrum Trainer, y Product Owner. Juan cumplió el 2014 con todos los requisitos del Scrum Alliance para ser un Certified Scrum Trainer® (CST) y es ademas un LeSS Friendly Scrum Trainer.

Como CST y LeSS Friendly Scrum Trainer Juan ha entrenado a más de cuatro mil trescientos estudiantes en cursos de CLB, CSM, CSPO. A-CSM y A-CSPO en diez países del continente americano. Juan también a dado cursos privados para compañías como: Citibanamex, Marsh, Slalom, Deloitte, Walmart, CGI, SAIC, Express Scripts, T.Rowe Price, Time Warner Cable, ViaSat, Garmin, Moffitt, Kyva Systems, Blue Book Network, Insurance Auto Auctions and BlueCross BlueShield.

Juan fue miembro voluntario del Board de Directores del Agile Alliance donde sirvió por dos periodos consecutivos de tres años hasta el 2019.

This is an Agile Alliance community blog post. Opinions represented are personal and belong solely to the author. They do not represent opinion or policy of Agile Alliance.

Agile Connect

Agile Alliance Learn

Agile Alliance Share

Stay Up-to-Date!

Get updates on Agile events, programs, and more by subscribing to the Agile Alliance Newsletter.

BYOC Member Lean Coffee

Recent Posts

Agile Coaching Network Signup