“When I talk to my colleagues and tell them that we have a team of developers that are performing manual testing guided by a test-plan nobody believes me… ”
Let me just start by saying that I’m very proud of our team and how far we have come.
Since my arrival, we have tripled in size and have significantly improved our quality standards. When I Joined Testim, we were a small R&D team based in Israeli. We had no QA department or defined testing process and had to implement it all from scratch. There are many challenges in trying to create a process where testing is integrated into our CI/CD workflows. For one, there is awareness. The process of assimilating the Idea that we have to test our produced features and acknowledging there are consequences if we don’t. We have a very talented and speed oriented Dev team. But, it comes to a point when you have to stop and take a step back to really plan for the future. As we strive for continuous improvement, we have to periodically evaluate our process from end to end, making sometimes small adjustments which can improve the flow holistically. We had to assimilate the idea that it is better to “waste” a week of testing than 2 weeks of refactoring. Or worse – to lose our customers trust in our product.
I’m very happy to say that one of the things I love most about our team – is that we do not have inner battles. (At least not anymore). We truly understand that we are all working for the same cause and moving as a team to get the job done.
Some history…
I started at Testim as a one-man show. I arrived with a background in quality assurance and solid automation experience working for larger companies. The startup environment was new to me. I took the first month to learn the culture, build rapport with the people, understand the customers and their objectives, and observe the various internal and external process end to end. From these learnings, I could conclude which things needed to be changed, what was good and should be preserved, and which tools or methodologies could suit our needs the best way.
For example, the product-dev-testing workflow needed some tuning. With that said, what suits an enterprise would be terrible overhead for a fast-paced startup. Typically, a startup requires a lot of risk analysis, fine-tuning and wise tool selection. We needed flexible tools that were easy to use and configure as our processes would quickly evolve over time. We did not have time for lengthy implementations which involved consultants or systems integrators, we needed to deliver value instantly without any setbacks.
I see a lot of people being recruited to a small startup coming from a big enterprise, and without fully evaluating their surroundings, start to incorporate heavy test management, reporting, and bug tracking tools just because “This is the way it should be done.” Frequently they will encounter resistance from the team because they simply do not see the need, or it creates an unnecessary burden on the routine without really delivering added value to the current work process. The thing is that there is no “single right way” to do things, and each team needs to collaborate to create an efficient workflow that works for them.
The 5 steps we took to building our quality driven development process
- Effective Product-Dev-Testing flows – We needed some more concrete requirements, tech designs, with lean and effective documentation. We also needed to establish what was the entrance and exit point to all of the “stops” in our cycle (And how the ping-pong would be addressed during the cycle). That took some work but I think that we are in a very good place now and we continue to learn and iterate.
2. Incorporating quality initiatives into our roadmap – An important thing to realize – plans tend to change in a fast-paced startup. With that said, if we don’t bring up things for the agenda – they will not go into our mindset (and eventually into our roadmap). I mapped the things I needed to implement in regards to quality and automation, ensuring it was a part of the plan and scheduled work procedure. One more thing I needed to change was proper time estimations for feature releases. While we do need to move fast and give reasonable time estimations, we started incorporating testing and automation coverage time into our time assessments for each feature. That way, the time we needed to release a well tested and “covered” product is being properly evaluated.
3. Tool choice – We decided on tools, frameworks and document formats that suited our needs the best way and worked on perfecting the flow with every sprint. The key guideline was to make the tool work without making us work for the tool.
4. Testing and automation coverage – We worked on reaching a point where our own automation gives us a very solid picture of our product and risk-analysis to cover what needed to be performed manually. In addition to that, we made it part of the process for each and every one of us to write automated coverage for the features we deliver. The standardized approach and guidelines are maintained with some insights from me and our Chief Architect. We also implemented an effective CI/CD process and pipeline to assure our fast pace and quality standards could go hand-in-hand.
5. Ownership & teamwork – It is true that in a startup there is a lot more versatility in everybody’s roles. With that said, over time we learned our strong points and took responsibility for things where we can contribute to making our work more productive. I took the Quality Assurance and Test-Automation part because it was my background. There was a time when testing was my sole responsibility. Time went by, we grew to understand that a bottleneck is not a good place to be stuck in (If you know what I mean). That lead us to adopt a whole new approach. We always worked as a team, and as such, everybody feels responsible to make their task come to an end with maximum quality and fast delivery time. A part of that is to take responsibility and test your own work. When I tell my colleagues that we have a team of developers writing test automation and performing manual testing according to a test plan – nobody believes me. It’s actually very simple math. We had a lot of work to do, and a limited staff. The alternative is to deliver features without testing or to delay releases, and that was not an option. Test your work and be responsible for it – it’s as simple as that. Ones you push that deploy button – you have to be confident that you did what you could to assure it’s quality. Psychologically speaking, when you know that you don’t have a “Gatekeeper” checking after you, you would do a more thorough job. With that said – at times you do need an additional set of eyes or it’s better to let someone else take a look after you have boiled your own code for a long time – and that’s where I come in – or someone else from the team who is available. The popular conception is that only testers can test a product. But, what is the difference between a tester and a developer? The main difference is that a tester decided to learn how to test and a developer decided to learn how to develop. Is it possible for someone to learn how to do both? Traditionally, organizations preferred that it was separate functions. However, today’s Agile and DevOps practices try to reduce the handoffs between various stages of the process to create a continuous workflow. Once you teach a developer how to test, and give them the correct mindset for testing – they can tackle any feature they develop. Since they know the code and its weak points, they can do an amazing job of testing it., Everybody who “Touches” the product and its feature at any stage of the development lifecycle has to do their best to deliver quality work. The bottom line is as I mentioned before, there is not a “one size fits all” model – it takes time and continuous improvement to adopt a process that works best for your team.
I think the main key here is making quality an organizational initiative. Whether you are in product, engineering, sales or marketing, focusing on the customers experience should be the responsibility of everyone. We try to leverage as much data as possible to provide cross-functional team transparency and not let emotion and anecdotal assessments get in the way of our decisions. We all share the same agenda and my mission is to influence and foster quality awareness and maintain a productive workflow which combines a detailed code review, effective CI/CD, unit testing and some TDD elements – our winning recipe for success.
