BlogTechnology and KnowHow

The Holy Trinity - the QA Role in Service Delivery

Symphony logo
Symphony
July 19th, 2022

"The most important single aspect of software development is to be clear about what you are trying to build."

Bjarne Stroustrup

I have been working in the IT industry for nine years. During this whole period, I have (one way or another) been working as a QA Engineer. Three years ago, I switched to a Service Delivery/Product Owner role. This career switch brought me to the topics that I wish I had discussed earlier in my career as a QA. Even though I am not a fan of long articles, I do feel that I can help out someone in specific situations with this "small" article with considerable value.

I frequently challenge and discuss the main benefits of the QA engineering role as part of the Agile methodologies. What does day-to-day business look like for a QA? What are the processes a QA team relates to? How does the QA team fit an Agile organization? What are the main contributions QAs enable teams with? When and what do you automate? How does a QA Engineer become a bottleneck in delivery? There is a myriad of questions that came to my mind as I moved away from a purely QA engineering role. One key question to take away is - When is the right time for QA Engineers to start getting involved?

So, just out of curiosity, I created a simple questionnaire with a couple of easy questions to explore general trends on this topic.

Survey

The survey consists of several questions regarding QA, careers, and of personal nature. All applicants were able to complete the survey anonymously under one condition - that they are either working or have worked as a QA. In total, 121 QA Engineers completed the survey. The number can help us get the answers we need.

I will start with some basic information about the survey. Let's first take a look at the engineers who took it. 33% were from Bosnia, 27% from Serbia, 18% from Croatia, and 18% from other countries. Out of 121 people, 64% are male, and 44% are female QA Engineers. 15% changed their job in the past year, and 40% have been working on the same project for more than two years.

Let's now focus on more relevant topics for this article and take a look at some numbers.

How many years of experience do you have as a QA Engineer?

How big is the project you are working on (number of engineers including management and product)?

Are you working on a Startup or Enterprise system?

When do you get involved in the product development lifecycle?

How well do you know your system?

Does your team think of you as a Tester or Engineer?

Results

As we can see above, I have included six important metrics to get some answers on how and when QA Engineers are integrated into the engineering team. Three of these questions showed some shocking results, which align with my understanding of the current QA market.

  • QA Engineers are involved in the product development lifecycle too late. Out of 121 engineers, 92 engineers are involved in features/releases/increments planning and understanding
  • 55% of the engineers understand the system way less than the Product Owner 
  • There were some comments that the team does not have a Product Owner, so they compared it to key stakeholders or anyone who owns the backlog
  • Almost 72% of QA Engineers feel that the team perceives them as testers and not engineers

The Holy Trinity of Service Delivery

As independent and self-sustainable, a QA is a highly appreciated, needed, and valued team member for any project, whether startup or enterprise level.

Agile methodology places a high emphasis on teams, people, communication, and interactions over formal processes or preferences for specific tools. This is even stated in the Agile Manifesto, but also in several articles on this topic. On the one hand, I agree, but there is one point that is very loose and undefined, and that is when this communication and interaction starts. As we can see in the results above, QA engineering teams start getting involved at the point when the backlog is already defined or development has started, which leads us to the first problem:

output > outcome

The later the engineers involve themselves (and not only QA Engineers), the less product empathy they have. The focus of the QA team becomes testing features and providing the green light for releases or identifying potential blockers. One of the questions in the survey was "How often is the release delayed or blocked based on your input?", where we got 49% for yes, and 51% for no. So, on the one hand, the QA Engineer loses the power of guaranteeing the release quality, either due to a lack of insurance the team has about the validity of testing, or they simply just want to release no matter what (mostly related to startups).

QA Engineers primarily focus on output, e.g., integrating new payment methods. A QA will test new payment method and give a thumbs up for the release. However, a QA must be aware of what clients we are serving, their needs, and why we are using the chosen methods. The outcome must be way above the output. The outcome of the specific scenario above could be, "How will this increase our revenue?", "How will this impact our NPS?", "How much will our cart value increase with this payment method?", "What are the clients we are targeting with this feature: Generation Y or Generation X?". In short, what is the success rate we can use to measure our output so that it defines the successful outcome? Implementing a crypto payment method might be interesting or trendy, but what outcome will that give us with all the fluctuations in that market?

QA Engineers must be aware of:

  • Who our targeted users are?
  • Why are we doing particular features?
  • What are the predicted scenarios in production (not the ones developers worked on)?
  • What would make me use this feature as a user?
  • Software Engineers cannot explain to QAs how the feature works since it is biased

Here we come to the term - The Holy Trinity of  Service Delivery.

The trinity involves three ingredients: Product Owner, Technical Lead/Architect, and QA Lead/ Engineer. Based on the results above, we see that in this symbiosis, a QA is probably the person who gets on top of things the latest among the three. This leads to less understanding of the users, system, and features and a lack of product empathy, which later makes the QA the main bottleneck of the cycle. How many times was the release late as the QA did not have all the information needed? How many times did the release have issues as the QA relies on the development explanation of the feature? How many times was the automation late because Automation Engineers saw the feature for the first time when they opened the test in any Test Management Tool (Either Jira, x-rail, or any other)?

These issues lead to the same conclusion -It is crucial to get involved as early as possible and be proactive rather than reactive.

As soon as the feature, initiative, or project is initiated, the Tech Lead, Product Owner, and QA must be involved in the phase of ideation and conceptualization. Of course, there are board meetings or high-management meetings that specific stakeholders can attend. But the first time an idea becomes a reality, there is a place to jump in.

The QA Engineer must understand why a feature became a priority in the portfolio, as well as the target and ROI.

Based on the results, 40% of the QA Engineers have been working on the project for more than two years. This qualifies them as a legit equivalent to a Product Owner when it comes to knowing the system. This way, as the Product Owner can challenge the engineering team in terms of delivery, the team can challenge the Product Owner in terms of features being developed.

What does the Holy Trinity look like?

  • Enforce involvement in the ideation phase (or at least the conceptualization phase). As legit owners of the quality, we need to know WHY we are developing something. This mostly ends with the Product Owner explaining the business side during Grooming (Refinement) sessions, but the QA Engineer is closer to the field (actual application usage). Furthermore, a QA will bring new insights to the proposed portfolio item in terms of previous experience, bugs/change requests, or less-used features in production
  • Often portfolio management decides on the initiative timeline, either delegating it to the Product Owner, Delivery Lead, or Project Manager to enforce it inside of the team. By combining the Product Owner, Q,  and Tech Lead/Architect roles in the early phases of ideation or conceptualization, the portfolio management can have realistic views about the idea being presented or prioritized. The Tech Lead and the QA will be able to determine if actual portfolio items are feasible in the proposed timeline
  • As the owner of the backlog and the main person who takes responsibility for the delivery of the specific item/project/feature, a Product Owner will gain more evidence and authority, and even knowledge to push back or deprioritize ideas when backed up with QA knowledge and architectural/development efforts from the Team Lead
  • There are many more factors here that I won't go into, but based on the above, I think you can get where I am going at

Benefits of the Holy Trinity

  • Prioritized portfolio items can be challenged on time 
  • Product Owner, QA, and DevOps Lead will have a head start and can think about implementation, architectural proposal, technical feasibility, and the general purpose of the specific portfolio item
  • The team can communicate all the changes and requests on time and define the dependencies of other teams
  • The team can plan testing efforts, automation timeline, and automation priorities on time
  • The team can define which parts of portfolio items are unclear and escalate questions on time
  • The team will increase product empathy
  • The team will be able to identify the capability to complete this kind of a portfolio item
  • The team will have a boost in confidence and understanding of the feature so that the right questions come at the right time
  • The team will have data needed to explain to other team members (Product Owner + Team Lead + QA )why we are doing something, which automatically increases team morale and motivation
  • The team will be able to conduct the POC or MVP way faster than if they hear about portfolio items during the grooming session
  • QA is no longer a TESTER but rather an engineer that can equally challenge implementation or business value/decision on time with authority to do so

Conclusion

As mentioned earlier, this article is based on a small sample audience, so the numbers should not be taken for granted. Maybe different sample groups would behave differently (which I doubt).

In personal experience, numbers match what I have seen, done, and managed. QAs and Tech Leads are included in the development lifecycle a lot later than they should be. Earlier inclusion in the ideation and conceptualization phase would benefit many highly discussed topics in IT Management:

  • Late releases or delayed releases
  • Production bugs due to a lack of defining all use cases
  • Over-commitment or under-commitment 
  • Communication issues and lack of communication with stakeholders and managers
  • Lack of authority for Team Lead or QA to say NO (for me, this is VERY IMPORTANT)
  • Collaboration with other teams
  • Product empathy 
  • Understanding how portfolio management works and behaves
  • Timely escalation and creating impediments

In summary, the Holy Trinity of QA - Product Owner - Team Lead involvement in the conceptualization or even ideation phase would bring a whole new light to the project. There will be problems at the beginning, pushbacks, and standard phrases like "That would require too many people involved too early," as if that is something bad.

To summarize -  involve yourselves in the product as early as possible, ask questions even before the idea becomes an initiative, challenge management, say NO, and communicate on time! Furthermore, ask who your clients are, what we are giving them, what our ROI is, and how it will increase your NPS.

Please note, as a Delivery Manager / Product Owner, sometimes this might be hard to enforce. However, once you are in the right place, with the right people, and at the right time - the development will rocket, bugs in production will set down, acceptance criteria will get clear, DOR and DOD will come with much better clarity, and most importantly, you will be recognized as an authority, equal in prioritization as stakeholders.

Thanks for reading, and I hope I will write more articles on similar topics. Feel free to ask any questions or disagree with me.