A QA’s insights from delivery projects
Nowadays, there are a lot of discussions about software testing and quality in the industry and from where a lot of neologisms are created, such as de-testers, uselessness of testers, technicalization of testing, testing engineering, testing and quality empowerment, agile testing, continuous testing, etc. Ignoring what these buzzwords are really about, it’s obvious that testing and professional testers are put firmly in the fire line.
As long as a project still pursues high quality, it’s necessary to implement a whole set of systematic and professional testing, in other words, quality assurance effort cannot be saved, which requires highly specialized knowledge.
Although some companies or teams claim that they can be delivered successfully without professional QAs, there are certain prerequisites, such as the size of the project is relatively small, the other roles on the team have professional testing and quality capabilities and they are willing to do testing and quality related work. Some other factors could be sufficient resources and flexible timelines, low level of quality requirements due to the nature of low-risky projects, or some projects that are still in the exploration and experimentation stage, etc.
But for a project which is pursuing high quality, if the team lacks expertise in testing and quality assurance, it is definitely in need of professional QA.
A staffing perspective
A typical delivery team encompasses roles like project managers, designers, business analysts, developers, testers etc. According to agile testing and quality built-in methodology, every role is responsible for the project’s quality. But when it comes to daily work, everyone has their own piece of work that requires some expertise; for developers, they are supposed to be skillful at coding in a specific language and framework; and for testers, making test analysis and conducting a performance test are typically in their field. As a result of technology evolution, each vertical domain gains too much knowledge and it becomes more and more difficult for one person to master a different domain in a short time.
As we know, a proper team structure could be very critical to the project’s success. Except for the cases mentioned above, the amount and required skill level of QA for one project can vary from case to case, based on the type and size of project, the target quality level and other factors.
Let’s take a simplified example, assume that we have a team of 10~12 people to develop a new insurance backend system, which requires a relatively high level of quality. The project is about to last for more than 12 months and along the development, the team found that they cannot test as a real end user before the final release, at the same time, business requirements keep changing which causes frequent rework in development.
The project has various roles, including PM, PO, UX, BA, DEV, QA, where QA is at least a Senior QA or Lead QA, and other QA can be the general level QA. The ratio of QA to Dev should be around 1:3. As the ratio decreases, testing work will become a bottleneck and block us from moving things to done. When the ratio reaches about 1:5, the limit of automated functional testing is reached. As the ratio continues to decrease, the development work of automated functional testing will also be reduced accordingly.
When the ratio is around 1:10, the QA barely has time to implement automated testing because regular testing and quality-related work has already taken up most of the time.
So basically all the work related to automated testing has to be handed over to developers for implementation. However, some specific kinds of tests, such as performance tests, still need to be implemented by QA staff, and they can only implement the main performance test cases, and cannot implement the full suite of performance tests in all aspects.
Notice that the ratio here is not just random numbers, I got them from my years project experience.
The most important and heavy workload includes: design and implementation of test strategy and test architecture, implementation and management of test process, test analysis and test design, test case execution (including manual and automation). For large teams, it is also necessary to empower the team with testing and even build a quality system.
For a business system that focuses on business complexity, if the team does not have capacity to implement automated testing, external resources can be introduced in this case to perform the work of manual testing. However, high level test analysis and test design needs to be implemented by the team itself with complete context and deep understanding of the business domain.
The ratio of QA staff needs to be adjusted accordingly if any constraint changes, and the most important factor is the level of quality requirements. As long as the quality requirements of the project are high, there must be enough time to implement enough specialized work related to testing and quality, and preferably by professional QA staff. If there is no professional QA on the team, other roles with sufficient professional skills are required to do it part-time, and in fact this person is wearing QA’s hat.
A person perspective
For companies that adopt agile testing and do not have dedicated QA departments, the training of junior QA staff has always been a headache problem. Without a dedicated QA team, every QA is scattered in different teams. If these individuals already have enough professional skills and the ability to work independently, they generally do their job well. But for junior QAs, they generally do not have enough professional skills and the ability to work independently, such a working environment often makes them frustrated and difficult, and some may even give up their QA career.
If these young QAs could get better support, such as systematic professional skill development, or continuous guidance from senior QAs and answer their question, they will gradually build up their confidence and skills to complete their jobs. But to provide and make these support and guidance sustainable, it requires organizational change and effort. In some professional service firms, a virtual QA department has been organized to implement such training, and the instructors are the most experienced QA personnel within the company. In addition to systematic training for QA staff growth, there is also a need for professional people to provide continuous coaching and assistance in professional aspects, as well as career paths. This kind of support is critical during the formative period of a junior QA and it even may change how a QA values their role and starts to enjoy their work. In many companies I have seen, it is led by the QA manager of a department or a portfolio manager. On top of that, a company should also have a clear career path and corresponding grades to guide and motivate junior QAs. A typical grade level for your reference could be: QA, Senior QA, Lead QA followed by QA Architect or QA Manager.
And to manage these QAs, if they are employees, they could become a qualified QA via a systematic training programme, and through the continuous coaching can help them perform their work well. On the other hand, the management of contractors could be a very different story. First of all, generally assigning contractors to a project is temporary and the staffing could change at any time. Even if it's a relatively long term assignment, employers have little interest in cultivating contractors, since they’re not their employees. So the previous solution may not work for this situation. As a result, the contractor QA may not be capable of completing complex tasks. In a subsequence, they’re only assigned some basic testing work such as executing manual test cases.
However, in the standard agile testing theory, manual testing is not a major task, which makes it even harder to leverage incapable contractor QAs in agile projects. Unless the coverage of automated testing is very low but the quality of the project is high in the meanwhile, then a large number of contractor QAs can be efficiently used to do large-scale functional verification testing and regression manual testing. If a contractor QA is competent enough, it is possible to do the same work as internal QA either directly or through systematic training.
Therefore, depending on the project requirement and the capability of a contractor QA, you may have different answers for the question of whether contractors are necessary or not. If the project has a serious shortage of manpower and recruitment is not realistic, contractor QA is a good plan B for you. Also we cannot simply use contractor QA to represent different levels of contractors. We know that some QA can only help on manual testing and execute test cases instead of designing them. But also there are contractors who have professional testing skills, after some basic training, they could be just as good as an employee QA and complete relatively complex work.
One working model is to have senior QAs analyzing and designing use cases, then having Contractor QA to execute the test cases manually and let Junior Dev automate the tests. I believe this can work with some specific prerequisites.
For cases where contractors are only supposed to do manual testing, there must be a large amount of effort spent on writing test cases with sufficient scenarios covered but no recourse to do automated testing. The project timeline allows the slow pace of manual testing and regression testing. Also there will be a significant effort spent on maintaining and updating the test cases. In other words, the project can accept this inefficient way of working.
No matter which approach your project is taking, there’s a common prerequisite - the department or the team need to understand the importance of testing analysis and design and treat the test cases as a valuable asset of the project, just as any application code. That’s how we make our QA believe their work is valuable and appreciated.
As a QA, my first priority is to ensure and improve the quality of a project, thus to meet the quality requirements. What makes a QA feel good is to deliver a high quality project. If a project doesn’t have a pursuit of high quality, there won’t be corresponding investment on the testing and quality assurance work, which means there won’t be much QA resource. As a result, the lonely and helpless QA on this kind of project will find that they can hardly make anything done. So in the end, if a project needs high quality, you should put enough QA resources on it, no matter they’re FTEs or contractors; and ensure the time and effort spent on quality assurance. In the meanwhile every role on the project should share the responsibility of testing and quality. This is the only answer to deliver a high quality result. There are specialties in the industry, and professional things are still left to professionals to do, not only to get better results, but also to save time.