_Third Party QA Testing with Xray

Hiring external QA teams to verify a product has become a common practice. The service, referred to by the industry as Third Party QA Testing, provides objective testing of a product because the team (third party testers) was not involved in the development process – they do not take the product personally, so they can test the correctness of the code without emotional burden. From the article you will learn why else to use this service, what it consists of and how Transition Technologies PSC approaches its implementation. You are invited!

Taking a critical look at a product in whose development you are involved from start to finish is very difficult. The participation of testers in the development process can cause trivial defects to become less apparent, to blend into the software over time, and to be unknowingly implemented. And later they cause a failure. This can be avoided with Third Party QA Testing.

Before I continue, I would like to emphasize right away that having an in-house QA team is not an obstacle to hiring an independent testing team. An external QA team, as an independent auditor, provides a guarantee of the highest software quality.

Third Party QA Testing – why is it worth it?

By choosing to outsource to an external QA team, you gain:

• Objective product testing.
• Better quality – an external auditor will catch this, your team no longer pays attention to.
• Savings – by investing in QA, you minimize the number of product fixes, and your team can take care of another area in the meantime.
• Accelerate verification – thanks to test automation, for example, the verification process goes faster, relieving the QA team of time-consuming regression testing, for example.

You don’t have to worry about security, either, because the entity to which you submit the tests is legally bound to complete confidentiality.

Third Party QA Testing – when to go for it?

The answer is simple: whenever there is a need for testing, regardless of whether the outsourcer has its own development or testing team or wants to use external, more experienced and independent resources.

None of the above options are mutually exclusive; on the contrary, when working well together, they can complement each other perfectly – the business knowledge of the company’s QA team combined with the Third Party QA team bringing an independent assessment from the end user’s perspective, guarantees the highest quality software.

Third Party QA Testing – what does it look like to work on testing for an outside company?

Transition Technologies PSC is a global provider and integrator of IT solutions for industry and business. Within the structures of the organization there is a QA team providing Third Party QA service, which through experience has developed an operating model for working in this area – and this is what I will share in this part of the text.

From preparation to testing – step by step

1. We start with conversations; communication is key, as it facilitates clarification of the paths and work methods required by the client. Based on this, we clarify the goals and select the most appropriate methodologies for software development and quality control.
2. We familiarize ourselves with the requirements, which are the pillar of testing. Verifying for completeness and accuracy helps eliminate potential problems and discrepancies at the planning stage. And this speeds up and facilitates work at subsequent stages, while offsetting costs associated with downtime.
3. Based on the verified requirements, a test plan is developed, defining the scope, objectives and business path for testing. An in-depth analysis determines the best approach to testing, the need for and number of test environments, and the test tools that will work best for a given test scope.
4. The next step is to configure the test environment(s) so that the tests are conducted under conditions as close as possible to actual use.
5. Once all the above aspects necessary for testing are prepared, we start testing and verification. The team performs tests according to the highest standards, which are placed in the test plan, allowing simultaneous verification of both requirements and functional specifications.

Testing and verification

• The types of tests are adapted to both the software development process and the advancement of the work; the ability to introduce automation allows for faster regression testing and provides a forward-looking background that the client can use in the further development process of the product.
• Defects or problems discovered during testing are documented and submitted to the software vendor for repair. The documentation is clear and detailed, describing all the steps necessary to reproduce the defect, as well as supporting documentation, screenshots or recordings.
• After the vendor fixes the defect, retests are performed to ensure that the errors have been eliminated and the corrected software works as expected.
• Before the software is finally deployed to the production environment, acceptance testing is performed – through this testing the team makes sure that the product meets the acceptance criteria and is ready for release. These tests can take the form of UAT as well as alpha or beta testing, depending on the vendor’s decision.

All testing processes are documented, providing assurance that the software provided by the vendor has been thoroughly tested and meets the standards and requirements of the organization.

Xray’s role in Third Party QA Testing service

One of the primary tools we use is Xray, an application proven by thousands of users. According to the manufacturer, it is used by more than 5,000 customers from 70 countries. Xray integrates with Atlassian‘s Jira and Confluence products, making test and defect management easy, intuitive, and requiring no complicated training.

Why choose Xray? Benefits:

• Easy planning and execution of tests.
• Monitoring testing activities directly in the Atlassian ecosystem.
• Functionality dedicated to both manual and automated testing; the tool can be used in various testing methodologies, including BDD approaches.
• Centralizing test results in the environment, without the need for external tools.
• Integration with frameworks such as JUnit, NUnit, Robot, Behave and Selenium.
• Compatibility with DevOps solutions – Jenkins, Bamboo or GitLab.

Xray test process

Based on previous analysis and business requirements (usually in Epic or Story form), test cases – Tests – are created in the Xray tool. Each Test can be grouped into thematic Test Sets, which, properly defined (e.g., for a selected Story or environment), give quick feedback on the status of the software piece under test.

Test Sets can be executed repeatedly thanks to Test Execution, which is a package of test cases executed either “ad hoc” or according to the software version release cycle. Both Test Execution and Test Sets (which are most easily compared to mathematical sets) can be captured in Test Plans – which can contain multiple Test Sets in the case of global tests for a software version release or UAT level.

Regardless of the advancement of the tests, at any time we can monitor their progress and generate reports that, transparently, show the real and current progress of the testing process. Their simplicity with simultaneous accuracy, makes it possible to present them to the business client, without worrying about lapses in the reports.

Xray is a tool that supports the entire software release process; it streamlines testing and reporting, saving time at both the QA and management team levels. Thanks to its intuitiveness, it is easy to use and requires no additional commitment to learn the tool. As a Partner of Xray, TT PSC’s Quality Assurance team is one of the few in Poland with as many as 35% of testers certified as Xray Partners. We also received an official award: Xray Partner Awards.

Summary

With years of experience in software development and testing, we can confidently say – testing is our hobby. We don’t do double testing unless the client requires it. We find bugs and gaps in testing, thus reducing project risks to a minimum. Furthermore, we do not generate unnecessary tests, thus reducing the cost of maintaining the team. Depending on the vendor – we are ready for automation, which can be maintained by us, or by the client’s QA team. We rely on experience and choose the best possible tools to support testing – including Xray.