The technology company highlighted in this case study is a household name. What many do not realize, however, is how many moving parts the technology company actually consists of. The company is a video equipment, telecommunications equipment, software, systems, and services provider that continues to acquire innovative companies in the space in hopes of providing customers with everything they need, all under one roof.
In 2016, they made an acquisition to help with this goal – a brand that manufactures security systems – and this new partnership allowed the company to sell security systems directly to consumers or through resellers to reach the greater market. We spoke with a quality assurance engineer who works on the application that is used to sell products to end customers and resellers. The acquired brand was the first to showcase its products on the company’s online eCommerce store, which has been quickly followed over recent years by other acquisitions. The store grew so much – to the tune of hosting tens of thousands of SKUs – that our QA engineer knew he needed a test automation solution to protect the application from risk. As the application is built on Salesforce Experience Cloud and involves thousands of configurations, this was no easy feat.
Challenges: Moving From Manually Testing a Complex System to Automating and Reducing Test Execution Time
When the QA engineer was hired, there was no system in place for test automation, and his team was doing manual testing only. His team of two (including himself) started by keeping track of all manual testing efforts using Microsoft Excel, which he admitted made it difficult to document properly and took up too much time. He had plenty of experience with Salesforce test automation in the past using Java and Cucumber, and he knew that Salesforce’s UI could be complex to test. He was faced with many challenges to consider before selecting a test automation solution, which he knew was necessary for the caliber of work ahead.
“Salesforce’s UI is dynamic, sometimes front-heavy, and takes time to load, and in the background, there are so many things that are changing. For instance, the control can be visible but you can’t interact with it. For each control, I’d have to right-click, investigate the control, find an xPath to locate for the control, rerun the test … and when you’re using free systems, their IDEs aren’t that great to start with, so when you’re writing your tests, you have to start from the very beginning. You can’t just jump in in the middle and work your way in.”
– Quality Assurance Engineer
He decided to start with Nightwatch for testing the framework, where he was able to automate around 60 tests. However, he quickly realized the maintenance challenge. As new SKUs were added to the application with each acquisition, the UI became more complex, and with Salesforce’s regular three releases per year added to the mix, the changes resulted in an incredible amount of maintenance for the team. He connected with Slalom, who had built the online eCommerce application, to see if they had any test automation solution recommendations, and they mentioned Provar Automation.
Choosing Provar: How 60 Automated Tests Turned into 700 with a Significant Test Cycle Time Decrease
Upon exploring Provar Automation’s features, the QA engineer knew it was the right solution for his team. He chose it after demoing a few other solutions because he knew exactly what he was looking for – and Provar had it covered.
“The thing that swayed me was the level of control Provar Automation had. I really liked how I could see the history of things that had happened. It was easy for me to go back and see if someone just missed something, if I needed to just copy a code that had worked in the past, etc. When you’re using Cloud test automation, everything lives in the Cloud, which makes it hard to keep track of the history of the changes. I’m biased because I’m used to working with frameworks and I like to keep track of the changes that are coming in, and I like to be able to go back to a previous version if something is breaking and see what changed.”
Quality Assurance Engineer
After making the move to Provar Automation, the QA engineer and his team converted the 60-or-so Nightwatch tests to Provar Automation. Additionally, the team used the original manual testing Excel spreadsheet to identify more candidates for automation, added more tests, and refactored along the way. Soon, what had once been 60 tests exploded into 700 automated tests using Provar Automation, with more being added as necessary with the expansion of the eCommerce store. The team’s original three-week test cycle was reduced to just 14 hours. Talk about a time saver!
“We have smoke tests that run nightly, and the regression suite which runs weekly. After we automated the tests, it took about 14 hours. So, we have reduced our testing cycle from about three weeks to 14 hours using Provar Automation.”
Quality Assurance Engineer
An Innovative Future Ahead
Due to the company’s success thus far using Provar Automation, the team has added two more QA professionals and looks forward to a future of even more employee growth. As the company continues acquiring brands, pushing to streamline and consolidate, and paving the way for innovation in a global capacity, its test automation needs will no doubt grow – and the team will continue looking to Provar’s solutions to speed up the integration process.
“If you can speed up a lot of things behind the scenes, it’s better for every aspect of the business, especially innovation. We’re thankful that Provar Automation helps us achieve this mission.”
Quality Assurance Engineer
Key Success Metrics
- Went from having 2 testers test 60 test cases to automating 700 test cases using Provar Automation
- Reduced testing cycle from 3 weeks to 14 hours
- Runs 700 test cases in approximately 14 hours
To learn more about how Provar’s solutions can meet your team at all stages of the quality journey, or to share your company’s success story, connect with us today.
