This case study focuses on one risk management company that helps companies anticipate, prepare, and respond to issues with software that provides organizations with data-driven insights.
“Provar helped us implement a more effective and thorough use case driven regression testing methodology. Manual testing alone is fundamentally flawed – you can’t test key scenarios extensively enough.”
– Product Engineering Executive
ABOUT THE COMPANY
This risk management company helps companies anticipate, prepare, and respond to any situation by equipping them with the software solutions they need to be successful. By empowering organizations with the right solutions to make data-driven decisions, leaders can proactively manage what’s to come.
Its core product offering helps organizations access their applications securely anytime, anywhere, and from any device – providing peace of mind regarding data security, system availability, and data storage.
- Established in 2006
- Company size: 225 employees
- Headquarters: Chicago, Illinois
- Industry: Risk management software
- QA team size: 3 full-time QA engineers
- Provar usage: Testing core product functionality
- Provar tests: ~1,000
- Test frequency: Prior to each major Salesforce and Fusion release (~6x per year)
“The tremendous value that Provar offers is having a QA automation solution that is built and aligned with the Salesforce roadmap. As a Salesforce ISV OEM, that’s incredibly valuable. The continued growth of Provar and the dedication to making the product more valuable and making sure that we’re set up for success has been incredible for our team and for our customers. Provar has helped to support the evolution of our QA function.”
– Product Engineering Executive
USING SALESFORCE
“We are seeing an increased need for more complex customizations from our clients, which in turn, demands greater diligence with testing. Provar is instrumental in helping us deliver those new features quickly and reliably.”
– Product Engineering Executive
As a leading provider of business continuity management software and services, the company specializes in helping companies build a resilient framework of operations.
The company’s core product offering is built on the Salesforce Lightning platform and provides a data framework with insights and personalized tools to help organizations become more risk-aware, prepared and ready to bounce back stronger from any event.
QA CHALLENGES: MANUAL TESTING LIMITATIONS WITH AN EXPANSIVE REGRESSION TEST SUITE
Since 2006, the company has been providing risk management solutions and services designed to help organizations spot potential issues and take action quickly and was recently recognized as a leader in the Q2 2021 Forrester WaveTM report for Business Continuity Software.
The company offers a variety of solutions and services with its core product offering. The system is designed to help teams map critical services and audit, analyze, and improve business operations.
In the early stages of product development, the company’s product team relied on manual testing alone to detect issues. Over the course of thirteen years of continuous enhancements to its core system, the company had developed an expansive suite of regression tests that were required in advance of each software release.
Prior to Provar, it would take three full-time QA engineers three weeks to manually execute the full suite of regression tests prior to six releases each year (three major Salesforce releases and three internal releases). While this helped to reduce their risk of introducing defects in new releases, the company was still limited in its ability to manage risk given the vast number of variables and combinations of environments, user permissions, browsers, operating systems, and languages that are encompassed by the core system.
In 2019, the executive team decided to explore test automation as an alternative to manual testing as a way to expand regression testing and support their goals for continuous innovation.
“The tipping point for us in moving from manual testing to test automation was realizing the sheer size of our regression testing needs. It got to a point where it just wasn’t manageable anymore. Manual testing alone isn’t going to scale with our aggressive goals for growth and innovation.”
– Product Engineering Executive
AUTOMATING REGRESSION TESTING WITH PROVAR
“The value-add that Provar has brought from a partnership perspective versus just a vendor has been incredibly valuable.”
– Product Engineering Executive
After evaluating several test automation vendors, the company selected Provar based on their Salesforce expertise and unique ability to help QA teams develop resilient tests. One common challenge faced by QA teams testing Salesforce is the recurring need to perform test maintenance after major Salesforce releases. Updates to the UI/UX, framework, and APIs can break tests that are built on the rendered UI of the page. One thing that sets Provar apart is that tests are built using the underlying data model of Salesforce which means that tests are more resilient and adaptable to change. This allows QA teams to reduce the rework spiral of update-related test maintenance and gives them more time in their day to test new Fusion features.
“It’s been a great partnership. It has felt like Provar has almost been an extension of our team’s testing practice.”
– Product Engineering Executive
THE IMPRESSIVE RESULTS
“With Provar, you are building tests within the application itself which helps our QA engineers build tests that more accurately reflect how individuals are using the system. You’re testing how things work instead of guessing how things should work.”
– Product Engineering Executive
As a result of implementing Provar, the company has been able to accelerate its release schedule and streamline regression testing by eliminating the vast majority of its manual processes.
With the efficiencies and time savings afforded by Provar, the company is positioned for further growth by automating nearly 70% of its existing regression tests.
“As a result of switching to Provar, we’re able to add more value-add throughput as part of our software development lifecycle.”
– Product Engineering Executive
