We were able to increase 12x productivity and one QE wrote the scripts in one week [with Provar].

Mohit Kumar, Senior Member, Technical Staff at Salesforce

Last week, Provar hit the Dreamforce stage with members of Salesforce and Alm. Brand Group will discuss “Transforming Test Automation and Quality,” an excellent opportunity for professionals from across the globe to hear about applicable solutions and experiences from two of our customers. For those who were unable to attend, here is a recap.

The session focused on how to measure quality in their software lifecycles, automate tests declaratively, manage test execution, and adopt a QA maturity hub to integrate all measures. Thought leaders included Ben Kim (Senior Manager, Software Engineering) Mohit Kumar (Senior Member, Technical Staff) at Salesforce, and Nicki Bang-Madsen (Technical Tester) at Alm. Brand Group. Alexa Cash (Director of Customer Success) and Michael Dailey (Director of Solution Engineering) at Provar facilitated the discussion.

Provar opened with an overview of its two products – Provar Manager, our quality hub for managing test cases, plans, and results. Provar Automation, our test automation solution, is built from the ground up specifically for Salesforce. Both Salesforce and Alm. Brand Group shared customer success stories using Provar’s solutions.

Salesforce: Challenges from the Employee Success Domain Team

Like many mature QA teams, frustrations mounted with Selenium’s lack of reliability and simultaneous increase in QA demand. Provar provided the solution; Ben Mohit and the team at Salesforce saw increased testing velocity and more reliable releases within 6 months in their V2MOM application regression testing.

The Salesforce team provided a detailed Q&A, including an introduction of their Business Technology Team within their Employee Success Domain Team. This team supports application testing for internal Human Resources applications, encompassing 80,000+ users. Their QA needs were in high demand. They needed increased feature releases. Their testing team was lean compared to QA demands. Manual testing wasn’t scalable. Selenium scripts failed in release regression testing. Selenium maintenance was problematic in a Salesforce release cycle.

After doing some research, we found out about the power of Provar. One of the biggest pain points we had when it came to automation, particularly on Selenium, was maintenance. We were spending 30-40% of our time continually redoing our automation test scripts in Selenium. With Provar, we were able to drastically reduce that entire maintenance effort, so it’s been really a big win for us.

Ben Kim, Senior Manager, Software Engineering at Salesforce

The impacts they saw as end users since incorporating Provar Automation as a solution were profound. Here are the results within six months, including onboarding and learning the tool and automation efforts:

  • Lower average hours/sprint per person: 88% reduction in test maintenance time per FTE per QA sprint  (2 dedicated QEs, 80 hours/person, 2 sprints of maintenance efforts to update and maintain Selenium test cases)
  • Faster time to automation: Selenium takes 12 weeks. Provar takes 1 week. Selenium only covers UI testing. Provar covers UI and API testing.

Our scripts were failing, and all our effort building them was wasted. So we introduced Provar, we onboarded, and we were incredibly amazed. We couldn’t believe the reduced time we now spent on scripts. We went from spending 3 months to doing the same effort in one week’s time with one human using Provar.

Mohit Kumar, Senior Member, Technical Staff at Salesforce

Alm. Brand Group: Increasing Confidence Across the Organization and Beyond

Nicki Bang-Madsen is a longtime Provar customer. She’s used it at multiple companies. I have also seen extensive growth, outcomes, and benefits with Provar products. She’s experienced adoption to interoperability. His team’s results range from increased regression speed to more nuanced success, such as more confident developers. 

Alm. Brand Group is Denmark’s second largest insurance group with 700k+ customers and the first financial institution in Denmark to use Salesforce. After incorporating Provar’s solutions, long-term outcomes included:

  • Ease of adoption to Salesforce teams: Teams appreciated the similar mindset between Provar solutions and their existing knowledge from working with Salesforce
  • More than just regression testing: 90% less time spent on regression testing, and 60% fewer errors reported by production (the outcome was better development ROI and more fun and satisfaction across the teams)  
  • Testing the gaps: Testing the teams’ end-to-end processes helps catch any gaps and increases trust and satisfaction from both users and stakeholders
  • 4x reduction in hotfixes in production: Due to unique Provar capabilities that aren’t accessible by using standard Selenium tools
  • Increased reliability that instills confidence and satisfaction: Reliability of tests and more team collaboration provide confidence and allow for faster delivery

[With Provar], by reducing errors, we increased confidence across our teams, stakeholders, end users, and through the entire organization.

Nicki Bang-Madsen, Technical Tester at Alm. Brand Group

A big thanks to Salesforce and Alm. Brand Group, thank you for partnering with us on this lively discussion. We are grateful for our partners in testing! For more Dreamforce news, connect with Provar on social media.