This is the second article in a four-part series, “Why Selenium Users Pick Provar for Salesforce Test Automation.” Don’t just take our word for it. Read along as we look at (1) the behind-the-scenes reality of using Salesforce and why testing is needed in the first place, (2) a comparison of the types of testing available, (3) why Salesforce and Provar are BFFs, and (4) what makes Provar so special. Be sure to read Part 1: One Size Does Not Fit All When It Comes to Test Automation, on our blog.

Salesforce is designed around a declarative coding paradigm – “clicks, not code” – meant to enable development by people without traditional coding skills. It’s a critical business application that is updated at least three times a year, and testing becomes a challenge because often the testing tools are more code-centric than Salesforce itself. There are plenty of ways to meet the testing challenge. Here’s a list of some of the most popular approaches along with a summary of their pros and cons.

1. Manual Testing

Pro:

Easy and fast to implement for functional testing. By definition, manual testing doesn’t require coding expertise, but it does require business process knowledge and a well-documented test plan.

Cons:
  • Error-prone for repetitive testing (large data sets, test permutations).
  • Doesn’t scale – the scope of a full regression test for a reasonably-sized Salesforce organization requires a LOT of manual tester hours to complete. One real world example: 130 hours and 3 weeks of manual regression testing got reduced to 24 hours and 1 day with test automation software.
  • Test consistency depends on the tester(s).
  • Zero integration with development, CI/CD ecosystem.

2. Coding and Scripting

Pro:

Selenium is the tool to use: It’s open source and has a large community.

Cons:
  • Requires technical expertise that takes time to develop. And, that expertise is getting more expensive and harder to find.
  • The test is built separately from the application, requiring an iterative process to code test, run test, debug test, and repeat until the test successfully completes.
  • Test code needs to be updated whenever the application changes.
  • Debugging, updating, and maintaining tests requires coding expertise.

3. Record and Playback

Pro:

This is a straightforward method of developing a test; simply turn on the recorder and step through the processes that you want to test – no technical expertise is needed to record.

Cons:
  • Tests built with record and playback are easy to break (any changes on the pages or steps), difficult to edit (record a new version), and you need to know the entire process in advance.
  • This approach doesn’t scale when building n permutations of the happy path.

4. Scan and Assemble

Pro:

Less removed from application.

Cons:
  • This approach requires tool-specific knowledge to be successful.
  • Although it is easier to create tests from elements, more step-by-step configuration is required. It’s like coding (#2 above): There is an iterative cycle of build-run-debug-repeat.
  • The test builder is presented with more choices than she needs to create a test step. This means it is going to take longer to build the test, and it will not always work properly the first time.

Additional Considerations

Developer-User Communication Gap

It’s been true forever. There is an impedance/perspective/vocabulary mismatch that exists in conversations between developers and users. For test automation, this means that lots of communication, and typically several iterations of test design, needs to take place before the test meets the business user requirements.

Try this theorem on for size: The more a test building solution is abstracted from the way the application is used (and the application user), the more likely it is to require multiple iterations to resolve a test plan. Manual testing – less of an issue. The test builder and the application user can be the same person and it’s easy for a user to see what is being tested. Coding and scripting – big issue. The user can’t read the code to understand the test and can’t validate the test plan until after the test is debugged enough to run successfully. Iterations are time-consuming and in big increments. 

Here’s an interesting question to ask when looking at test automation solutions:

Is the test builder more focused on the test building tool or the application and the business process?

If it’s the former, you’re going to have to work harder to close the gap.

Building a Complete Test Before Debugging is Problematic

One of the challenges with most traditional web application test building approaches is that you need to build the entire test before attempting to run it. This creates the build-run-debug-repeat cycle noted above. It’s time-consuming to constantly step out of the building process to debug a step and then start over. 

Enhancing Generic Web Testing Tech is a Bandaid, Not a Fix

We’ve noted that Salesforce presents some specific challenges to typical web element locator strategies – basically, lots of change and difficult-to-identify fields. There are many technologies in play to help improve the success rate of locators, plus some others designed to reduce the level of abstraction created by the tool. These include natural language, machine learning, OCR, and image recognition. (It’s a blog for another day, but “AI” is often added as a test automation tool feature – whether or not the marketers are adhering to a strict definition of AI. Some are spot on while others are, at best, wildly optimistic.)

Here’s the thing: We’re talking about generic web testing applications, and this approach is definitely delivering some innovative best-in-class solutions. But, for Salesforce, this approach will always fall short because the generic tools are a) creating tests that fail frequently and need a lot of maintenance to keep up with change and b) more abstracted from Salesforce and the Salesforce user. OCR, image recognition, ML, NL, and self-healing tests are all just another way of saying “we need better web locator strategies.”

If all of this depressingly sounds like you’re out of viable options, you’re not. There’s another approach.

A Test Automation Solution Engineered for Salesforce

Salesforce implementations are important enough and complex enough to warrant a tailor-made test automation solution designed specifically for the Salesforce platform instead of adjusting a generic tool to fit. You don’t use a GP when you need a neurosurgeon, you don’t use a plumber when you’re fixing the cooling system of a nuclear reactor, and you shouldn’t use a generic test automation solution on one of the most important platforms in your business. 

One synonym for generic is “off the rack.” Great utility, more affordable, more available. Just not the best fit. A bespoke suit may cost a bit more and take longer to make, but you will look fabulous.

→ Stay tuned for our third blog in the series, “Community and Culture Matters.”

Can’t wait for the next blog post in the series, and are already itching to set up a demo? Let’s talk. Connect with us, tell us about your testing needs, and we’ll get back to you to chat further!