* part one of a two-part series takes a deep dive into choosing the right test automation solution for your team’s needs. Part One focuses on what you should look for in the technology itself, while Part Two focuses on looking beyond the technology and evaluating the vendor’s business model. Part Two will be posted on 11/17/22.
The guide below will clarify what to look for and what questions to ask to ensure you select the right solution for your team’s needs. Of course, we will start by discussing the technology.
The key considerations for a product evaluation of a Salesforce-specific testing solution fall into four categories:
| Test resilience or fragility | How well does the solution morph to the needs of the rest of the organization by using a single test that can run across numerous contexts? |
| Polymorphism and reusability | How well can the system run end-to-end tests that reach adjacent non-Salesforce systems? |
| Ease of use and learning | What skills and experience levels are required to create, maintain, and run tests |
| Testing adjacent systems | How well the can system run end-to-end tests that reach adjacent non-Salesforce systems |
Let’s break this down in a little more detail.
Test Resilience or Fragility
Here are some questions you can ask to decide if a test automation solution has the test resilience or fragility you need:
- Can a test created for Visualforce run without modification under Aura (Classic and Lightning)?
If Salesforce changes how a page is rendered (such as changes to the HTML, CSS, or JavaScript), will the test continue to work without modification? - If we change a Salesforce layout, will a test created for the previous page layout work without modification?
- If Salesforce changes an API used in a test, will it continue to work without modification?
Polymorphism and Reusability
The technology that makes tests resilient also enables polymorphism, where a single test can run across numerous contexts, and reusability reduces the number of tests you need to create and maintain. Reusable polymorphic tests make for a higher-quality solution that evolves quickly and easily.
Here are some questions you can ask to gauge a test automation solution’s polymorphism and reusability:
- Can I create a single test and run it for different user profiles or page layouts without modification or creating additional tests?
- Will a test created in English work without modification in other languages like French or Japanese?
- Can a single test run across PC, Mac, and mobile users on various browsers without creating additional tests?
- Can a test validate field visibility and accessibility based on user permissions for several profiles or permission sets without modification?
- Can a test created in the development environment run without changes in the validation or production environment?
Ease of Use and Learning
Ease of use and learning are other significant factors to consider in your evaluation. Many Salesforce users are “citizen developers,” meaning they may not have expertise in programming languages and frameworks. Your entire team must use your test automation solution so you can confidently deploy your tests.
Here are some questions you can ask to see if a test automation solution champions ease of use and learning:
- Are test steps added from a Salesforce screen (“right-click to add test step”)? Can you build your test in one location without toggling back and forth between multiple platforms or screens?
- Can I add a test step and see the results before adding the next step? Can I step back and remove a prior step? Can I pause and resume a test as I run it?
- If I select a button in the Salesforce user interface and right-click to add a test step, does the test step automatically default to click? Does it default to “set” for an input field? What other default actions should we capture? Does your approach automatically detect the locators, such as elements you want to test before you test, or do I have to label these elements? When you execute a test script, do you observe the test passing or failing in real time, or do you need to rectify and reapply it to verify that the issue has been resolved?
Testing Adjacent Systems
Finally, regarding the technology itself, you must evaluate a test automation solution’s ability to test adjacent systems. Every Salesforce platform connects to other enterprise systems and custom integration points. Your testing solution must work well with these systems to test your workflows end-to-end.
Here are some questions that’ll help you evaluate whether your test automation solution is ideal for testing adjacent systems:
- Can I test that a triggered email was sent? Can I test email to case flow?
- Can I verify that a Salesforce action triggered a connected action in an external system, such as creating an invoice in an ERP? Can I verify that an action on our website created the correct records/activities in Salesforce?
- Can I test that an external API has been called from a Salesforce customization?
Have more questions? Connect with us today to discuss your evaluation process and what your team’s goals are. We look forward to helping you choose the right test automation tool!
