Partners, customers, and colleagues often ask me how we decide on the features and changes to support in upcoming Salesforce releases. I thought I’d share some inner secrets about our process. This includes how we handled the Spring ’21 release and every release over the last two years.

Before we get into the details, let’s first consider the main changes that come from Salesforce:

  1. Three releases per year, updates to existing products. These changes fall into one of the following sub-categories:
    • Changes that take effect immediately
    • Changes that an admin or developer needs to configure or implement
    • Changes that a future Release Update will enable (see below) or that are currently auto-activated with the release
  2. Release Updates that are time-activated or can be enabled and tested at your leisure 
  3. Acquisition of new technologies by Salesforce

I’m focusing primarily on the first item for this article, which is the Salesforce release schedule. For the others, we cope with these as part of our regular product release planning.

Now let me share with you when we start testing Salesforce releases, as Michael Caine said…

“Not a lot of people know that.”

– Michael caine
 
 

If you’re interested, we will cover the Priority 4 and 5 items in our release webinars, videos, and blogs.

Salesforce release schedule on Spring 21 Alignment

Yes, since 2018, we’ve been invited to test Salesforce releases scheduled weeks before anyone else. We’ve used that time to provide earlier feedback on breaking changes and even help report bugs before they reach pre-release or sandbox preview. Cool, huh? 

 

During this process, we first focus on running our approximately 5000 regression tests. We report unusual issues to Salesforce and amend Provar to cope with the changes if necessary. With so many tests to run, we prioritize the areas most likely to change. This means we start with the Lightning UI to get results as quickly as possible.

Still, we’re continually extending this test pack to include Field Service, CPQ, and Industry Cloud solutions.

 

It’s worth noting that a minority of changes require the user to update Provar test cases.

 

For example, when we add a new transition screen, users must provide specific interactions.

(For example, the Spring ’21 release includes UI changes for Lightning Web Components.) As we can’t fix these for you (we deliberately don’t store your test cases on our servers), we capture them and share them with you in our regular Salesforce release Webinars, Videos, Blogs, and Twitter. I encourage you to use these resources!

Going back to the release timeline above, we aim to release an update to the Salesforce Business Scenario Testing (BST) team as early as possible after the pre-release starts. This helps our customers on the BST program get valuable feedback weeks before the sandbox preview on potential breaking changes.

 

We can access the draft release notes up to three weeks before the sandbox preview starts and check them a few days before the official announcement, just like you do.

If you’re unaware, useful Trailhead content for Release Planning walks you through what you should consider as a team and how to track changes. We follow something similar already, so I wrote this blog first!

We take these changes and prioritize them as follows:

  1. Breaking changes that affect or may affect test execution 
  2. Breaking changes that affect or may affect test mapping/creation

  3. New features that customers will likely want to create new tests for but do not enable by default

    Changes and new features that testers will likely not address

  4. Changes and new features that do not impact Provar but may be interesting for future backlog support

From this prioritized list, we will address all the items discovered with Priority 1 before our next release. We aim to ensure you can focus on reporting issues with your application, not ours when you start sandbox previewing. Where we do not have first-class support for a feature, or you have chosen to use page objects or custom APIs, you sometimes need to amend your custom mappings.

We raise internal change requests for the Priority 2 items. Two weeks before delivery, if we have the capacity and the change has a low regression risk, we always try to include it.

 

Our product roadmap includes Priority 2 changes that are not feasible due to their complexity or potential for regression. We prioritize their delivery over the next few Provar releases and evaluate them in conjunction with our anticipated feature roadmap.

 Sometimes, we push other scheduled features back. Other times, we pull forward changes in a related area that would benefit from simultaneous delivery.

Finally, we will raise change requests for Priority 3 items, but these stay on the backlog until there is a customer or business demand to support the feature. In most cases, Provar customers can still create and execute tests for these Priority 3 features. Still, we aren’t running our daily regression for them, and the tests depend on the user selecting robust locators rather than Provar suggesting the best locator for you.

 

If you are interested, we will address the Priority 4 and 5 items in our release webinars, videos, and blogs.

For example, the change from renaming Critical Updates to Release Updates was important for awareness. Still, it made no difference to most users unless you’re using Provar to automate interaction with the Salesforce Setup as an RPA function.

Finally, I’d like to remind you that Provar can test many Salesforce features intelligently and robustly using metadata and our unique understanding of Salesforce as the only tool dedicated to Salesforce first and foremost. In addition, we also provide API testing, pro-code, and our ProvarXTM page object integration to help you create tests for any other scenario as part of your end-to-end test strategy. Where we don’t have official support for a feature or application, 99% of the time, we still test using ProvarXTM.

Suppose you’ve questioned how we decide which features to include and how little time to implement such changes.

 

I hope you’ve found this article on how Provar plans for the Salesforce release schedule exciting and helpful. If you have comments on our changes or specific questions about a feature, feel free to contact us.