Salesforce is at the heart of enterprise business processes, and 59% of companies deploy a new build daily. That is why we need to identify common Salesforce test automation challenges.

Brittle automated tests remain challenging, costing teams time and money and undermining a core Salesforce benefit – agility. Modern leaders seeking to drive revenue for the business must first address the continuous rework spiral in their software delivery.

Enjoy a conversation about the Salesforce test automation challenges and how Provar’s Salesforce-aligned architecture and solutions can help teams get out of the rework spiral with industry leader Andy Menzies, CRO at Provar.

Question and Answer with Andy Menzies on Common Challenges in Salesforce Test Automation

Tristan Lombard: Andy, we are thrilled you’re here and the topic of today’s conversation. We know many leaders are having challenges automating test automation strategies, and we wanted to talk with a leader like yourself. And we have a few questions for you if that’s all right.

Andy Menzies: Yeah, of course…go ahead.

Tristan Lombard: So we know that Salesforce is at the heart of enterprise business processes and that 59% of companies are deploying a new build daily…as a leader in this field, why is now the time for companies to start investing in Salesforce test automation?

Andy Menzies: Well, let’s be clear about this. Testing used to be a conversation that was had purely on the left-hand side of the equation, i.e., development and IT… testing is a boardroom topic… there’s a direct correlation between the quality of your applications and the metrics you use to run your business, whether it be top line, bottom line, customer satisfaction, NPS score, retention, churn.

Whatever the metric is, there’s a direct correlation between the quality of your application and your ability to compete in your marketplace.

Tristan Lombard: Why does Provar Salesforce-aligned architecture make our solutions unique in the market?

Andy Menzies: We’ve gone to market with four critical pillars, if you will. Unbreakable. What do we mean by that? Well, I’ve been involved in test automation for some time now. I’ve been involved in test automation at the cutting edge, where artificial intelligence and machine learning were incorporated into the technology stack. Obviously, with good reason and motivation to improve quality generally, but what has happened with that evolution is we’ve introduced a lot of new work and a lot of time to market disadvantage. 

Suppose people have to build frameworks for supporting AI, and they’re building digital twins or state transition models that are blueprints, digital blueprints of their application. In that case, they’re entering into a whole heap of work that they didn’t have to do. Now, what we mean by unbreakable is this fact, undisputed.

Salesforce releases multiple releases every year. You have no choice. If you go back to the days of Salesforce’s entry into the marketplace, they talked about no software. You will use the upgrades they release; consequently, if you’ve used traditional tools to build your test automation framework, they’ll break. They are brittle equally if you were to build all of your test automation framework Selenium—consequently, every time your underlying application changes, your code breaks.

You’ll enter what we call the rework spiral. You’ll spend more time maintaining your test automation framework than you will be building that or building it—couple that with what I’ve just discussed regarding new approaches using AI and machine learning. For example, when you build a digital blueprint, you’re doubling the necessary work effort.

So why is that important? It’s important because I’ve already insinuated, and again, I think it’s undisputed, that the quality directly correlates to the metrics you use to run the business. So why introduce a whole heap of additional workload, a whole heap of additional expense to pay attention to quality? So the fact that we’re unbreakable, that we work from within our DNA if you like with Salesforce, that we support the metadata, means every time those changes occur, Provar keeps running. It’s unbreakable.

You also mentioned polymorphism… It’s a word we borrowed from biology…where an organism can have many different forms, and we’ve stolen that term because you can build a test script in Provar that can take on many different forms. In other words, you don’t have to rewrite that script every time that particular journey occurs in a business process.

You can use that one script you’ve developed in a polymorphic fashion and drag and drop it into as many different scripts or journeys as you like. That saves a whole heap of time and, therefore, money.

Tristan Lombard: And I think that’s something that you touched on previously as well when you’re talking about metrics when we’re talking about digital blueprints. We hear this in the industry as well from a testing approach. Right? You can have the best test approach but fail to help the business achieve its goals. You touched on this a little bit. What advice would you give leaders to help better align their testing goals to drive results for the business?

Andy Menzies: Yeah. I don’t know whether you guys are familiar with the term BizDevOps. So I would expect you are. I think Gartner coined the phrase many, many years ago. It should be BizDevOps because the whole idea of DevOps was bringing development and operations together to operate more harmoniously so that you could increase the speed and the velocity at which you were developing and releasing technology to the market.

Well, the business has a vital role to play in that. So my advice is to get business involved in building your test organization and framework as early as possible and make sure that what you’re doing is identifying with your user’s experience in the field. You’re maximizing the experience in terms of quality that they have.

Salespeople come to work every day, and I’m talking most specifically about CRM here rather than all of the other plethoras of Salesforce apps we test. Salespeople come to work every day to hit targets, to close deals, and if you like to feed the pipeline. And they can’t do this if they’re stuck on a screen. It’s essential to consider the employees and your customers when testing technology. What we want to do is we want to empower our employees and give them productivity tools that help them do their job more effectively. The only way to do that is to make sure that what you deploy works and works in a way that satisfies them to deliver on that goal of productivity.

Tristan Lombard: I think that’s well said. And just as a follow-up kind of question, when we’re thinking about usability, maintenance, and scalability, I’m just wondering, what are your top three recommendations for leaders considering investing in new testing tools? What would you have them consider when they’re looking at investing?

Andy Menzies: Well, I mean, indeed, if we’re talking about Salesforce, which I genuinely hope that we are on the fact that that’s what we’re all about. Firstly, ignore every other technology that’s generic. Ignore every technology explicitly built to test anything and everything and focus on a technology built from the ground up to test Salesforce. I think that’s important. Many business people aren’t necessarily familiar with LWC classic lightning. Still, they will not necessarily be familiar with some of the frameworks within Salesforce, Visual Force, etc. Still, these are critical considerations when it comes to testing.

If your test tool hasn’t been built specifically to work with Salesforce, you will go through a heap of pain trying to get it in the first instance to represent the app you’re testing. And in the second instance, as I’ve alluded to already, it will be challenging work and very costly to maintain.

Tristan Lombard: Yeah. And I appreciate that. And I think that was when we’re touching back on … thinking about the value props for Provar in the first place. Intuitive, right? So whether you’re a Salesforce administrator spending too much manual testing and pre-release or a Salesforce developer looking for custom-coded solutions, you want a tool that can do both and has that flexibility. I think that’s one of the things our many customers have been saying about why they love Provar, and just something to consider. I have another quick question for you…You have often stated that we need to shift from POC as a term we’re using to POV in our industry. Can you elaborate on this?

Andy Menzies: In the world in which we live, where test automation, I believe, has already proven its value, we probably all do not, from a sales cycle perspective, have to do either a POC or proof of value. Certainly not a POC, which is where my belief system originates from because proof of concept, as described, is proving the concept of test automation, and I don’t think that needs to be proven.

Everybody is responsible for quality and understands the relationship between quality and outcome, in this case, positive business outcomes. So proof of value, unlike a concept, is to focus on those outcomes.

Why are we testing this? What do we believe we will realize regarding the positive business outcome due to having a higher-quality Salesforce deployment? And I think the answer to that is, as I alluded to earlier, the metrics you use to run your business. So, for me, if we’re entering into a proof of value with a prospect or indeed a customer in a different geography or a different subsidiary or department, we should be focusing on the outcomes that test investment is realizing in terms of metric rather than specifically necessarily testing the features and the functions of the tool.

Now, don’t get me wrong. The features and functions of the tool drive positive business outcomes. So, for example, a feature of our technology is that we work with Salesforce metadata. That means that every time there is a new release, forgive me for potentially repeating myself. Every time a new release is made available to you, which you must take, your tests if built in something other than Provar, will break. So the fact that we worked with the metadata means that we refresh that metadata, and then all of the tests that you’ve put a lot of time and effort into building, potentially, if you’re looking for 80, 90% coverage of your app, that’s still a reasonable investment. That’ll work with Provar.

Tristan Lombard: So, the follow-up question to that, Andy, is whether a recent world quality report states that automation is the biggest bottleneck holding back the evolution of QA and testing today. And we see this sometimes regarding effective implementation for any leader. And I’m wondering, from your experience, what’s the number one mistake you see leaders make when implementing their test automation solutions, and how does Provar help address this issue?

Andy Menzies: I don’t want to spend all of my time trying to necessarily differentiate between Provar and every other technology on the market because there are some advantages in some other technologies, particularly outside of Salesforce. But as test automation has evolved, one thing that’s remained common outside of Provar is that it takes a whole heap of time to build the environment, build the framework, and get going. As a consequence, what a lot of folks are doing is simply building a happy path. They’re building the regression pack, and that’s it. They’re not necessarily shining the torch into all of the journeys the users take as they interact or navigate their way around the application.

So, from my perspective, the primary comparative advantage for business leaders is time-to-market advantage. Let’s just put this in perspective, and there’s a whole heap of references that we can give the audience regarding Splunk’s use of our technology, the ALM brand’s use of our technology, and many others. And I would encourage everybody to visit our website and look at those case studies.

I’ve been hiring many people into Provar of late…and within two days, these guys are productive in using our product. Within two days, these guys are building test suites, using Provar, and running those test suites to determine the quality of the application deployed to the field. That is a distinct advantage that I don’t believe any other vendor can claim to the same extent that we can. So business leaders, leaders, and anyone who cares about quality, particularly in and around Salesforce applications, must consider how quickly you can deploy test automation. With Provar, that’s super quick. And then again, as we’ve already talked about, start monitoring the success or failure of that application or that deployment using the metrics you use to run your business. So, the value you realized resulted from deploying and investing in test automation.

Tristan Lombard: Yeah, I think that’s well said, and I also want to do a special shout-out to the University of Provar. It’s free and a great way to uplevel your skills—also, a shout out to our fantastic customer success leaders and support. So, Andy, we want to end on a fun note with a fun question here. In your opinion, what is the most underrated Provar feature and why?

Andy Menzies: Yeah. The support for metadata is by far the most powerful feature set that drives the minimum required capabilities of technology to realize the positive business outcomes I discussed. So, I always try to take a feature set of technology and understand from a minimum required capability standpoint, which will drive the outcome that the business requires. And if I think about that, rather than a fun feature… it’s the metadata support. And I’ve seen, read, and experienced people talking about how they can potentially do something similar, and they can’t. So, I’ve always identified three different types of differentiation. Right?

There’s unique, there’s comparative, and there’s holistic. So, a unique differentiator is something that you can do that nobody else can do. You’ve probably invented it. You might even have a patent or patent for it. We have a whole heap of unique features in our technology. However, in many other companies, as is the case, when you come up with something unique that drives value, people start to copy your ideas, and that’s fine because if that’s delivering value to other customers, I’m happy with that. But by consequence, your unique differentiator becomes comparative. And 99.9% of deals have won on comparative differentiators.

I urge anybody who wants to test Salesforce to think about Provar because this is our bread and butter. This is how we built the business. This is how we’ve scaled. This is how we have the velocity that we have. And this is why we have so many happy customers. Metadata support is my favorite feature, Tristan.

Tristan Lombard: This has been quite the interview with you all, and I have to say I wouldn’t want to do it with anyone else except our CRO, Andy. He has seen it all and will continue to be there to provide additional guidance. This is so much fun, Andy. Thank you so much for all you do for our customers and everyone in the community. You’re quite a light, full of a wealth of information. We should do this again sometime.

Andy Menzies: Yeah, we should. Yeah, we can finish off what we started.

Tristan Lombard: With that said, I appreciate all you’ve done. And we’re going to do this again soon. Okay? From all of us here to you, take care and happy testing.

Andy Menzies: Thank you so much. Happy testing to you, too. Bye.

Testing and test automation are crucial to improving the quality of your Salesforce releases. Knowing the Salesforce test automation challenges is essential to make our test automation efficient and effective. Learn how Provar’s critical applications can help you scale your testing efforts here.