Salesforce is at the heart of enterprise business processes, and 59% of companies deploy a new build daily. That is why we need to identify common Salesforce test automation challenges.
Brittle automated tests remain challenging, costing teams time and money and undermining a core Salesforce benefit: agility. Modern leaders seeking to drive business revenue must first address the continuous rework spiral in their software delivery.
Enjoy a conversation about the Salesforce test automation challenges and how Provar’s Salesforce-aligned architecture and solutions can help teams get out of the rework spiral with industry leader Andy Menzies, CRO at Provar.
Question and Answer with Andy Menzies on Common Challenges in Salesforce Test Automation
Tristan Lombard: Andy, we are thrilled you’re here and the topic of today’s conversation. We know many leaders are having challenges automating test automation strategies, and we wanted to talk with a leader like yourself. And we have a few questions for you if that’s all right.
Andy Menzies: Yeah, of course…go ahead.
Tristan Lombard: So we know that Salesforce is at the heart of enterprise business processes and that 59% of companies are deploying a new build daily…as a leader in this field, why is now the time for companies to start investing in Salesforce test automation?
Andy Menzies: Well, let’s be clear about this. Testing used to be a conversation that was had purely on the left-hand side of the equation, i.e., development and IT… testing is a boardroom topic… there’s a direct correlation between the quality of your applications and the metrics you use to run your business, whether it be top line, bottom line, customer satisfaction, NPS score, retention, churn.
Whatever the metric is, there’s a direct correlation between the quality of your application and your ability to compete in your marketplace.
Tristan Lombard: Why does Provar Salesforce-aligned architecture make our solutions unique in the market?
Andy Menzies: We’ve gone to market with four critical pillars, if you will. Unbreakable. What do we mean by that? Well, I’ve been involved in test automation for some time now. I’ve been involved in test automation at the cutting edge, where artificial intelligence and machine learning were incorporated into the technology stack. Obviously, with good reason and motivation to improve quality generally, but what has happened with that evolution is we’ve introduced a lot of new work and a lot of time to market disadvantage.
Suppose people have to build frameworks for supporting AI, and they’re building digital twins or state transition models that are blueprints, digital blueprints of their application. In that case, they’re entering into a whole heap of work that they didn’t have to do. Now, what we mean by unbreakable is this fact, undisputed.
Salesforce releases multiple releases every year. You have no choice. If you go back to the days of Salesforce’s entry into the marketplace, they talked about no software. You will use the upgrades they release; consequently, if you’ve used traditional tools to build your test automation framework, they’ll break. They are brittle, as if you were to build all of your test automation framework Selenium. Consequently, every time your underlying application changes, your code breaks.
You’ll enter what we call the rework spiral. You’ll spend more time maintaining your test automation framework than building it—couple that with what I’ve just discussed regarding new approaches using AI and machine learning. For example, when you build a digital blueprint, you’re doubling the necessary work effort.
So why is that important? It’s important because I’ve already insinuated, and again, I think it’s undisputed, that the quality directly correlates to the metrics you use to run the business. So why introduce a whole heap of additional workload, a whole heap of additional expense to pay attention to quality? So the fact that we’re unbreakable, that we work from within our DNA if you like with Salesforce, that we support the metadata, means every time those changes occur, Provar keeps running. It’s unbreakable.
You also mentioned polymorphism… It’s a word we borrowed from biology…where an organism can have many different forms, and we’ve stolen that term because you can build a test script in Provar that can take on many different forms. In other words, you don’t have to rewrite that script every time that particular journey occurs in a business process.
You can use that one script you’ve developed in a polymorphic fashion and drag and drop it into as many different scripts or journeys as you like. That saves a whole heap of time and, therefore, money.
Tristan Lombard: And I think that’s something that you touched on previously as well when you’re talking about metrics when we’re talking about digital blueprints. We hear this in the industry as well from a testing approach. Right? You can have the best test approach but fail to help the business achieve its goals. You touched on this a little bit. What advice would you give leaders to help better align their testing goals to drive results for the business?
Andy Menzies: Yeah. I don’t know whether you guys are familiar with the term BizDevOps. So I would expect you are. I think Gartner coined the phrase many, many years ago. It should be BizDevOps because the whole idea of DevOps was bringing development and operations together to operate more harmoniously so that you could increase the speed and the velocity at which you were developing and releasing technology to the market.
Well, the business has a vital role to play in that. So my advice is to get the business involved in building your test organization and framework as early as possible and make sure that what you’re doing is aligned with your users’ experience in the field. You’re maximizing their experience in terms of quality.
Salespeople come to work every day, and I’m talking most specifically about CRM here rather than all of the other plethoras of Salesforce apps we test. Salespeople come to work every day to hit targets, to close deals, and if you like to feed the pipeline. And they can’t do this if they’re stuck on a screen. It’s essential to consider the employees and your customers when testing technology. What we want to do is we want to empower our employees and give them productivity tools that help them do their job more effectively. The only way to do that is to make sure that what you deploy works and works in a way that satisfies them to deliver on that goal of productivity.
Tristan Lombard: I think that’s well said. And just as a follow-up kind of question, when we’re thinking about usability, maintenance, and scalability, I’m just wondering, what are your top three recommendations for leaders considering investing in new testing tools? What would you have them consider when they’re looking at investing?
Andy Menzies: Well, I mean, indeed, if we’re talking about Salesforce, which I genuinely hope that we are on the fact that that’s what we’re all about. Firstly, ignore every other technology that’s generic. Ignore every technology explicitly built to test anything and everything and focus on a technology built from the ground up to test Salesforce. I think that’s important. Many business people aren’t necessarily familiar with LWC classic lightning. Still, they will not necessarily be familiar with some of the frameworks within Salesforce, Visual Force, etc. Still, these are critical considerations when it comes to testing.
If your test tool hasn’t been built specifically to work with Salesforce, you will experience a heap of pain trying to get it to represent the app you’re testing. As I’ve alluded to, maintaining it will be challenging and very costly.
Tristan Lombard: Yeah. And I appreciate that. And I think that was when we’re touching back on … thinking about the value props for Provar in the first place. Intuitive, right? So whether you’re a Salesforce administrator spending too much manual testing and pre-release or a Salesforce developer looking for custom-coded solutions, you want a tool that can do both and has that flexibility. I think that’s one of the things our many customers have been saying about why they love Provar and just something to consider. I have another quick question for you…You have often stated that we need to shift from POC as a term we’re using to POV in our industry. Can you elaborate on this?
Andy Menzies: In the world in which we live, where test automation, I believe, has already proven its value, we probably all do not, from a sales cycle perspective, have to do either a POC or proof of value. Certainly not a POC, which is where my belief system originates from because proof of concept, as described, is proving the concept of test automation, and I don’t think that needs to be proven.
Everybody is responsible for quality and understands the relationship between quality and outcome, in this case, positive business outcomes. So proof of value, unlike a concept, is to focus on those outcomes.
Why are we testing this? What do we believe we will realize regarding the positive business outcome due to having a higher-quality Salesforce deployment? The answer lies in the metrics you use to run your business. When entering a proof of value with a prospect or customer, focus on the outcomes that test investment delivers in terms of metrics. It’s more important to measure these outcomes than to focus solely on testing the features and functions of the tool.
Now, don’t get me wrong. The features and functions of the tool drive positive business outcomes. For example, a feature of our technology is that we work with Salesforce metadata. That means that every time there is a new release, forgive me for potentially repeating myself. With each new release, your tests will break if they are built with something other than Provar. However, because Provar works with metadata, we can refresh it. This means the tests you’ve invested time and effort into building, especially if you’re aiming for 80-90% coverage of your app, will still work with Provar.
Tristan Lombard: So, the follow-up question to that, Andy, is whether a recent world quality report states that automation is the biggest bottleneck holding back the evolution of QA and testing today. We sometimes see this regarding effective implementation for any leader. And I’m wondering, from your experience, what’s the number one mistake you see leaders make when implementing their test automation solutions, and how does Provar help address this issue?
Andy Menzies: I don’t want to spend all of my time trying to necessarily differentiate between Provar and every other technology on the market because there are some advantages in some other technologies, particularly outside of Salesforce. But as test automation has evolved, one thing that’s remained common outside of Provar is that it takes a whole heap of time to build the environment, build the framework, and get going. As a consequence, a lot of folks simply build a happy path. They’re building the regression pack, and that’s it. They’re not necessarily shining the torch into all of the journeys the users take as they interact or navigate their way around the application.
So, from my perspective, the primary comparative advantage for business leaders is a time-to-market advantage. Let’s just put this in perspective. There is a whole heap of references that we can give the audience regarding Splunk’s use of our technology, the ALM brand’s use of our technology, and many others. I would encourage everybody to visit our website and look at those case studies.
I’ve been hiring many people at Provar recently, and within two days, they’re productive with our product. In just two days, they are building test suites, using Provar, and running them to assess the quality of deployed applications. This quick productivity is a distinct advantage that sets us apart from other vendors. Business leaders and anyone who cares about quality, especially in Salesforce applications, should consider how quickly test automation can be deployed. With Provar, it’s incredibly fast. After deployment, you can start monitoring the success or failure of the application using your business metrics. The value comes from deploying and investing in test automation.
Tristan Lombard: Yeah, I think that’s well said, and I also want to do a special shout-out to the University of Provar. It’s free and a great way to uplevel your skills—also, a shout out to our fantastic customer success leaders and support. So, Andy, we want to end on a fun note with a fun question here. In your opinion, what is the most underrated Provar feature and why?
Andy Menzies: Yeah. Metadata support is the most powerful feature for achieving positive business outcomes. I always evaluate a technology’s feature set to understand the minimum capabilities needed. This approach ensures that the technology drives the outcomes the business requires. And if I think about that, rather than a fun feature… it’s the metadata support. And I’ve seen, read, and experienced people talking about how they can potentially do something similar, and they can’t. So, I’ve always identified three different types of differentiation. Right?
There’s unique, there’s comparative, and there’s holistic. So, a unique differentiator is something that you can do that nobody else can do. You’ve probably invented it. You might even have a patent or patent for it. We have a whole heap of unique features in our technology. I’ve been hiring many people at Provar recently, and within two days, they’re productive with our product. In just two days, they are building test suites, using Provar, and running them to assess the quality of deployed applications. This quick productivity is a distinct advantage that sets us apart from other vendors. Business leaders and anyone who cares about quality, especially in Salesforce applications, should consider how quickly test automation can be deployed. But by consequence, your unique differentiator becomes comparative. And 99.9% of deals have won on comparative differentiators.
I urge anybody who wants to test Salesforce to think about Provar because this is our bread and butter. The business was established in this manner. We have approached scaling in this manner. The velocity that we possess is a result of this. And this is why we have so many happy customers. Metadata support is my favorite feature, Tristan.
Tristan Lombard: This has been quite the interview with you all, and I have to say I wouldn’t want to do it with anyone else except our CRO, Andy. He has seen it all and will continue to be there to provide additional guidance. This is so much fun, Andy. Thank you so much for all you do for our customers and everyone in the community. You’re quite a light, full of a wealth of information. We should do this again sometime.
Andy Menzies: Yeah, we should. Yeah, we can finish off what we started.
Tristan Lombard: With that said, I appreciate all you’ve done. And we’re going to do this again soon. Okay? From all of us here to you, take care and happy testing.
Andy Menzies: Thank you so much. Happy testing to you, too. Bye.
Testing and test automation are crucial to improving the quality of your Salesforce releases. Knowing the Salesforce test automation challenges is essential to make our test automation efficient and effective. Learn how Provar’s critical applications can help you scale your testing efforts here.
