Provar’s Chief Strategy Officer, Richard Clark, has been excellent in sharing his thoughts on AI in the test automation and software quality spaces – take a look through our recent blog posts for more of his insight. Today, we’re diving deeper into AI in this Q&A with Richard.
Be sure to follow Provar on this blog and social media for more on this hot topic!
How will the rapid evolution of AI impact the overall growth and dynamics of the Salesforce AppExchange marketplace?
Initially, I think the primary difference will be how listings present themselves. We’ll see many false AI claims when tools use an algorithm. Hopefully, Salesforce will implement changes to allow people to understand what external services are used by listings and what compatibility there is with their own Einstein GPT features.
Arguably, Conversational and Generative AI tools should be creating more inbound leads to the AppExchange, too as they become more popular over Google Search for specific inquiries. For sure, it’s in Salesforce’s hands to make better recommendations of applications to admins from within Salesforce about apps that could improve both their role and company org to optimize performance or reduce maintenance. Whether or not those apps use AI is irrelevant if they add the most benefit to the customer by using AI to evaluate the recommendation.
How can AI-driven apps on the AppExchange differentiate themselves from competitors and provide value to Salesforce users?
Interesting point. I’d argue that AI-driven apps don’t inherently provide additional value to Salesforce users. They can provide additional value to the authors of those applications if it means they have to release fewer updates or their apps become more dynamic to the org in which they’re installed.
The biggest problem remains false claims about AI in apps. There’s a book by Hannah Fry, a professor and regular TV panel guest at University College London, and she says, “If you replace all the technical words with magic and the sentence still makes grammatical sense, you know it’s going to be [excuse my language] bollocks.”
Don’t get me wrong, AI-driven apps can provide value to users, like one that generates In-App help without needing to be written in advance or auto-completes data entry with suggested values based on previous similar interactions.
The apps that throw AI chat windows onto every screen so you type in what you want in a sentence instead of clicking buttons are either misguided or trying to hide the fact they have a poor user experience. When we look at expert users, they use a combination of voice, keyboard shortcuts, and automation to improve their productivity.
To find real AI apps with real benefits on the AppExchange, we need Salesforce to provide some level of tagging and scoring for this that avoids the ability of marketing teams to exaggerate the benefits. There needs to be reasons why the AI in use is better or faster than the alternatives.
What are AI’s most significant opportunities for your product on the AppExchange platform?
We have OpenAI API integration in our Provar Manager product. For customers to use that feature, they need an OpenAI API subscription. Those come with an operating monthly cost based on your number of queries. I’m excited by the potential of Salesforce to help us get better pricing for our customers based on shared accounts to access both OpenAI and other AI services through collective purchasing power.
Furthermore, a massive opportunity for us is where Salesforce goes with Data Cloud and EinsteinGPT. We can generate billions of test results that would be horrifically expensive for us to build our LLM. Letting customers apply our models to their data to tweak the results is incredibly exciting. Doing that in a secure and trusted environment is critical for enterprise adoption.
Instead of asking QA teams to write and execute test cases to verify acceptance criteria blindly, we can analyze what test coverage already exists, modify the data on those tests, and generate new test cases or data combinations to increase application quality continually. Feeding back in the bugs that hit production is incredibly valuable to that process and the results that can be achieved.
Thinking outside of the AppExchange, the same tools help us develop our product. We use them in our research, marketing, and content feedback, but most of all, AI is driving a massive increase in our online event sign-ups and product interest.
What are the significant risks and challenges associated with implementing AI-driven solutions on the AppExchange, and how can businesses mitigate these risks?
The most significant risks are the potential for bias, toxicity, and protection of data, plus intellectual property. There are plenty of exaggerated horror stories, which, to be fair, are essentially no different from when people paste code or data into an email, Google search, or, god forbid, stack overflow.
To mitigate, ISO27001 policies should be immediately updated, and staff should be retrained to understand the risks and threats. People also need to be urgently educated that the results of Generative AI are no more trustworthy than going to Facebook for fact-checking. You need to challenge and check the results before quoting them. This ability to exaggerate and invent is also why those tools are so exciting. The answer to these challenges isn’t to block access – those businesses choosing to prevent access are like those that refused to go onto the cloud until the 2020 pandemic forced them to.
As always, it comes down to adequate education and governance of what is being shared, and how the results are used. It’s not black and white. Suppose a developer wants a regex to validate a phone number. In that case, it’s an entirely different information category (public domain) than if I wanted to pass an AI my unique algorithm for calculating insurance premiums (company confidential). Information classification is critical.
Regarding data protection, toxicity, and bias, it’s worth looking into the announcements from Salesforce on their AI Cloud and what layers they’re putting in place. I’m excited to see this is being opened up to ISV partners, too.
What challenges have you faced while developing and implementing AI features in your AppExchange solutions, and how have you overcome them?
I’ll be honest. It was straightforward to implement. Our biggest challenge at first was the performance and time-outs when OpenAI first hit the mainstream market. They’ve responded with their geofenced services, and the paid plans have made this much more reliable.
Also well documented are the limits and requirements for how to prompt and break down queries and responses, but to be honest, that’s just a learning curve, not a challenge.
How is your company addressing ethical considerations when developing AI-powered applications for the Salesforce AppExchange?
Regarding security, we’re not transferring or recording any data onto our cloud platform that isn’t documented, non-PII, or anonymized against our product licenses.
For transparency, we’ve communicated through our customer advisory board an advanced notification of what data we share and when. We also consulted specific large enterprise customers with regulatory compliance needs before implementing changes.
Regarding accountability, all our use of data in AI is optional – our users can disable those integrations if they wish. When public AI models are used, we add multiple options for them to choose their trusted providers.
For fairness and guarding against bias, we template our prompts to avoid being a chatbot for AI. However, this is where the news from Salesforce on their AI strategy will be most helpful, including implementing data masking to protect customer data from external LLMs.
How can businesses evaluate and select the most appropriate AI-driven AppExchange solutions for their specific needs?
Again, I’d suggest not picking solutions based on their AI-driven capability. Instead, pick the best tool that meets your requirements rather than the shiniest one. Likewise, AI-driven solutions have an operational cost, so for any such product, you should consider your budget for variable costs and ensure those solutions can limit your cost exposure or manage it through a pre-pay or fair usage policy that prevents you from having unexpectedly large bills at the end of the month, quarter, or year.
First, calculate the cost of not having a solution to measure if any solution will have the desired effect to meet your needs. Regardless of AI, too many solutions are implemented without checking the ROI and payback period or measuring whether said ROI was delivered.
How does your company stay informed about the latest AI research and ethical guidelines to ensure the responsible development and deployment of your AppExchange solutions?
That is a real challenge. For the same reason, we all face a lot of misinformation. We work with industry analysts such as Gartner and Forrester to help support us, and my role is explicitly to stay abreast of the latest threats and opportunities.
Unesco has also published an Ethical Guidelines for AI paper, so research plays a significant role.
While that’s aimed at countries, not companies, evaluating the concerns and areas of potential impact is helpful. We also work closely with Salesforce, and I expect to see their security review process adjust to consider ethical guidelines and detect bias.
Currently, we’re restricting user input to stay within the boundaries of our application scope (i.e., a User Story from Elements Cloud, or Jira or ADO in a specific structure and format using Templates and controlling the prompts being sent). I think apps allowing uncontrolled Generative AI access risk being misused quite quickly with potentially very damaging results. Imagine if I asked my DevOps release management tool for share advice on what went wrong instead of which changes to deploy that were low risk? If people can, they will. That’s something customer service and testing taught me a long time ago!
Most of this is common sense, though. If we created an application that automatically tested websites using fuzzy logic to test different data field entries, it would be pretty obvious we’d want to avoid the malicious use of this to try and crack passwords or gain backdoor access.
Can you share any exciting new AI-driven features or upcoming releases that your company is working on with us?
Yes, we’re collaborating with several strategic partners to use AI to help enhance the integration experience. Where our products already integrate technically, there’s additional expertise that AI can provide to augment customer teams and streamline the process.
We can’t all afford to employ the best business analyst, test architect, or DevOps Engineer, so having some of those capabilities automated end-to-end and providing feedback on their effectiveness is going to be quite transformational and deliver on the real benefits of DevOps over following just one of those solutions. We talk about being a continuous quality company and want that to apply to our solution and our customer’s use of it.
There’s a saying in sales: there are two winners at every opportunity. The vendor that wins and the vendor who qualified out early. It’s the same with application development. We ultimately want to maximize the business benefit from making changes while using adoption and historical success information to determine when a change shouldn’t be made and suggest alternative ways to achieve the same result.
So, in the longer term, when we do have a valid requirement, our goal is to ultimately automate the complete implementation of that requirement and be able to evaluate its effectiveness through reinforcement learning. We’re not going to see whole apps created this way anytime soon, but it makes sense to use this feature for minor changes first and work our way up from there. We stop where it’s not adding additional value or accuracy.
That’s a bit cryptic, but I can’t go into specifics for obvious reasons. Wait and see what’s announced around Dreamforce this year and next year!
Want to learn more about how Provar’s robust solutions can help your organization? Connect with us today!
