In a recent webinar, Testing Agentforce with Provar, we looked into Agentforce — its various elements, key considerations for testing, and how Provar complements the Salesforce Testing Center to cover essential end-to-end testing.   

In this blog, we’re talking through the big themes from the webinar. We’ll tackle the challenges of testing Agentforce, highlight key focus areas for testing, compare the differences between Provar and Agentforce Testing Center, and answer some of the key questions that were asked during the webinar.

What is Agentforce Studio?

Agentforce Studio is Salesforce’s AI-driven automation framework designed to create and manage AI agents efficiently. It includes several components:

  • Prompt Builder: Allows users to create and test different prompt templates, useful for generative AI applications.
  • Agent Builder: A low-code tool for creating enhanced chatbots capable of executing various actions, workflows, and custom Apex scripts.
  • Model Builder: Enables the development and integration of custom AI models.
  • Testing Center: Salesforce’s scalable solution for testing AI agents using different utterances and topic matching.

Agentforce also integrates seamlessly with APIs like Slack, Google, and OpenAI, ensuring secure and robust AI automation within the Salesforce ecosystem.

Key Terminology in Agentforce Testing

Before diving into testing methodologies, let’s clarify the definitions of a few crucial key terms: 

  • Channels: Different communication mediums where AI agents operate, including email, WhatsApp, or experience sites.
  • Utterances: AI prompts that need to be processed and acted upon.
  • Topics: Categories that organize utterances and route them to appropriate actions.
  • Topic Actions: The actions performed based on identified topics, including API calls and database updates.
  • Atlas Reasoning Engine: The AI logic behind Agentforce, responsible for determining intent, topic matching, and action execution.

Challenges in Agentforce Testing

With new technologies come new challenges, and testing AI-driven agents presents a unique set of challenges compared to traditional Salesforce testing. Some of these new challenges include: 

1. Data Validation

Ensuring that AI agents process and validate data accurately is critical. This involves verifying:

  • Data completeness and quality.
  • Relevance of data to the given AI prompt.

2. Scalability Testing

AI agents must handle high traffic and large volumes of data efficiently. Testing should include:

  • High-volume performance simulations.
  • Load testing to assess response accuracy under stress.

3. Goal Alignment & Hallucination Prevention

AI responses should align with intended outcomes — without unexpected deviations. Testing should include:

  • Evaluating AI responses for accuracy.
  • Detecting and mitigating hallucinations (incorrect AI-generated information).

4. Multi-Agent Interaction

Many AI systems work collaboratively. Testing ensures:

  • Agents interact correctly within different workflows.
  • Data integrity across multi-agent interactions.

5. Ethical Constraints & Compliance

AI must adhere to ethical guidelines, especially in regulated industries. Testing should verify:

  • Adherence to security and ethical constraints.
  • Prevention of bias and harmful behavior.

Agentforce Testing Methods

1. Manual Testing in Agent Builder

Agent Builder provides a low-code environment to create AI agents and manually test their responses. However, manual testing has limitations:

  • It is time-consuming and labor-intensive.
  • It is limited to predefined test cases.
  • It lacks the scalability needed for robust AI agent evaluation.

2. Agentforce Testing Center

Agentforce Testing Center introduces AI-driven test automation, enabling:

  • Single-turn validations: Tests individual utterances against expected responses.
  • Scalability: Conducts bulk testing in a controlled sandbox environment.
  • Automated topic and action matching: Ensures the AI agent routes requests correctly.

However, Testing Center primarily focuses on isolated utterances rather than full conversational flows, which limits its ability to assess real-world interactions.

3. End-to-End Testing with Provar

Provar offers a more comprehensive testing approach, addressing the limitations of manual testing and single-turn validations. Key features include:

  • Persona-Based Testing: Ensures AI agents function correctly for different user roles within Salesforce.
  • Multi-Channel Testing: Tests AI agent interactions across various communication channels.
  • Multi-Step Testing: Simulates complete AI-driven workflows rather than isolated responses.
  • Prompt Variation Testing: Evaluates AI agent accuracy across diverse user inputs.
  • Response Evaluation: Assesses whether AI-generated responses meet expected quality standards.
  • Conversation Evaluation: Tests AI agent behavior across full conversational interactions.

Agentforce Testing Center vs. Provar: Key Differences

FeatureAgentforce Testing CenterProvar
Testing ScopeSingle-turn validationsEnd-to-end testing
AutomationAI-generated test casesComprehensive test automation
ScalabilityHigh-scale batch testingMulti-step, real-world scenarios
Conversation TestingLimited to isolated promptsFull conversational evaluations
IntegrationFocused on AgentforceSupports Salesforce UI, APIs, and databases

Some common questions on Agentforce testing: 

Do Agentforce tests consume credits?

  • No, tests in Agent Builder are free. Charges apply only when an agent is deployed live.

How do you determine an AI agent’s end-to-end process?

  • It depends on integrations. If an agent only interacts with Salesforce, it’s not fully end-to-end. However, if it connects to external databases (ERP, Oracle, etc.), it qualifies as end-to-end. These processes should be mapped as you build your Agent workflows.

Is Provar testing exclusive to Salesforce?

  • No. Provar supports web applications and generic web pages, similar to open-source frameworks like Selenium.

Do test responses require exact words or general elements?

  • It depends on the test setup. Agentforce Testing Center uses AI-based validation, allowing a range of acceptable answers. Provar lets users define strict or flexible validation.

What’s the difference between Provar and Agentforce Testing Center?

  • Testing Center: Focuses on single-turn utterance validation.
  • Provar: Handles full conversation flows and end-to-end scenarios.

Should you test AI agents via API, UI, or SOQL?

  • APIs and SOQL are preferred for speed and reliability.
  • UI testing is useful for persona-based scenarios and layout validation.

How does Provar handle agent responses with multiple required inputs?

  • Provar supports both single-input and sequential-input testing to simulate real-world user interactions.

Does Provar support multiple connections in a single test case?

  • Yes, multiple Salesforce, UI, API, and database connections can be tested together.

How does Provar validate dynamic agent responses?

  • Provar provides dynamic waiting mechanisms for page loads and background processes to prevent false negatives.

Does Agentforce Testing support parallel execution?

How does Provar identify chat text properties?

  • Provar’s Test Builder extracts text values, HTML attributes, and visibility properties dynamically.

When will Agentforce Testing be available in Provar?

  • It is available now! You can already test Agentforce using current Provar features.

Conclusion

Comprehensive testing is essential to ensuring accuracy, reliability, and compliance in AI-driven agents. While Agentforce Testing Center provides scalable AI-generated test cases, it primarily focuses on single-turn validations. Provar’s capabilities extend further, enabling end-to-end automation, real-world scenario testing, and multi-channel validation.

To maximize the effectiveness of Agentforce, organizations can leverage both Testing Center and Provar, ensuring their AI solutions align with enterprise goals, enhance user experiences, and meet security and compliance standards.

To learn more about testing Agentforce and to see a live demo of Provar, check out our Testing Agentforce with Provar webinar, or book some time to chat with one of our test automation experts.