Basalt

Description

Basalt is designed to make your AI agents learn from every user interaction, capturing customer behavior to create a self-improving infrastructure. By analyzing conversations from the previous day, Basalt detects issues and sends one high-quality pull request (PR) every morning to enhance your AI assistant's capabilities.

The process begins by connecting your observability tools, such as Langfuse, Braintrust, or Datadog. Basalt automatically ingests every conversation your users had overnight. Using advanced analysis, it clusters conversations, identifies failure patterns, and pinpoints interactions that reveal quality gaps. For instance, if 34 conversations are affected by a specific error pattern, Basalt generates a targeted fix, which may include prompt updates or tool definitions, validated against your golden dataset before implementation.

One of the key advantages of Basalt is that it operates without manual annotation. It relies on real user behavior as the signal for improvement. Each morning, Basalt sends a testable PR to your repository, ensuring that any changes are based on actual performance data. This approach allows for continuous improvement without disrupting existing functionalities. Before any PR is opened, Basalt tests the proposed fix against your golden dataset, which consists of handpicked examples that define correct behavior. If any regression is detected, the PR is blocked, ensuring that only effective changes are merged.

Basalt's pricing model is straightforward: you only pay when you merge a PR. Each PR is free to review, allowing you to decide what changes to implement. This model is particularly beneficial for teams merging 20 or more PRs per month, as volume discounts are available. With Basalt, you can significantly accelerate your iteration cycles, making it an essential tool for teams looking to enhance their AI agents efficiently.

Basalt's Core Features

• Automated daily PR generation

• Real-time conversation analysis

• Error pattern detection

• Targeted fix generation

• Validation against golden dataset

• No manual annotation required

• Pay-per-merge pricing model

• Volume discounts for high usage

How to use Basalt?

1. 1

   Connect your observability tools to Basalt.

2. 2

   Allow Basalt to ingest user conversations automatically overnight.

3. 3

   Review the daily PR generated by Basalt each morning.

4. 4

   Test the proposed changes against your golden dataset.

5. 5

   Merge the PR if the changes are satisfactory.

6. 6

   Monitor the performance of your AI agent post-merge.

7. 7

   Repeat the process for continuous improvement.

Basalt's Use Cases

• Customer Support Improvement
• Error Detection
• Performance Optimization
• Automated Learning
• Cost Management

FAQ from Basalt

• What is Basalt?

  Basalt is a tool that captures customer behavior to enhance AI agents through automated learning from user interactions.

• How does Basalt improve AI agents?

  It analyzes conversations, detects issues, and generates daily pull requests to enhance agent performance.

• What tools can I connect to Basalt?

  You can connect observability tools like Langfuse, Braintrust, and Datadog.

• Is manual annotation required for Basalt?

  No, Basalt operates without manual annotation, relying on real user behavior.

• How does Basalt validate changes?

  Basalt tests proposed changes against a golden dataset before opening a pull request.

• What is the pricing model for Basalt?

  You only pay when you merge a pull request; each PR is free to review.

• Are there discounts for high usage?

  Yes, volume discounts are available for teams merging 20 or more PRs per month.

• How often does Basalt generate updates?

  Basalt generates updates automatically every morning based on previous day’s conversations.