LangChain and Clay Put Self-Improving Agents Under the Microscope

The first wave of agent development focused on what an agent could do. The next wave is focused on whether the system can identify weak behavior, learn from production evidence, and improve without turning every failure into another manual patch.

LangChain NY Meetup: Building Agents with Agents brings that conversation to Midtown Manhattan on Monday, August 3, 2026. Speakers from LangChain and Clay will examine self-improving agents, durable memory updates, and the evaluation systems required to understand whether agent behavior is actually getting better.

The event is fully in person, will not be livestreamed, and requires an approved registration. The exact address is shared only with accepted attendees.

About Building Agents with Agents

Robert Xu of LangChain will discuss how the LangChain team built a self-improving agent through LangSmith Engine. The session will examine how production traces can become durable memory updates that affect later behavior.

Jeff Barg, Vyshu Khota, and Soroush Khadem will represent Clay in a discussion of the company’s agentic evaluation harness and the systems used to evaluate agents with other automated components.

The agenda includes presentations, Q&A, food, drinks, and a closing networking session. That format gives builders an opportunity to move from architecture into operating questions: what gets measured, which failures matter, and how a team can distinguish apparent improvement from reliable improvement.

Why Agent Evaluation Is Different

Traditional software testing begins with a useful expectation: given the same input and state, the system should produce the same output. Agent applications weaken that assumption.

Model responses can vary. Tool calls can fail in different ways. Context changes behavior. External systems return new information. A successful demonstration can hide weak performance across a broader set of tasks, users, and edge cases.

That does not make testing impossible. It changes what the test system must capture. Teams need representative task sets, production traces, failure categories, human judgment, and repeatable scoring. They also need to decide which changes are safe to promote and which apparent gains are merely the result of overfitting to a small benchmark.

Evaluation becomes part of the product architecture rather than a gate at the end.

From Traces to Durable Memory

Production traces show what an agent saw, which tools it chose, what it returned, and where the workflow broke. The interesting question is what happens next.

A team can review the failure and patch a prompt. It can add a rule, rewrite a tool description, or create another branch in the orchestration. Those interventions may solve one case while adding complexity that becomes difficult to understand.

The self-improving-agent thesis is more ambitious. It asks whether evidence from production can be converted into durable memory that changes future behavior without requiring a developer to hand-author every correction.

That introduces a new control problem. A memory update can preserve useful experience, but it can also encode a bad conclusion, an unrepresentative exception, or information that should expire. Teams need provenance, review, versioning, and rollback around memory just as they need those controls around code.

Building Agents With Agents

The event title captures a broader market direction. AI systems are increasingly being used not only to perform work, but to inspect, test, and refine other AI systems.

An evaluation agent can classify failures across thousands of traces. A synthetic user can probe a workflow before customers encounter it. A judge model can compare outputs against a rubric. Another component can propose a memory update or identify which tool description caused the wrong decision.

These systems create leverage, but they do not remove the need for human judgment. Automated evaluators can be inconsistent, biased toward surface fluency, or blind to business consequences. A result that scores well syntactically can still violate a customer’s expectation or create operational risk.

The strongest evaluation harness combines automated coverage with deliberate human review. Automation expands the number of cases a team can inspect. People define which failures matter.

The Operators Behind the Event

LangChain develops tooling for building, testing, deploying, and monitoring agent applications. LangSmith provides observability and evaluation infrastructure for teams operating those systems.

Clay has become a visible operator in the agentic workflow market, particularly across data enrichment and go-to-market use cases. Its participation gives the session a production perspective from a company applying agents inside complex business workflows where weak data or an incorrect action can travel quickly downstream.

Robert Xu is listed as Deployed Engineering Manager at LangChain. Clay’s speakers include Jeff Barg, Head of AI; Vyshu Khota, ML Engineer; and Soroush Khadem, Software Engineer.

The combination matters because agent reliability is not solved by one role. It sits across model behavior, data, product design, application engineering, observability, and the people accountable for the workflow.

Questions Builders Should Bring

The most useful meetup conversation will not be about whether agents can improve. It will be about what improvement means.

What is the unit of evaluation: a single answer, a completed task, a customer outcome, or the cost of intervention? Which production traces should become training or memory evidence? Who approves a durable change? How does the team detect regression in a behavior it was not measuring before?

Builders should also ask what must remain reversible. If an agent learns from a noisy week of production data, can the team explain the resulting behavior and return to a known state?

Those questions turn self-improvement from a feature claim into an operating discipline.

What This Signals

The agent market is moving from capability demonstrations toward lifecycle management. The durable advantage will not come only from building an agent that works once. It will come from creating an operating system that shows why it failed, what changed, and whether the next version is meaningfully better.

That makes evaluation, memory, observability, and rollback central infrastructure for the next generation of agent companies.

Event Details

• Official event: LangChain NY Meetup: Building Agents with Agents
• Date: Monday, August 3, 2026
• Time: 6:00 PM to 8:30 PM EDT
• Location: Midtown Manhattan; exact address shared after approval
• Format: In-person AI engineering meetup; no livestream
• Hosted by: LangChain Events and Amada
• Access: Registration requires approval
• Official page: Request to join on Luma

Speakers

• Robert Xu: Deployed Engineering Manager, LangChain
• Jeff Barg: Head of AI, Clay
• Vyshu Khota: ML Engineer, Clay
• Soroush Khadem: Software Engineer, Clay