如何从工具中流式传输事件

先决条件

本指南假定您熟悉以下概念

• LangChain 工具
• 自定义工具
• 使用流事件
• 在自定义工具中访问 RunnableConfig

如果您有调用聊天模型、检索器或其他 runnables 的工具，您可能希望访问来自这些 runnables 的内部事件或使用其他属性配置它们。本指南向您展示如何正确手动传递参数，以便您可以使用 .streamEvents() 方法执行此操作。

兼容性

为了支持更广泛的 JavaScript 环境，默认情况下，基本的 LangChain 包不会自动将配置传播到子 runnables。这包括 .streamEvents() 所需的回调。这是您可能无法看到从自定义 runnables 或工具发出事件的常见原因。

您需要手动将 RunnableConfig 对象传播到子 runnable。有关如何手动传播配置的示例，请参阅下面 bar RunnableLambda 的实现。

本指南还需要 @langchain/core>=0.2.16。

假设您有一个自定义工具，它调用一个链，该链通过提示聊天模型仅返回 10 个单词来压缩其输入，然后反转输出。首先，以一种朴素的方式定义它

选择您的聊天模型

Groq

安装依赖项

提示

请参阅 此部分，了解有关安装集成包的通用说明.

npm

npm i @langchain/groq 

yarn

yarn add @langchain/groq 

pnpm

pnpm add @langchain/groq 

添加环境变量

GROQ_API_KEY=your-api-key

实例化模型

import { ChatGroq } from "@langchain/groq";

const model = new ChatGroq({
  model: "llama-3.3-70b-versatile",
  temperature: 0
});

OpenAI

安装依赖项

提示

请参阅 此部分，了解有关安装集成包的通用说明.

npm

npm i @langchain/openai 

yarn

yarn add @langchain/openai 

pnpm

pnpm add @langchain/openai 

添加环境变量

OPENAI_API_KEY=your-api-key

实例化模型

import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  model: "gpt-4o-mini",
  temperature: 0
});

Anthropic

安装依赖项

提示

请参阅 此部分，了解有关安装集成包的通用说明.

npm

npm i @langchain/anthropic 

yarn

yarn add @langchain/anthropic 

pnpm

pnpm add @langchain/anthropic 

添加环境变量

ANTHROPIC_API_KEY=your-api-key

实例化模型

import { ChatAnthropic } from "@langchain/anthropic";

const model = new ChatAnthropic({
  model: "claude-3-5-sonnet-20240620",
  temperature: 0
});

FireworksAI

安装依赖项

提示

请参阅 此部分，了解有关安装集成包的通用说明.

npm

npm i @langchain/community 

yarn

yarn add @langchain/community 

pnpm

pnpm add @langchain/community 

添加环境变量

FIREWORKS_API_KEY=your-api-key

实例化模型

import { ChatFireworks } from "@langchain/community/chat_models/fireworks";

const model = new ChatFireworks({
  model: "accounts/fireworks/models/llama-v3p1-70b-instruct",
  temperature: 0
});

MistralAI

安装依赖项

提示

请参阅 此部分，了解有关安装集成包的通用说明.

npm

npm i @langchain/mistralai 

yarn

yarn add @langchain/mistralai 

pnpm

pnpm add @langchain/mistralai 

添加环境变量

MISTRAL_API_KEY=your-api-key

实例化模型

import { ChatMistralAI } from "@langchain/mistralai";

const model = new ChatMistralAI({
  model: "mistral-large-latest",
  temperature: 0
});

VertexAI

安装依赖项

提示

请参阅 此部分，了解有关安装集成包的通用说明.

npm

npm i @langchain/google-vertexai 

yarn

yarn add @langchain/google-vertexai 

pnpm

pnpm add @langchain/google-vertexai 

添加环境变量

GOOGLE_APPLICATION_CREDENTIALS=credentials.json

实例化模型

import { ChatVertexAI } from "@langchain/google-vertexai";

const model = new ChatVertexAI({
  model: "gemini-1.5-flash",
  temperature: 0
});

import { ChatAnthropic } from "@langchain/anthropic";
const model = new ChatAnthropic({
  model: "claude-3-5-sonnet-20240620",
  temperature: 0,
});

import { z } from "zod";
import { tool } from "@langchain/core/tools";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";

const specialSummarizationTool = tool(
  async (input) => {
    const prompt = ChatPromptTemplate.fromTemplate(
      "You are an expert writer. Summarize the following text in 10 words or less:\n\n{long_text}"
    );
    const reverse = (x: string) => {
      return x.split("").reverse().join("");
    };
    const chain = prompt
      .pipe(model)
      .pipe(new StringOutputParser())
      .pipe(reverse);
    const summary = await chain.invoke({ long_text: input.long_text });
    return summary;
  },
  {
    name: "special_summarization_tool",
    description: "A tool that summarizes input text using advanced techniques.",
    schema: z.object({
      long_text: z.string(),
    }),
  }
);

直接调用该工具可以正常工作

const LONG_TEXT = `
NARRATOR:
(Black screen with text; The sound of buzzing bees can be heard)
According to all known laws of aviation, there is no way a bee should be able to fly. Its wings are too small to get its fat little body off the ground. The bee, of course, flies anyway because bees don't care what humans think is impossible.
BARRY BENSON:
(Barry is picking out a shirt)
Yellow, black. Yellow, black. Yellow, black. Yellow, black. Ooh, black and yellow! Let's shake it up a little.
JANET BENSON:
Barry! Breakfast is ready!
BARRY:
Coming! Hang on a second.`;

await specialSummarizationTool.invoke({ long_text: LONG_TEXT });

.yad noitaudarg rof tiftuo sesoohc yrraB ;scisyhp seifed eeB

但是，如果您想访问来自聊天模型的原始输出，而不是完整的工具，您可以尝试使用 .streamEvents() 方法并查找 on_chat_model_end 事件。以下是发生的情况

const stream = await specialSummarizationTool.streamEvents(
  { long_text: LONG_TEXT },
  { version: "v2" }
);

for await (const event of stream) {
  if (event.event === "on_chat_model_end") {
    // Never triggers!
    console.log(event);
  }
}

您会注意到，子 run 中没有发出任何聊天模型事件！

这是因为上面的示例没有将工具的配置对象传递到内部链中。要解决此问题，请重新定义您的工具以采用类型为 RunnableConfig 的特殊参数（有关更多详细信息，请参阅 本指南）。您还需要在执行内部链时将该参数传递到内部链中

const specialSummarizationToolWithConfig = tool(
  async (input, config) => {
    const prompt = ChatPromptTemplate.fromTemplate(
      "You are an expert writer. Summarize the following text in 10 words or less:\n\n{long_text}"
    );
    const reverse = (x: string) => {
      return x.split("").reverse().join("");
    };
    const chain = prompt
      .pipe(model)
      .pipe(new StringOutputParser())
      .pipe(reverse);
    // Pass the "config" object as an argument to any executed runnables
    const summary = await chain.invoke({ long_text: input.long_text }, config);
    return summary;
  },
  {
    name: "special_summarization_tool",
    description: "A tool that summarizes input text using advanced techniques.",
    schema: z.object({
      long_text: z.string(),
    }),
  }
);

现在，使用您的新工具尝试与之前相同的 .streamEvents() 调用

const stream = await specialSummarizationToolWithConfig.streamEvents(
  { long_text: LONG_TEXT },
  { version: "v2" }
);

for await (const event of stream) {
  if (event.event === "on_chat_model_end") {
    // Never triggers!
    console.log(event);
  }
}

{
  event: 'on_chat_model_end',
  data: {
    output: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'Bee defies physics; Barry chooses outfit for graduation day.',
      name: undefined,
      additional_kwargs: [Object],
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: [Object]
    },
    input: { messages: [Array] }
  },
  run_id: '27ac7b2e-591c-4adc-89ec-64d96e233ec8',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}

太棒了！这次发出了一个事件。

对于流式传输，如果可能，.streamEvents() 会自动在启用流式传输的情况下调用链中的内部 runnables，因此，如果您想要从聊天模型生成的令牌流，您可以简单地过滤以查找 on_chat_model_stream 事件，而无需进行其他更改

const stream = await specialSummarizationToolWithConfig.streamEvents(
  { long_text: LONG_TEXT },
  { version: "v2" }
);

for await (const event of stream) {
  if (event.event === "on_chat_model_stream") {
    // Never triggers!
    console.log(event);
  }
}

{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'Bee',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' def',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'ies physics',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ';',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' Barry',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' cho',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'oses outfit',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' for',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' graduation',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' day',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: '.',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}

自动传递配置（高级）

如果您使用过 LangGraph，您可能已经注意到您不需要在嵌套调用中传递配置。这是因为 LangGraph 利用了一个名为 async_hooks 的 API，该 API 在许多（但并非所有）环境中不受支持。

如果您希望启用自动配置传递，可以运行以下代码来全局导入和启用 AsyncLocalStorage

import { AsyncLocalStorageProviderSingleton } from "@langchain/core/singletons";
import { AsyncLocalStorage } from "async_hooks";

AsyncLocalStorageProviderSingleton.initializeGlobalInstance(
  new AsyncLocalStorage()
);

下一步

您现在已经了解了如何从工具内部流式传输事件。接下来，查看以下指南以了解有关使用工具的更多信息

• 将 运行时值传递给工具
• 将 工具结果传递回模型
• 分派自定义回调事件

您还可以查看工具调用的更多特定用途

• 构建 使用工具的链和代理
• 从模型中获取 结构化输出