如何从工具流式传输事件

先决条件

本指南假定您熟悉以下概念

• LangChain 工具
• 自定义工具
• 使用流式传输事件
• 在自定义工具中访问 RunnableConfig

如果您有调用聊天模型、检索器或其他 Runnable 的工具，您可能希望访问这些 Runnable 的内部事件或使用其他属性对其进行配置。本指南向您展示如何手动正确传递参数，以便您可以使用 .streamEvents() 方法执行此操作。

兼容性

为了支持更多种类的 JavaScript 环境，LangChain 基础包默认情况下不会自动将配置传播到子 Runnable。这包括 .streamEvents() 所需的回调。这是您可能无法看到从自定义 Runnable 或工具中发出事件的常见原因。

您需要手动将 RunnableConfig 对象传播到子 Runnable。有关如何手动传播配置的示例，请参阅下面 bar RunnableLambda 的实现。

本指南还要求 @langchain/core>=0.2.16。

假设您有一个自定义工具，它调用一个链，该链通过提示聊天模型仅返回 10 个单词来压缩其输入，然后反转输出。首先，以一种简单的方式定义它

选择您的聊天模型

OpenAI

安装依赖项

提示

参见 本节介绍有关安装集成包的一般说明.

npm

npm i @langchain/openai 

yarn

yarn add @langchain/openai 

pnpm

pnpm add @langchain/openai 

添加环境变量

OPENAI_API_KEY=your-api-key

实例化模型

import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  model: "gpt-4o-mini",
  temperature: 0
});

Anthropic

安装依赖项

提示

参见 本节介绍有关安装集成包的一般说明.

npm

npm i @langchain/anthropic 

yarn

yarn add @langchain/anthropic 

pnpm

pnpm add @langchain/anthropic 

添加环境变量

ANTHROPIC_API_KEY=your-api-key

实例化模型

import { ChatAnthropic } from "@langchain/anthropic";

const model = new ChatAnthropic({
  model: "claude-3-5-sonnet-20240620",
  temperature: 0
});

FireworksAI

安装依赖项

提示

参见 本节介绍有关安装集成包的一般说明.

npm

npm i @langchain/community 

yarn

yarn add @langchain/community 

pnpm

pnpm add @langchain/community 

添加环境变量

FIREWORKS_API_KEY=your-api-key

实例化模型

import { ChatFireworks } from "@langchain/community/chat_models/fireworks";

const model = new ChatFireworks({
  model: "accounts/fireworks/models/llama-v3p1-70b-instruct",
  temperature: 0
});

MistralAI

安装依赖项

提示

参见 本节介绍有关安装集成包的一般说明.

npm

npm i @langchain/mistralai 

yarn

yarn add @langchain/mistralai 

pnpm

pnpm add @langchain/mistralai 

添加环境变量

MISTRAL_API_KEY=your-api-key

实例化模型

import { ChatMistralAI } from "@langchain/mistralai";

const model = new ChatMistralAI({
  model: "mistral-large-latest",
  temperature: 0
});

Groq

安装依赖项

提示

参见 本节介绍有关安装集成包的一般说明.

npm

npm i @langchain/groq 

yarn

yarn add @langchain/groq 

pnpm

pnpm add @langchain/groq 

添加环境变量

GROQ_API_KEY=your-api-key

实例化模型

import { ChatGroq } from "@langchain/groq";

const model = new ChatGroq({
  model: "mixtral-8x7b-32768",
  temperature: 0
});

VertexAI

安装依赖项

提示

参见 本节介绍有关安装集成包的一般说明.

npm

npm i @langchain/google-vertexai 

yarn

yarn add @langchain/google-vertexai 

pnpm

pnpm add @langchain/google-vertexai 

添加环境变量

GOOGLE_APPLICATION_CREDENTIALS=credentials.json

实例化模型

import { ChatVertexAI } from "@langchain/google-vertexai";

const model = new ChatVertexAI({
  model: "gemini-1.5-flash",
  temperature: 0
});

import { ChatAnthropic } from "@langchain/anthropic";
const model = new ChatAnthropic({
  model: "claude-3-5-sonnet-20240620",
  temperature: 0,
});

import { z } from "zod";
import { tool } from "@langchain/core/tools";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";

const specialSummarizationTool = tool(
  async (input) => {
    const prompt = ChatPromptTemplate.fromTemplate(
      "You are an expert writer. Summarize the following text in 10 words or less:\n\n{long_text}"
    );
    const reverse = (x: string) => {
      return x.split("").reverse().join("");
    };
    const chain = prompt
      .pipe(model)
      .pipe(new StringOutputParser())
      .pipe(reverse);
    const summary = await chain.invoke({ long_text: input.long_text });
    return summary;
  },
  {
    name: "special_summarization_tool",
    description: "A tool that summarizes input text using advanced techniques.",
    schema: z.object({
      long_text: z.string(),
    }),
  }
);

直接调用工具可以正常工作

const LONG_TEXT = `
NARRATOR:
(Black screen with text; The sound of buzzing bees can be heard)
According to all known laws of aviation, there is no way a bee should be able to fly. Its wings are too small to get its fat little body off the ground. The bee, of course, flies anyway because bees don't care what humans think is impossible.
BARRY BENSON:
(Barry is picking out a shirt)
Yellow, black. Yellow, black. Yellow, black. Yellow, black. Ooh, black and yellow! Let's shake it up a little.
JANET BENSON:
Barry! Breakfast is ready!
BARRY:
Coming! Hang on a second.`;

await specialSummarizationTool.invoke({ long_text: LONG_TEXT });

.yad noitaudarg rof tiftuo sesoohc yrraB ;scisyhp seifed eeB

但是，如果您想访问聊天模型的原始输出而不是完整的工具，您可以尝试使用 .streamEvents() 方法并查找 on_chat_model_end 事件。以下是发生的情况

const stream = await specialSummarizationTool.streamEvents(
  { long_text: LONG_TEXT },
  { version: "v2" }
);

for await (const event of stream) {
  if (event.event === "on_chat_model_end") {
    // Never triggers!
    console.log(event);
  }
}

您会注意到，子运行中没有发出聊天模型事件！

这是因为上面的示例没有将工具的配置对象传递到内部链中。要解决此问题，请重新定义您的工具以接受一个类型为 RunnableConfig 的特殊参数（有关详细信息，请参阅 本指南）。您还需要在执行内部链时将该参数传递到内部链中

const specialSummarizationToolWithConfig = tool(
  async (input, config) => {
    const prompt = ChatPromptTemplate.fromTemplate(
      "You are an expert writer. Summarize the following text in 10 words or less:\n\n{long_text}"
    );
    const reverse = (x: string) => {
      return x.split("").reverse().join("");
    };
    const chain = prompt
      .pipe(model)
      .pipe(new StringOutputParser())
      .pipe(reverse);
    // Pass the "config" object as an argument to any executed runnables
    const summary = await chain.invoke({ long_text: input.long_text }, config);
    return summary;
  },
  {
    name: "special_summarization_tool",
    description: "A tool that summarizes input text using advanced techniques.",
    schema: z.object({
      long_text: z.string(),
    }),
  }
);

现在，尝试使用新的工具执行与之前相同的 .streamEvents() 调用

const stream = await specialSummarizationToolWithConfig.streamEvents(
  { long_text: LONG_TEXT },
  { version: "v2" }
);

for await (const event of stream) {
  if (event.event === "on_chat_model_end") {
    // Never triggers!
    console.log(event);
  }
}

{
  event: 'on_chat_model_end',
  data: {
    output: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'Bee defies physics; Barry chooses outfit for graduation day.',
      name: undefined,
      additional_kwargs: [Object],
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: [Object]
    },
    input: { messages: [Array] }
  },
  run_id: '27ac7b2e-591c-4adc-89ec-64d96e233ec8',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}

太棒了！这次发出了一个事件。

对于流式传输，.streamEvents() 会自动以启用流式传输的方式调用链中的内部 Runnable（如果可能），因此如果您想在聊天模型生成令牌时获得令牌流，您只需过滤以查找 on_chat_model_stream 事件而无需进行任何其他更改

const stream = await specialSummarizationToolWithConfig.streamEvents(
  { long_text: LONG_TEXT },
  { version: "v2" }
);

for await (const event of stream) {
  if (event.event === "on_chat_model_stream") {
    // Never triggers!
    console.log(event);
  }
}

{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'Bee',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' def',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'ies physics',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ';',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' Barry',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' cho',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'oses outfit',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' for',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' graduation',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' day',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: '.',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}

自动传递配置（高级）

如果您使用过 LangGraph，您可能已经注意到，您不需要在嵌套调用中传递配置。这是因为 LangGraph 利用了一个名为 async_hooks 的 API，该 API 在许多（但并非所有）环境中都不受支持。

如果您希望，可以通过运行以下代码来全局导入和启用 AsyncLocalStorage，从而启用自动配置传递

import { AsyncLocalStorageProviderSingleton } from "@langchain/core/singletons";
import { AsyncLocalStorage } from "async_hooks";

AsyncLocalStorageProviderSingleton.initializeGlobalInstance(
  new AsyncLocalStorage()
);

下一步

现在您已经了解了如何在工具中流式传输事件。接下来，查看以下指南，了解更多关于使用工具的信息

• 传递 运行时值到工具
• 传递 工具结果回模型
• 调度自定义回调事件

您还可以查看一些更具体的工具调用用例

• 构建 使用工具的链和代理
• 从模型获取 结构化输出