如何从工具流式传输事件

先决条件

本指南假定您熟悉以下概念

• LangChain 工具
• 自定义工具
• 使用流式事件
• 在自定义工具中访问 RunnableConfig

如果您有调用聊天模型、检索器或其他 Runnable 的工具，您可能希望访问这些 Runnable 的内部事件或使用其他属性对其进行配置。本指南向您展示如何手动正确传递参数，以便您可以使用.streamEvents() 方法来执行此操作。

兼容性

为了支持更广泛的 JavaScript 环境，基础 LangChain 包默认情况下不会自动将配置传播到子 Runnable。这包括.streamEvents() 所需的回调。这是您可能无法看到自定义 Runnable 或工具发出事件的常见原因。

您需要手动将RunnableConfig 对象传播到子 Runnable。有关如何手动传播配置的示例，请参见下面bar RunnableLambda 的实现。

本指南还需要@langchain/core>=0.2.16。

假设您有一个自定义工具，该工具调用一个链，该链通过提示聊天模型仅返回 10 个单词来压缩其输入，然后反转输出。首先，以一种简单的方式定义它

选择您的聊天模型

OpenAI

安装依赖项

提示

见 本节了解有关安装集成包的常规说明.

npm

npm i @langchain/openai 

yarn

yarn add @langchain/openai 

pnpm

pnpm add @langchain/openai 

添加环境变量

OPENAI_API_KEY=your-api-key

实例化模型

import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  model: "gpt-4o-mini",
  temperature: 0
});

Anthropic

安装依赖项

提示

见 本节了解有关安装集成包的常规说明.

npm

npm i @langchain/anthropic 

yarn

yarn add @langchain/anthropic 

pnpm

pnpm add @langchain/anthropic 

添加环境变量

ANTHROPIC_API_KEY=your-api-key

实例化模型

import { ChatAnthropic } from "@langchain/anthropic";

const model = new ChatAnthropic({
  model: "claude-3-5-sonnet-20240620",
  temperature: 0
});

FireworksAI

安装依赖项

提示

见 本节了解有关安装集成包的常规说明.

npm

npm i @langchain/community 

yarn

yarn add @langchain/community 

pnpm

pnpm add @langchain/community 

添加环境变量

FIREWORKS_API_KEY=your-api-key

实例化模型

import { ChatFireworks } from "@langchain/community/chat_models/fireworks";

const model = new ChatFireworks({
  model: "accounts/fireworks/models/llama-v3p1-70b-instruct",
  temperature: 0
});

MistralAI

安装依赖项

提示

见 本节了解有关安装集成包的常规说明.

npm

npm i @langchain/mistralai 

yarn

yarn add @langchain/mistralai 

pnpm

pnpm add @langchain/mistralai 

添加环境变量

MISTRAL_API_KEY=your-api-key

实例化模型

import { ChatMistralAI } from "@langchain/mistralai";

const model = new ChatMistralAI({
  model: "mistral-large-latest",
  temperature: 0
});

Groq

安装依赖项

提示

见 本节了解有关安装集成包的常规说明.

npm

npm i @langchain/groq 

yarn

yarn add @langchain/groq 

pnpm

pnpm add @langchain/groq 

添加环境变量

GROQ_API_KEY=your-api-key

实例化模型

import { ChatGroq } from "@langchain/groq";

const model = new ChatGroq({
  model: "mixtral-8x7b-32768",
  temperature: 0
});

VertexAI

安装依赖项

提示

见 本节了解有关安装集成包的常规说明.

npm

npm i @langchain/google-vertexai 

yarn

yarn add @langchain/google-vertexai 

pnpm

pnpm add @langchain/google-vertexai 

添加环境变量

GOOGLE_APPLICATION_CREDENTIALS=credentials.json

实例化模型

import { ChatVertexAI } from "@langchain/google-vertexai";

const model = new ChatVertexAI({
  model: "gemini-1.5-flash",
  temperature: 0
});

import { ChatAnthropic } from "@langchain/anthropic";
const model = new ChatAnthropic({
  model: "claude-3-5-sonnet-20240620",
  temperature: 0,
});

import { z } from "zod";
import { tool } from "@langchain/core/tools";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";

const specialSummarizationTool = tool(
  async (input) => {
    const prompt = ChatPromptTemplate.fromTemplate(
      "You are an expert writer. Summarize the following text in 10 words or less:\n\n{long_text}"
    );
    const reverse = (x: string) => {
      return x.split("").reverse().join("");
    };
    const chain = prompt
      .pipe(model)
      .pipe(new StringOutputParser())
      .pipe(reverse);
    const summary = await chain.invoke({ long_text: input.long_text });
    return summary;
  },
  {
    name: "special_summarization_tool",
    description: "A tool that summarizes input text using advanced techniques.",
    schema: z.object({
      long_text: z.string(),
    }),
  }
);

直接调用工具可以正常工作

const LONG_TEXT = `
NARRATOR:
(Black screen with text; The sound of buzzing bees can be heard)
According to all known laws of aviation, there is no way a bee should be able to fly. Its wings are too small to get its fat little body off the ground. The bee, of course, flies anyway because bees don't care what humans think is impossible.
BARRY BENSON:
(Barry is picking out a shirt)
Yellow, black. Yellow, black. Yellow, black. Yellow, black. Ooh, black and yellow! Let's shake it up a little.
JANET BENSON:
Barry! Breakfast is ready!
BARRY:
Coming! Hang on a second.`;

await specialSummarizationTool.invoke({ long_text: LONG_TEXT });

.yad noitaudarg rof tiftuo sesoohc yrraB ;scisyhp seifed eeB

但是，如果您想访问聊天模型的原始输出而不是完整的工具，您可能会尝试使用.streamEvents() 方法并查找on_chat_model_end 事件。以下是发生的情况

const stream = await specialSummarizationTool.streamEvents(
  { long_text: LONG_TEXT },
  { version: "v2" }
);

for await (const event of stream) {
  if (event.event === "on_chat_model_end") {
    // Never triggers!
    console.log(event);
  }
}

您会注意到，子运行没有发出聊天模型事件！

这是因为上面的示例没有将工具的配置对象传递到内部链中。要解决此问题，请重新定义您的工具以接受类型为RunnableConfig 的特殊参数（有关详细信息，请参见本指南）。您还需要在执行内部链时将该参数传递到内部链中

const specialSummarizationToolWithConfig = tool(
  async (input, config) => {
    const prompt = ChatPromptTemplate.fromTemplate(
      "You are an expert writer. Summarize the following text in 10 words or less:\n\n{long_text}"
    );
    const reverse = (x: string) => {
      return x.split("").reverse().join("");
    };
    const chain = prompt
      .pipe(model)
      .pipe(new StringOutputParser())
      .pipe(reverse);
    // Pass the "config" object as an argument to any executed runnables
    const summary = await chain.invoke({ long_text: input.long_text }, config);
    return summary;
  },
  {
    name: "special_summarization_tool",
    description: "A tool that summarizes input text using advanced techniques.",
    schema: z.object({
      long_text: z.string(),
    }),
  }
);

现在尝试与之前相同的.streamEvents() 调用，使用您的新工具

const stream = await specialSummarizationToolWithConfig.streamEvents(
  { long_text: LONG_TEXT },
  { version: "v2" }
);

for await (const event of stream) {
  if (event.event === "on_chat_model_end") {
    // Never triggers!
    console.log(event);
  }
}

{
  event: 'on_chat_model_end',
  data: {
    output: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'Bee defies physics; Barry chooses outfit for graduation day.',
      name: undefined,
      additional_kwargs: [Object],
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: [Object]
    },
    input: { messages: [Array] }
  },
  run_id: '27ac7b2e-591c-4adc-89ec-64d96e233ec8',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}

太棒了！这次发出了一个事件。

对于流式传输，.streamEvents() 会自动以流式传输模式调用链中的内部 Runnable（如果可能的话），因此，如果您想从聊天模型中生成代币流，您只需过滤以查找on_chat_model_stream 事件即可，无需进行其他更改

const stream = await specialSummarizationToolWithConfig.streamEvents(
  { long_text: LONG_TEXT },
  { version: "v2" }
);

for await (const event of stream) {
  if (event.event === "on_chat_model_stream") {
    // Never triggers!
    console.log(event);
  }
}

{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'Bee',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' def',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'ies physics',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ';',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' Barry',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' cho',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: 'oses outfit',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' for',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' graduation',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: ' day',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}
{
  event: 'on_chat_model_stream',
  data: {
    chunk: AIMessageChunk {
      lc_serializable: true,
      lc_kwargs: [Object],
      lc_namespace: [Array],
      content: '.',
      name: undefined,
      additional_kwargs: {},
      response_metadata: {},
      id: undefined,
      tool_calls: [],
      invalid_tool_calls: [],
      tool_call_chunks: [],
      usage_metadata: undefined
    }
  },
  run_id: '938c0469-83c6-4dbd-862e-cd73381165de',
  name: 'ChatAnthropic',
  tags: [ 'seq:step:2' ],
  metadata: {
    ls_provider: 'anthropic',
    ls_model_name: 'claude-3-5-sonnet-20240620',
    ls_model_type: 'chat',
    ls_temperature: 0,
    ls_max_tokens: 2048,
    ls_stop: undefined
  }
}

自动传递配置（高级）

如果您使用过 LangGraph，您可能已经注意到在嵌套调用中不需要传递配置。这是因为 LangGraph 利用了一个名为 async_hooks 的 API，该 API 在许多（但并非所有）环境中都不受支持。

如果您希望，可以通过运行以下代码来导入并全局启用 AsyncLocalStorage，从而启用自动配置传递

import { AsyncLocalStorageProviderSingleton } from "@langchain/core/singletons";
import { AsyncLocalStorage } from "async_hooks";

AsyncLocalStorageProviderSingleton.initializeGlobalInstance(
  new AsyncLocalStorage()
);

下一步

您现在已经了解了如何在工具内部流式传输事件。接下来，请查看以下指南，了解有关使用工具的更多信息

• 传递 运行时值到工具
• 将 工具结果传递回模型
• 调度自定义回调事件

您还可以查看工具调用的更多特定用例

• 构建 使用工具的链和代理
• 从模型获取 结构化输出