如何流式传输工具调用

在流式上下文中调用工具时，消息块 将通过 .tool_call_chunks 属性填充 工具调用块 对象的列表。ToolCallChunk 包含工具的可选字符串字段 name、args 和 id，并包含一个可选的整数字段 index，该字段可用于将块连接在一起。字段是可选的，因为工具调用的部分可能跨越不同的块流式传输（例如，包含参数子字符串的块可能为工具名称和 ID 具有空值）。

由于消息块继承自其父消息类，因此包含工具调用块的 AIMessageChunk 也将包含 .tool_calls 和 .invalid_tool_calls 字段。这些字段是根据消息的工具调用块进行最佳努力解析的。

请注意，并非所有提供商目前都支持工具调用的流式传输。在开始之前，让我们定义工具和模型。

import { z } from "zod";
import { tool } from "@langchain/core/tools";
import { ChatOpenAI } from "@langchain/openai";

const addTool = tool(
  async (input) => {
    return input.a + input.b;
  },
  {
    name: "add",
    description: "Adds a and b.",
    schema: z.object({
      a: z.number(),
      b: z.number(),
    }),
  }
);

const multiplyTool = tool(
  async (input) => {
    return input.a * input.b;
  },
  {
    name: "multiply",
    description: "Multiplies a and b.",
    schema: z.object({
      a: z.number(),
      b: z.number(),
    }),
  }
);

const tools = [addTool, multiplyTool];

const model = new ChatOpenAI({
  model: "gpt-4o",
  temperature: 0,
});

const modelWithTools = model.bindTools(tools);

现在，让我们定义查询并流式传输输出

const query = "What is 3 * 12? Also, what is 11 + 49?";

const stream = await modelWithTools.stream(query);

for await (const chunk of stream) {
  console.log(chunk.tool_call_chunks);
}

[]
[
  {
    name: 'multiply',
    args: '',
    id: 'call_MdIlJL5CAYD7iz9gTm5lwWtJ',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: '{"a"',
    id: undefined,
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: ': 3, ',
    id: undefined,
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: '"b": 1',
    id: undefined,
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: '2}',
    id: undefined,
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'add',
    args: '',
    id: 'call_ihL9W6ylSRlYigrohe9SClmW',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: '{"a"',
    id: undefined,
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: ': 11,',
    id: undefined,
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: ' "b": ',
    id: undefined,
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: '49}',
    id: undefined,
    index: 1,
    type: 'tool_call_chunk'
  }
]
[]
[]

请注意，添加消息块将合并其相应的工具调用块。这是 LangChain 的各种 工具输出解析器 支持流式传输的原理。

例如，在下面，我们将累积工具调用块

import { concat } from "@langchain/core/utils/stream";

const stream = await modelWithTools.stream(query);

let gathered = undefined;

for await (const chunk of stream) {
  gathered = gathered !== undefined ? concat(gathered, chunk) : chunk;
  console.log(gathered.tool_call_chunks);
}

[]
[
  {
    name: 'multiply',
    args: '',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a"',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, ',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 1',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a"',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a": 11,',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a": 11, "b": ',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a": 11, "b": 49}',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a": 11, "b": 49}',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a": 11, "b": 49}',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]

最后，我们可以看到最终的聚合工具调用块包含完全收集的原始字符串值

console.log(typeof gathered.tool_call_chunks[0].args);

string

我们还可以看到最后完全解析的工具调用是一个对象

console.log(typeof gathered.tool_calls[0].args);

object