Ollama 函数
危险
LangChain Ollama 集成包已正式支持工具调用。 点击此处查看文档。
LangChain 提供了一个实验性包装器,围绕通过 Ollama 本地运行的开源模型,使其具有与 OpenAI Functions 相同的 API。
请注意,更强大和更有能力的模型在处理复杂模式和/或多个函数时表现更佳。 以下示例使用 Mistral。
危险
这是一个实验性包装器,旨在为本身不支持工具调用的模型添加工具调用支持。 请谨慎使用。
设置
按照这些说明设置并运行本地 Ollama 实例。
初始化模型
您可以像初始化标准 ChatOllama
实例一样初始化此包装器
import { OllamaFunctions } from "@langchain/community/experimental/chat_models/ollama_functions";
const model = new OllamaFunctions({
temperature: 0.1,
model: "mistral",
});
传入函数
现在您可以像 OpenAI 一样传入函数
import { OllamaFunctions } from "@langchain/community/experimental/chat_models/ollama_functions";
import { HumanMessage } from "@langchain/core/messages";
const model = new OllamaFunctions({
temperature: 0.1,
model: "mistral",
}).bind({
functions: [
{
name: "get_current_weather",
description: "Get the current weather in a given location",
parameters: {
type: "object",
properties: {
location: {
type: "string",
description: "The city and state, e.g. San Francisco, CA",
},
unit: { type: "string", enum: ["celsius", "fahrenheit"] },
},
required: ["location"],
},
},
],
// You can set the `function_call` arg to force the model to use a function
function_call: {
name: "get_current_weather",
},
});
const response = await model.invoke([
new HumanMessage({
content: "What's the weather in Boston?",
}),
]);
console.log(response);
/*
AIMessage {
content: '',
additional_kwargs: {
function_call: {
name: 'get_current_weather',
arguments: '{"location":"Boston, MA","unit":"fahrenheit"}'
}
}
}
*/
API 参考
- OllamaFunctions 来自
@langchain/community/experimental/chat_models/ollama_functions
- HumanMessage 来自
@langchain/core/messages
用于提取
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
import { OllamaFunctions } from "@langchain/community/experimental/chat_models/ollama_functions";
import { PromptTemplate } from "@langchain/core/prompts";
import { JsonOutputFunctionsParser } from "@langchain/core/output_parsers/openai_functions";
const EXTRACTION_TEMPLATE = `Extract and save the relevant entities mentioned in the following passage together with their properties.
Passage:
{input}
`;
const prompt = PromptTemplate.fromTemplate(EXTRACTION_TEMPLATE);
// Use Zod for easier schema declaration
const schema = z.object({
people: z.array(
z.object({
name: z.string().describe("The name of a person"),
height: z.number().describe("The person's height"),
hairColor: z.optional(z.string()).describe("The person's hair color"),
})
),
});
const model = new OllamaFunctions({
temperature: 0.1,
model: "mistral",
}).bind({
functions: [
{
name: "information_extraction",
description: "Extracts the relevant information from the passage.",
parameters: {
type: "object",
properties: zodToJsonSchema(schema),
},
},
],
function_call: {
name: "information_extraction",
},
});
// Use a JsonOutputFunctionsParser to get the parsed JSON response directly.
const chain = prompt.pipe(model).pipe(new JsonOutputFunctionsParser());
const response = await chain.invoke({
input:
"Alex is 5 feet tall. Claudia is 1 foot taller than Alex and jumps higher than him. Claudia has orange hair and Alex is blonde.",
});
console.log(JSON.stringify(response, null, 2));
/*
{
"people": [
{
"name": "Alex",
"height": 5,
"hairColor": "blonde"
},
{
"name": "Claudia",
"height": {
"$num": 1,
"add": [
{
"name": "Alex",
"prop": "height"
}
]
},
"hairColor": "orange"
}
]
}
*/
API 参考
- OllamaFunctions 来自
@langchain/community/experimental/chat_models/ollama_functions
- PromptTemplate 来自
@langchain/core/prompts
- JsonOutputFunctionsParser 来自
@langchain/core/output_parsers/openai_functions
提示
您可以在此处查看此操作的简单 LangSmith 跟踪
自定义
在幕后,这使用 Ollama 的 JSON 模式将输出约束为 JSON,然后将工具模式作为 JSON 模式传递到提示中。
由于不同的模型有不同的优势,因此传入您自己的系统提示可能会有所帮助。 这是一个示例
import { OllamaFunctions } from "@langchain/community/experimental/chat_models/ollama_functions";
import { HumanMessage } from "@langchain/core/messages";
// Custom system prompt to format tools. You must encourage the model
// to wrap output in a JSON object with "tool" and "tool_input" properties.
const toolSystemPromptTemplate = `You have access to the following tools:
{tools}
To use a tool, respond with a JSON object with the following structure:
{{
"tool": <name of the called tool>,
"tool_input": <parameters for the tool matching the above JSON schema>
}}`;
const model = new OllamaFunctions({
temperature: 0.1,
model: "mistral",
toolSystemPromptTemplate,
}).bind({
functions: [
{
name: "get_current_weather",
description: "Get the current weather in a given location",
parameters: {
type: "object",
properties: {
location: {
type: "string",
description: "The city and state, e.g. San Francisco, CA",
},
unit: { type: "string", enum: ["celsius", "fahrenheit"] },
},
required: ["location"],
},
},
],
// You can set the `function_call` arg to force the model to use a function
function_call: {
name: "get_current_weather",
},
});
const response = await model.invoke([
new HumanMessage({
content: "What's the weather in Boston?",
}),
]);
console.log(response);
/*
AIMessage {
content: '',
additional_kwargs: {
function_call: {
name: 'get_current_weather',
arguments: '{"location":"Boston, MA","unit":"fahrenheit"}'
}
}
}
*/
API 参考
- OllamaFunctions 来自
@langchain/community/experimental/chat_models/ollama_functions
- HumanMessage 来自
@langchain/core/messages