Ollama 函数

危险

LangChain Ollama 集成包正式支持工具调用。点击这里查看文档.

LangChain 为通过Ollama在本地运行的开源模型提供了一个实验性包装器，使其具有与 OpenAI 函数相同的 API。

请注意，更强大、更有效率的模型在使用复杂模式和/或多个函数时会表现更好。以下示例使用Mistral.

危险

这是一个实验性包装器，它试图将工具调用支持附加到那些本身不支持它的模型上。请谨慎使用。

设置

按照这些说明来设置和运行本地 Ollama 实例。

初始化模型

您可以按照初始化标准 ChatOllama 实例的方式来初始化此包装器

import { OllamaFunctions } from "@langchain/community/experimental/chat_models/ollama_functions";

const model = new OllamaFunctions({
  temperature: 0.1,
  model: "mistral",
});

传递函数

现在您可以像使用 OpenAI 一样传递函数

import { OllamaFunctions } from "@langchain/community/experimental/chat_models/ollama_functions";
import { HumanMessage } from "@langchain/core/messages";

const model = new OllamaFunctions({
  temperature: 0.1,
  model: "mistral",
}).bind({
  functions: [
    {
      name: "get_current_weather",
      description: "Get the current weather in a given location",
      parameters: {
        type: "object",
        properties: {
          location: {
            type: "string",
            description: "The city and state, e.g. San Francisco, CA",
          },
          unit: { type: "string", enum: ["celsius", "fahrenheit"] },
        },
        required: ["location"],
      },
    },
  ],
  // You can set the `function_call` arg to force the model to use a function
  function_call: {
    name: "get_current_weather",
  },
});

const response = await model.invoke([
  new HumanMessage({
    content: "What's the weather in Boston?",
  }),
]);

console.log(response);

/*
  AIMessage {
    content: '',
    additional_kwargs: {
      function_call: {
        name: 'get_current_weather',
        arguments: '{"location":"Boston, MA","unit":"fahrenheit"}'
      }
    }
  }
*/

API 参考

• OllamaFunctions 来自 @langchain/community/experimental/chat_models/ollama_functions
• HumanMessage 来自 @langchain/core/messages

用于提取

import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

import { OllamaFunctions } from "@langchain/community/experimental/chat_models/ollama_functions";
import { PromptTemplate } from "@langchain/core/prompts";
import { JsonOutputFunctionsParser } from "@langchain/core/output_parsers/openai_functions";

const EXTRACTION_TEMPLATE = `Extract and save the relevant entities mentioned in the following passage together with their properties.

Passage:
{input}
`;

const prompt = PromptTemplate.fromTemplate(EXTRACTION_TEMPLATE);

// Use Zod for easier schema declaration
const schema = z.object({
  people: z.array(
    z.object({
      name: z.string().describe("The name of a person"),
      height: z.number().describe("The person's height"),
      hairColor: z.optional(z.string()).describe("The person's hair color"),
    })
  ),
});

const model = new OllamaFunctions({
  temperature: 0.1,
  model: "mistral",
}).bind({
  functions: [
    {
      name: "information_extraction",
      description: "Extracts the relevant information from the passage.",
      parameters: {
        type: "object",
        properties: zodToJsonSchema(schema),
      },
    },
  ],
  function_call: {
    name: "information_extraction",
  },
});

// Use a JsonOutputFunctionsParser to get the parsed JSON response directly.
const chain = prompt.pipe(model).pipe(new JsonOutputFunctionsParser());

const response = await chain.invoke({
  input:
    "Alex is 5 feet tall. Claudia is 1 foot taller than Alex and jumps higher than him. Claudia has orange hair and Alex is blonde.",
});

console.log(JSON.stringify(response, null, 2));

/*
{
  "people": [
    {
      "name": "Alex",
      "height": 5,
      "hairColor": "blonde"
    },
    {
      "name": "Claudia",
      "height": {
        "$num": 1,
        "add": [
          {
            "name": "Alex",
            "prop": "height"
          }
        ]
      },
      "hairColor": "orange"
    }
  ]
}
*/

API 参考

• OllamaFunctions 来自 @langchain/community/experimental/chat_models/ollama_functions
• PromptTemplate 来自 @langchain/core/prompts
• JsonOutputFunctionsParser 来自 @langchain/core/output_parsers/openai_functions

提示

您可以在这里看到此功能的简单 LangSmith 跟踪

定制

在幕后，它使用 Ollama 的 JSON 模式将输出限制为 JSON，然后将工具模式作为 JSON 模式传递到提示中。

由于不同的模型具有不同的优势，因此传递您自己的系统提示可能会有所帮助。以下是一个示例

import { OllamaFunctions } from "@langchain/community/experimental/chat_models/ollama_functions";
import { HumanMessage } from "@langchain/core/messages";

// Custom system prompt to format tools. You must encourage the model
// to wrap output in a JSON object with "tool" and "tool_input" properties.
const toolSystemPromptTemplate = `You have access to the following tools:

{tools}

To use a tool, respond with a JSON object with the following structure:
{{
  "tool": <name of the called tool>,
  "tool_input": <parameters for the tool matching the above JSON schema>
}}`;

const model = new OllamaFunctions({
  temperature: 0.1,
  model: "mistral",
  toolSystemPromptTemplate,
}).bind({
  functions: [
    {
      name: "get_current_weather",
      description: "Get the current weather in a given location",
      parameters: {
        type: "object",
        properties: {
          location: {
            type: "string",
            description: "The city and state, e.g. San Francisco, CA",
          },
          unit: { type: "string", enum: ["celsius", "fahrenheit"] },
        },
        required: ["location"],
      },
    },
  ],
  // You can set the `function_call` arg to force the model to use a function
  function_call: {
    name: "get_current_weather",
  },
});

const response = await model.invoke([
  new HumanMessage({
    content: "What's the weather in Boston?",
  }),
]);

console.log(response);

/*
  AIMessage {
    content: '',
    additional_kwargs: {
      function_call: {
        name: 'get_current_weather',
        arguments: '{"location":"Boston, MA","unit":"fahrenheit"}'
      }
    }
  }
*/

API 参考

• OllamaFunctions 来自 @langchain/community/experimental/chat_models/ollama_functions
• HumanMessage 来自 @langchain/core/messages

相关

• 聊天模型概念指南
• 聊天模型操作指南