ChatOllama

Ollama 允许您在本地运行开源大型语言模型，例如 Llama 3.1。

Ollama 将模型权重、配置和数据捆绑到一个单一包中，由 Modelfile 定义。它优化了设置和配置细节，包括 GPU 使用。

本指南将帮助您开始使用 ChatOllama 聊天模型。有关所有 ChatOllama 功能和配置的详细文档，请访问 API 参考。

概述

集成详细信息

Ollama 允许您使用各种具有不同功能的模型。以下详细信息表中的某些字段仅适用于 Ollama 提供的模型子集。

有关支持的模型和模型变体的完整列表，请查看 Ollama 模型库，并按标签进行搜索。

类	包	本地	可序列化	PY 支持	包下载	包最新
ChatOllama	`@langchain/ollama`	✅	beta	✅

模型功能

请参阅以下表头中的链接，了解有关如何使用特定功能的指南。

工具调用	结构化输出	JSON 模式	图像输入	音频输入	视频输入	令牌级流式传输	令牌使用	Logprobs
✅	✅	✅	✅	❌	❌	✅	✅	❌

设置

按照这些说明设置并运行本地 Ollama 实例。然后，下载 @langchain/ollama 包。

凭据

如果您希望获得模型调用的自动跟踪，也可以通过取消以下注释来设置您的 LangSmith API 密钥

# export LANGCHAIN_TRACING_V2="true"
# export LANGCHAIN_API_KEY="your-api-key"

安装

LangChain ChatOllama 集成位于 @langchain/ollama 包中

提示

请参阅此部分，了解有关安装集成包的常规说明。

npm
yarn
pnpm

npm i @langchain/ollama

yarn add @langchain/ollama

pnpm add @langchain/ollama

实例化

现在我们可以实例化我们的模型对象并生成聊天完成

import { ChatOllama } from "@langchain/ollama";

const llm = new ChatOllama({
  model: "llama3",
  temperature: 0,
  maxRetries: 2,
  // other params...
});

调用

const aiMsg = await llm.invoke([
  [
    "system",
    "You are a helpful assistant that translates English to French. Translate the user sentence.",
  ],
  ["human", "I love programming."],
]);
aiMsg;

AIMessage {
  "content": "Je adore le programmation.\n\n(Note: \"programmation\" is the feminine form of the noun in French, but if you want to use the masculine form, it would be \"le programme\" instead.)",
  "additional_kwargs": {},
  "response_metadata": {
    "model": "llama3",
    "created_at": "2024-08-01T16:59:17.359302Z",
    "done_reason": "stop",
    "done": true,
    "total_duration": 6399311167,
    "load_duration": 5575776417,
    "prompt_eval_count": 35,
    "prompt_eval_duration": 110053000,
    "eval_count": 43,
    "eval_duration": 711744000
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 35,
    "output_tokens": 43,
    "total_tokens": 78
  }
}

console.log(aiMsg.content);

Je adore le programmation.

(Note: "programmation" is the feminine form of the noun in French, but if you want to use the masculine form, it would be "le programme" instead.)

链接

我们可以像这样使用提示模板将我们的模型链接

import { ChatPromptTemplate } from "@langchain/core/prompts";

const prompt = ChatPromptTemplate.fromMessages([
  [
    "system",
    "You are a helpful assistant that translates {input_language} to {output_language}.",
  ],
  ["human", "{input}"],
]);

const chain = prompt.pipe(llm);
await chain.invoke({
  input_language: "English",
  output_language: "German",
  input: "I love programming.",
});

AIMessage {
  "content": "Ich liebe Programmieren!\n\n(Note: \"Ich liebe\" means \"I love\", \"Programmieren\" is the verb for \"programming\")",
  "additional_kwargs": {},
  "response_metadata": {
    "model": "llama3",
    "created_at": "2024-08-01T16:59:18.088423Z",
    "done_reason": "stop",
    "done": true,
    "total_duration": 585146125,
    "load_duration": 27557166,
    "prompt_eval_count": 30,
    "prompt_eval_duration": 74241000,
    "eval_count": 29,
    "eval_duration": 481195000
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 30,
    "output_tokens": 29,
    "total_tokens": 59
  }
}

工具

Ollama 现在支持本地工具调用适用于其可用模型的子集。以下示例演示了如何从 Ollama 模型调用工具。

import { tool } from "@langchain/core/tools";
import { ChatOllama } from "@langchain/ollama";
import { z } from "zod";

const weatherTool = tool((_) => "Da weather is weatherin", {
  name: "get_current_weather",
  description: "Get the current weather in a given location",
  schema: z.object({
    location: z.string().describe("The city and state, e.g. San Francisco, CA"),
  }),
});

// Define the model
const llmForTool = new ChatOllama({
  model: "llama3-groq-tool-use",
});

// Bind the tool to the model
const llmWithTools = llmForTool.bindTools([weatherTool]);

const resultFromTool = await llmWithTools.invoke(
  "What's the weather like today in San Francisco? Ensure you use the 'get_current_weather' tool."
);

console.log(resultFromTool);

AIMessage {
  "content": "",
  "additional_kwargs": {},
  "response_metadata": {
    "model": "llama3-groq-tool-use",
    "created_at": "2024-08-01T18:43:13.2181Z",
    "done_reason": "stop",
    "done": true,
    "total_duration": 2311023875,
    "load_duration": 1560670292,
    "prompt_eval_count": 177,
    "prompt_eval_duration": 263603000,
    "eval_count": 30,
    "eval_duration": 485582000
  },
  "tool_calls": [
    {
      "name": "get_current_weather",
      "args": {
        "location": "San Francisco, CA"
      },
      "id": "c7a9d590-99ad-42af-9996-41b90efcf827",
      "type": "tool_call"
    }
  ],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 177,
    "output_tokens": 30,
    "total_tokens": 207
  }
}

`.withStructuredOutput`

对于支持工具调用的模型，您还可以调用 .withStructuredOutput() 以从工具获取结构化输出。

import { ChatOllama } from "@langchain/ollama";
import { z } from "zod";

// Define the model
const llmForWSO = new ChatOllama({
  model: "llama3-groq-tool-use",
});

// Define the tool schema you'd like the model to use.
const schemaForWSO = z.object({
  location: z.string().describe("The city and state, e.g. San Francisco, CA"),
});

// Pass the schema to the withStructuredOutput method to bind it to the model.
const llmWithStructuredOutput = llmForWSO.withStructuredOutput(schemaForWSO, {
  name: "get_current_weather",
});

const resultFromWSO = await llmWithStructuredOutput.invoke(
  "What's the weather like today in San Francisco? Ensure you use the 'get_current_weather' tool."
);
console.log(resultFromWSO);

{ location: 'San Francisco, CA' }

JSON 模式

Ollama 还支持适用于所有聊天模型的 JSON 模式，该模式强制模型输出仅返回 JSON。以下是如何使用此功能进行提取的示例

import { ChatOllama } from "@langchain/ollama";
import { ChatPromptTemplate } from "@langchain/core/prompts";

const promptForJsonMode = ChatPromptTemplate.fromMessages([
  [
    "system",
    `You are an expert translator. Format all responses as JSON objects with two keys: "original" and "translated".`,
  ],
  ["human", `Translate "{input}" into {language}.`],
]);

const llmJsonMode = new ChatOllama({
  baseUrl: "http://localhost:11434", // Default value
  model: "llama3",
  format: "json",
});

const chainForJsonMode = promptForJsonMode.pipe(llmJsonMode);

const resultFromJsonMode = await chainForJsonMode.invoke({
  input: "I love programming",
  language: "German",
});

console.log(resultFromJsonMode);

AIMessage {
  "content": "{\n\"original\": \"I love programming\",\n\"translated\": \"Ich liebe Programmierung\"\n}",
  "additional_kwargs": {},
  "response_metadata": {
    "model": "llama3",
    "created_at": "2024-08-01T17:24:54.35568Z",
    "done_reason": "stop",
    "done": true,
    "total_duration": 1754811583,
    "load_duration": 1297200208,
    "prompt_eval_count": 47,
    "prompt_eval_duration": 128532000,
    "eval_count": 20,
    "eval_duration": 318519000
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 47,
    "output_tokens": 20,
    "total_tokens": 67
  }
}

多模态模型

Ollama 支持开源多模态模型，例如 LLaVA（在版本 0.1.15 及更高版本中）。您可以将图像作为消息 content 字段的一部分传递给像这样支持多模态的模型

import { ChatOllama } from "@langchain/ollama";
import { HumanMessage } from "@langchain/core/messages";
import * as fs from "node:fs/promises";

const imageData = await fs.readFile("../../../../../examples/hotdog.jpg");
const llmForMultiModal = new ChatOllama({
  model: "llava",
  baseUrl: "http://127.0.0.1:11434",
});
const multiModalRes = await llmForMultiModal.invoke([
  new HumanMessage({
    content: [
      {
        type: "text",
        text: "What is in this image?",
      },
      {
        type: "image_url",
        image_url: `data:image/jpeg;base64,${imageData.toString("base64")}`,
      },
    ],
  }),
]);
console.log(multiModalRes);

AIMessage {
  "content": " The image shows a hot dog in a bun, which appears to be a footlong. It has been cooked or grilled to the point where it's browned and possibly has some blackened edges, indicating it might be slightly overcooked. Accompanying the hot dog is a bun that looks toasted as well. There are visible char marks on both the hot dog and the bun, suggesting they have been cooked directly over a source of heat, such as a grill or broiler. The background is white, which puts the focus entirely on the hot dog and its bun. ",
  "additional_kwargs": {},
  "response_metadata": {
    "model": "llava",
    "created_at": "2024-08-01T17:25:02.169957Z",
    "done_reason": "stop",
    "done": true,
    "total_duration": 5700249458,
    "load_duration": 2543040666,
    "prompt_eval_count": 1,
    "prompt_eval_duration": 1032591000,
    "eval_count": 127,
    "eval_duration": 2114201000
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 1,
    "output_tokens": 127,
    "total_tokens": 128
  }
}

API 参考

有关所有 ChatOllama 功能和配置的详细文档，请访问 API 参考：https://api.js.langchain.com/classes/langchain_ollama.ChatOllama.html

聊天模型概念指南
聊天模型操作指南

ChatOllama

概述

集成详细信息

模型功能

设置

凭据

安装

实例化

调用

链接

工具

`.withStructuredOutput`

JSON 模式

多模态模型

API 参考

此页面对您有帮助吗？

您也可以留下详细的反馈在 GitHub 上.

概述​

集成详细信息​

模型功能​

设置​

凭据​

安装​

实例化​

调用​

链接​

工具​

.withStructuredOutput​

JSON 模式​

多模态模型​

API 参考​

相关​

此页面对您有帮助吗？

您也可以留下详细的反馈 在 GitHub 上.

概述

集成详细信息

模型功能

设置

凭据

安装

实例化

调用

链接

工具

`.withStructuredOutput`

JSON 模式

多模态模型

API 参考

相关

您也可以留下详细的反馈在 GitHub 上.