跳至主要内容

回退

先决条件

本指南假设您熟悉以下概念

在使用语言模型时，您可能会遇到来自底层 API 的问题，例如速率限制或停机。随着您将 LLM 应用程序投入生产，拥有错误的应急方案变得越来越重要。这就是我们引入回退概念的原因。

至关重要的是，回退不仅可以应用于 LLM 级别，还可以应用于整个 Runnable 级别。这一点很重要，因为通常不同的模型需要不同的提示。因此，如果您的 OpenAI 调用失败，您不仅希望将相同的提示发送到 Anthropic - 您可能还希望使用例如不同的提示模板。

处理 LLM API 错误​

这可能是回退最常见的用例。对 LLM API 的请求可能会因多种原因而失败 - API 可能已停用，您可能已达到速率限制，或者可能存在其他问题。

重要提示：默认情况下，LangChain 的许多 LLM 包装器会捕获错误并重试。在使用回退时，您很可能希望关闭这些功能。否则，第一个包装器将继续重试，而不是失败。

提示

请参阅 此部分以获取有关安装集成软件包的一般说明.

npm install @langchain/anthropic @langchain/openai @langchain/core
import { ChatOpenAI } from "@langchain/openai";
import { ChatAnthropic } from "@langchain/anthropic";

// Use a fake model name that will always throw an error
const fakeOpenAIModel = new ChatOpenAI({
  model: "potato!",
  maxRetries: 0,
});

const anthropicModel = new ChatAnthropic({});

const modelWithFallback = fakeOpenAIModel.withFallbacks([anthropicModel]);

const result = await modelWithFallback.invoke("What is your name?");

console.log(result);

/*
  AIMessage {
    content: ' My name is Claude. I was created by Anthropic.',
    additional_kwargs: {}
  }
*/

API 参考

RunnableSequences 的回退​

我们还可以为序列本身是序列的序列创建回退。在这里，我们使用两个不同的模型来实现：ChatOpenAI 以及普通的 OpenAI（不使用聊天模型）。由于 OpenAI 不是聊天模型，因此您可能需要一个不同的提示。

import { ChatOpenAI, OpenAI } from "@langchain/openai";
import { StringOutputParser } from "@langchain/core/output_parsers";
import { ChatPromptTemplate, PromptTemplate } from "@langchain/core/prompts";

const chatPrompt = ChatPromptTemplate.fromMessages<{ animal: string }>([
  [
    "system",
    "You're a nice assistant who always includes a compliment in your response",
  ],
  ["human", "Why did the {animal} cross the road?"],
]);

// Use a fake model name that will always throw an error
const fakeOpenAIChatModel = new ChatOpenAI({
  model: "potato!",
  maxRetries: 0,
});

const prompt =
  PromptTemplate.fromTemplate(`Instructions: You should always include a compliment in your response.

Question: Why did the {animal} cross the road?

Answer:`);

const openAILLM = new OpenAI({});

const outputParser = new StringOutputParser();

const badChain = chatPrompt.pipe(fakeOpenAIChatModel).pipe(outputParser);

const goodChain = prompt.pipe(openAILLM).pipe(outputParser);

const chain = badChain.withFallbacks([goodChain]);

const result = await chain.invoke({
  animal: "dragon",
});

console.log(result);

/*
  I don't know, but I'm sure it was an impressive sight. You must have a great imagination to come up with such an interesting question!
*/

API 参考

处理长输入​

LLM 的一个主要限制因素是它们的上下文窗口。有时您可以在将提示发送到 LLM 之前计算并跟踪提示的长度，但在这种情况难以做到或很复杂的情况下，您可以回退到具有更长上下文长度的模型。

import { ChatOpenAI } from "@langchain/openai";

// Use a model with a shorter context window
const shorterLlm = new ChatOpenAI({
  model: "gpt-3.5-turbo",
  maxRetries: 0,
});

const longerLlm = new ChatOpenAI({
  model: "gpt-3.5-turbo-16k",
});

const modelWithFallback = shorterLlm.withFallbacks([longerLlm]);

const input = `What is the next number: ${"one, two, ".repeat(3000)}`;

try {
  await shorterLlm.invoke(input);
} catch (e) {
  // Length error
  console.log(e);
}

const result = await modelWithFallback.invoke(input);

console.log(result);

/*
  AIMessage {
    content: 'The next number is one.',
    name: undefined,
    additional_kwargs: { function_call: undefined }
  }
*/

API 参考

回退到更好的模型​

通常，我们要求模型以特定格式（如 JSON）输出格式。像 GPT-3.5 这样的模型可以很好地做到这一点，但有时会遇到困难。这自然会指向回退 - 我们可以尝试使用更快的更便宜的模型，但如果解析失败，我们可以使用 GPT-4。

import { z } from "zod";
import { OpenAI, ChatOpenAI } from "@langchain/openai";
import { PromptTemplate } from "@langchain/core/prompts";
import { StructuredOutputParser } from "@langchain/core/output_parsers";

const prompt = PromptTemplate.fromTemplate(
  `Return a JSON object containing the following value wrapped in an "input" key. Do not return anything else:\n{input}`
);

const badModel = new OpenAI({
  maxRetries: 0,
  model: "gpt-3.5-turbo-instruct",
});

const normalModel = new ChatOpenAI({
  model: "gpt-4",
});

const outputParser = StructuredOutputParser.fromZodSchema(
  z.object({
    input: z.string(),
  })
);

const badChain = prompt.pipe(badModel).pipe(outputParser);

const goodChain = prompt.pipe(normalModel).pipe(outputParser);

try {
  const result = await badChain.invoke({
    input: "testing0",
  });
} catch (e) {
  console.log(e);
  /*
  OutputParserException [Error]: Failed to parse. Text: "

  { "name" : " Testing0 ", "lastname" : " testing ", "fullname" : " testing ", "role" : " test ", "telephone" : "+1-555-555-555 ", "email" : " [email protected] ", "role" : " test ", "text" : " testing0 is different than testing ", "role" : " test ", "immediate_affected_version" : " 0.0.1 ", "immediate_version" : " 1.0.0 ", "leading_version" : " 1.0.0 ", "version" : " 1.0.0 ", "finger prick" : " no ", "finger prick" : " s ", "text" : " testing0 is different than testing ", "role" : " test ", "immediate_affected_version" : " 0.0.1 ", "immediate_version" : " 1.0.0 ", "leading_version" : " 1.0.0 ", "version" : " 1.0.0 ", "finger prick" :". Error: SyntaxError: Unexpected end of JSON input
*/
}

const chain = badChain.withFallbacks([goodChain]);

const result = await chain.invoke({
  input: "testing",
});

console.log(result);

/*
  { input: 'testing' }
*/

API 参考

此页面是否有帮助？


您也可以留下详细的反馈 在 GitHub 上.