跳至主要内容

文本摘要

一个常见的用例是希望总结长篇文档。这自然会遇到上下文窗口的限制。与问答不同，您不能只做一些语义搜索技巧来仅选择与问题最相关的文本块（因为在这种情况下，没有特定问题——您希望概括所有内容）。那么你该怎么办呢？

要开始，我们建议您查看摘要链，它以递归的方式解决这个问题。

示例​

以下是如何使用 RefineDocumentsChain 总结从 YouTube 视频加载的文档的示例

提示

查看 此部分以获取有关安装集成包的常规说明。

npm install @langchain/anthropic @langchain/core
import { loadSummarizationChain } from "langchain/chains";
import { SearchApiLoader } from "@langchain/community/document_loaders/web/searchapi";
import { TokenTextSplitter } from "@langchain/textsplitters";
import { PromptTemplate } from "@langchain/core/prompts";
import { ChatAnthropic } from "@langchain/anthropic";

const loader = new SearchApiLoader({
  engine: "youtube_transcripts",
  video_id: "WTOm65IZneg",
});

const docs = await loader.load();

const splitter = new TokenTextSplitter({
  chunkSize: 10000,
  chunkOverlap: 250,
});

const docsSummary = await splitter.splitDocuments(docs);

const llmSummary = new ChatAnthropic({
  model: "claude-3-sonnet-20240229",
  temperature: 0.3,
});

const summaryTemplate = `
You are an expert in summarizing YouTube videos.
Your goal is to create a summary of a podcast.
Below you find the transcript of a podcast:
--------
{text}
--------

The transcript of the podcast will also be used as the basis for a question and answer bot.
Provide some examples questions and answers that could be asked about the podcast. Make these questions very specific.

Total output will be a summary of the video and a list of example questions the user could ask of the video.

SUMMARY AND QUESTIONS:
`;

const SUMMARY_PROMPT = PromptTemplate.fromTemplate(summaryTemplate);

const summaryRefineTemplate = `
You are an expert in summarizing YouTube videos.
Your goal is to create a summary of a podcast.
We have provided an existing summary up to a certain point: {existing_answer}

Below you find the transcript of a podcast:
--------
{text}
--------

Given the new context, refine the summary and example questions.
The transcript of the podcast will also be used as the basis for a question and answer bot.
Provide some examples questions and answers that could be asked about the podcast. Make
these questions very specific.
If the context isn't useful, return the original summary and questions.
Total output will be a summary of the video and a list of example questions the user could ask of the video.

SUMMARY AND QUESTIONS:
`;

const SUMMARY_REFINE_PROMPT = PromptTemplate.fromTemplate(
  summaryRefineTemplate
);

const summarizeChain = loadSummarizationChain(llmSummary, {
  type: "refine",
  verbose: true,
  questionPrompt: SUMMARY_PROMPT,
  refinePrompt: SUMMARY_REFINE_PROMPT,
});

const summary = await summarizeChain.run(docsSummary);

console.log(summary);

/*
  Here is a summary of the key points from the podcast transcript:

  - Jimmy helps provide hearing aids and cochlear implants to deaf and hard-of-hearing people who can't afford them. He helps over 1,000 people hear again.

  - Jimmy surprises recipients with $10,000 cash gifts in addition to the hearing aids. He also gifts things like jet skis, basketball game tickets, and trips to concerts.

  - Jimmy travels internationally to provide hearing aids, visiting places like Mexico, Guatemala, Brazil, South Africa, Malawi, and Indonesia. 

  - Jimmy donates $100,000 to organizations around the world that teach sign language.

  - The recipients are very emotional and grateful to be able to hear their loved ones again.

  Here are some example questions and answers about the podcast:

  Q: How many people did Jimmy help regain their hearing?
  A: Jimmy helped over 1,000 people regain their hearing.

  Q: What types of hearing devices did Jimmy provide to the recipients?
  A: Jimmy provided cutting-edge hearing aids and cochlear implants.

  Q: In addition to the hearing devices, what surprise gifts did Jimmy give some recipients?
  A: In addition to hearing devices, Jimmy surprised some recipients with $10,000 cash gifts, jet skis, basketball game tickets, and concert tickets.

  Q: What countries did Jimmy travel to in order to help people?
  A: Jimmy traveled to places like Mexico, Guatemala, Brazil, South Africa, Malawi, and Indonesia.

  Q: How much money did Jimmy donate to organizations that teach sign language?
  A: Jimmy donated $100,000 to sign language organizations around the world.

  Q: How did the recipients react when they were able to hear again?
  A: The recipients were very emotional and grateful, with many crying tears of joy at being able to hear their loved ones again.
*/

API 参考

此页面是否有帮助？


您也可以留下详细的反馈 在 GitHub 上.