如何从目录加载数据

这涵盖了如何加载目录中的所有文档。

第二个参数是文件扩展名到加载器工厂的映射。每个文件将被传递给匹配的加载器，生成的文档将被连接在一起。

示例文件夹

src/document_loaders/example_data/example/
├── example.json
├── example.jsonl
├── example.txt
└── example.csv

示例代码

import { DirectoryLoader } from "langchain/document_loaders/fs/directory";
import {
  JSONLoader,
  JSONLinesLoader,
} from "langchain/document_loaders/fs/json";
import { TextLoader } from "langchain/document_loaders/fs/text";
import { CSVLoader } from "@langchain/community/document_loaders/fs/csv";

const loader = new DirectoryLoader(
  "src/document_loaders/example_data/example",
  {
    ".json": (path) => new JSONLoader(path, "/texts"),
    ".jsonl": (path) => new JSONLinesLoader(path, "/html"),
    ".txt": (path) => new TextLoader(path),
    ".csv": (path) => new CSVLoader(path, "text"),
  }
);
const docs = await loader.load();
console.log({ docs });