Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: adds ecosystem docs for UnstructuredLoader #653

Merged
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions docs/docs/ecosystem/unstructured.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Unstructured

This page covers how to use [Unstructured](https://unstructured.io) within LangChain.

## What is Unstructured?

Unstructured is an [open source](https://github.com/Unstructured-IO/unstructured) Python package
for extracting text from raw documents for use in machine learning applications. Currently,
Unstructured supports partitioning Word documents (in `.doc` or `.docx` format),
PowerPoints (in `.ppt` or `.pptx` format), PDFs, HTML files, images,
emails (in `.eml` or `.msg` format), epubs, markdown, and plain text files.
`unstructured` is a Python package and cannot be used directly with TS/JS, Unstructured
also maintains a [REST API](https://github.com/Unstructured-IO/unstructured-api) to support
pre-processing pipelines written in other programming languages. The endpoint for the
hosted Unstructured API is `https://api.unstructured.io/general/v0/general`, or you can run
the service locally using the instructions found
[here](https://github.com/Unstructured-IO/unstructured-api#dizzy-instructions-for-using-the-docker-image).

## Quick start

You can use Unstructured in`langchainjs` with the following code.
Replace the filename with the file you would like to process.
If you are running the container locally, switch the url to
`https://api.unstructured.io/general/v0/general`.

```typescript
import { UnstructuredLoader } from "langchain/document_loader";

const loader = new UnstructuredLoader(
"https://api.unstructured.io/general/v0/general",
"langchain/src/document_loaders/tests/example_data/example.txt"
);
const docs = await loader.load();
```

Stayed tuned for future updates, including functionality equivalent to
`UnstructuredDirectoryLoader` in `langchain`!.