Home

The capabilities of LlamaParse, Reducto, and Unstructured.io — layout-aware parsing, schema-driven extraction, hybrid retrieval, and text-to-SQL — as one open-source Python library that runs entirely on your own stack. No document ever leaves your network.

The pipeline

Eight capabilities, one call each

• Parse

  Layout-aware markdown per page — tables as HTML, formulas as LaTeX, charts as data tables, figures as PNG crops with AI descriptions. Every block traces to a bounding box on the page.

• Classify

  A document-type label — open-ended or constrained to your own categories — with confidence and ranked alternatives. Works standalone, no OCR.

• Split

  Sections grouped by role, containing natural chunks: boundaries follow the content, tables never split, each chunk carries a context breadcrumb.

• Extract

  Your Pydantic schema, filled and cited — every field grounded against the source text with honest, verification-capped confidence.

• Ingest

  The whole pipeline in one call — queryable output to the registry, bytes to the artifact store, vectors upserted, deduplicated by content checksum.

• Retrieve

  Hybrid search (dense + lexical) → rerank → cited hits with scores and a prompt-ready context block.

• Query databases

  The same retrieve() also answers from your SQL databases — read-only generated SQL behind a permission boundary, merged with document results.

• Serve to agents

  ingestlib mcp exposes the whole loop as MCP tools — point Claude Desktop or Cursor at your self-hosted corpus.

Provenance

Every answer knows where it came from

Not just the document — the page, and the bounding-box regions on that page. That provenance chain, not just parsing quality, is what ingestlib is built around.

from ingestlib.services import retrieve

result = retrieve("what were the total revenues?")
for hit in result.hits:
    print(hit.citation, "→", hit.chunk.heading)
# doc 3f9c2ab81e04 · p.42 · financial_statements → Consolidated Revenues

Your stack, your choices

Everything pluggable, one config file

• AI providers — Amazon Bedrock (Nova), OpenAI (GPT-5), or a local Ollama server. Mix them: one for chat, another for embeddings.
• Eight vector stores — SQLite (zero setup, the default), Pinecone, Qdrant, Postgres/pgvector, MongoDB, Milvus, OpenSearch, Weaviate — all hybrid dense + lexical.
• Artifacts — AWS S3, a self-hosted MinIO, or a plain local folder. Registry — the built-in Postgres metadata hub that makes the whole corpus queryable.
• OCR — PaddleOCR-VL (0.9B), served from your own GPU.

Bedrock Nova OpenAI GPT-5 Ollama Pinecone Qdrant pgvector MongoDB Milvus OpenSearch Weaviate SQLite

Why self-hosted? Hosted parsing APIs mean your documents — contracts, filings, patient records — leave your network and meter by the page. ingestlib runs the same class of pipeline on infrastructure you control: your GPU for OCR, your provider (or a fully local Ollama) for the LLM, your vector store, your object storage. ~$0.002 a page in LLM spend, or nothing at all on the local stack.

---

Where to go

• New here?

  Install, run one document through the pipeline, and get a cited answer in about five minutes.

  Quickstart

• Keeping data in-house?

  LLM, embeddings, vectors, and artifacts all on your machine — no API keys, nothing leaves your network.

  Run fully local

• Building on top?

  Task-focused guides: your own categories, namespaces and filters, corpus lifecycle, and building a citations UI.

  How-to guides

• Looking something up?

  Every function, every config key, every CLI flag — with defaults.

  Reference