← Home Actian VectorAI DB

How Actian Integration Works

Semantic search over NEPA documents: embeddings, K-NN search, and a local fallback when Actian isn’t available.

What it does

Actian VectorAI DB is our vector store. We turn document chunks into dense vectors (embeddings) and store them in Actian. When you type a query, we embed it the same way and ask Actian for the most similar chunks by meaning—not just keyword match. That powers semantic search and the “Ask the Archive” RAG answers.

Indexing (getting data into Actian)

When you click Index Documents on the Semantic Search page:

We load documents from our SQLite database (prioritising EIS → EA → CE).
Each document is split into chunks (600 characters, 80-character overlap).
Each chunk is sent to OpenAI text-embedding-3-small → we get a 1536‑dimensional vector.
We send those vectors plus metadata (project_id, agency, state, process_type) to Actian via the Cortex client: we create a collection (if needed) and batch upsert (id + vector + payload) into the collection nepa_chunks.

After indexing, the same chunks can be retrieved by semantic similarity in milliseconds.

Search (semantic similarity)

When you run a Semantic Search query:

1 Your query is embedded with the same OpenAI model.

2 We call Actian’s K-NN search: “return the top‑k vectors most similar to this query vector” (cosine similarity).

3 Optional filters (e.g. process_type = EIS, state = NV) restrict results to matching chunks.

4 Actian returns the top chunks with similarity scores; we show them in the UI.

So: your question → one vector → Actian finds the closest stored vectors → we show those passages. No keyword index is used for this path.

Ask the Archive (RAG)

When you use Ask the Archive:

We do the same as search: embed your question, run K-NN in Actian, get the top‑k chunks.
We build a prompt that says “answer only from these excerpts” and send it to Ollama (local LLM).
Ollama generates an answer grounded in those chunks—so we get a natural-language answer without letting the model invent facts.

Actian’s job is retrieval; Ollama’s job is synthesis. The document text in the prompt never leaves your machine.

If Actian isn’t available

We check once (with a short timeout) whether Actian’s gRPC port is reachable. If it isn’t—e.g. on Apple Silicon without the x86 Docker image, or in an air-gapped environment—we switch to a local numpy store:

Vectors are kept in memory (and persisted to disk under the index/ folder).
We do cosine similarity with a single matrix multiply (vectors are L2‑normalised).
The same “search” and “ask” APIs are used—callers can’t tell whether Actian or the fallback is in use.

So the app always supports semantic search; Actian is the preferred backend when it’s running.

Key files to look at

backend/intelligence/vector_store.py — Actian Cortex client and numpy fallback
backend/intelligence/embedder.py — OpenAI embedding calls
backend/api/semantic.py — /status, /search, /ask, /index endpoints
backend/config.py — ACTIAN_HOST, COLLECTION_NAME, EMBED_DIM, OPENAI_API_KEY

In one sentence

We embed NEPA chunks with OpenAI, store the vectors in Actian VectorAI DB (or a local numpy fallback), and use K-NN search to find the most relevant chunks for your query or for RAG with Ollama.