Building an Intelligent Document Indexing & Search System with OpenSearch Vector DB, and Ollama AI
Organisations today generate enormous volumes of documents — contracts, statements, reports, compliance evidence, engineering drawings, emails, and more. Traditional keyword search struggles to extract meaning from this information. Today’s users want answers and deep document insights, not just matching text.
By combining Spring Boot, OpenSearch Vector DB, and Ollama AI, you can build a modern document intelligence platform capable of:
- Indexing documents into vector embeddings
- Performing semantic and hybrid search
- Running Retrieval‑Augmented Generation (RAG)
- Delivering grounded, context‑aware answers
Architecture Overview
An intelligent document management system consists of four major components:
- Document ingestion → extract text and chunk it
- Embedding generation → use Ollama to convert text into vectors
- Vector indexing → store chunks in OpenSearch
- RAG query flow → retrieve relevant chunks and generate answers
This separation of concerns keeps the system modular, scalable, and easy to extend.
Application Configuration
Before interacting with Ollama and OpenSearch, the Spring Boot application needs to know where these services are running. The configuration below defines the chat model, embedding model, and OpenSearch connection.
Application configuration for Spring AI and OpenSearch
spring:
ai:
vectorstore:
opensearch:
uris: https://localhost:9200
username: admin
password: SecureP@ssword1
initialize-schema: true
index-name: docman-vector-index
dimensions: 768
similarity: cosinesimil
ollama:
base-url: http://localhost:11434
chat:
options:
model: llama3
embedding:
options:
model: nomic-embed-text
This sets up:
- llama3 for answering questions
- nomic‑embed‑text for generating embeddings
- OpenSearch as the vector store backend
Creating the Vector Store Bean
Spring AI provides a VectorStore abstraction that hides the complexity of embedding generation and OpenSearch indexing. The following bean wires OpenSearch and the embedding model together.
@Bean
public VectorStore vectorStore(OpenSearchClient client, EmbeddingModel embeddingModel) {
return OpenSearchVectorStore.builder(client, embeddingModel)
.index("docman-vector-index")
.initializeSchema(true)
.build();
}
This allows you to call vectorStore.add() or vectorStore.similaritySearch() anywhere in your application.
Indexing Documents into OpenSearch
When a document is uploaded, the system must:
- Extract text
- Split it into chunks
- Convert each chunk into embeddings
- Store them in OpenSearch with metadata
The service below performs this workflow.
Service that chunks text and indexes it into the vector store
@Service
public class DocumentIndexService {
private final VectorStore vectorStore;
public DocumentIndexService(VectorStore vectorStore) {
this.vectorStore = vectorStore;
}
public void index(String docId, String text) {
List<String> chunks = Chunker.split(text, 800);
List<Document> docs = chunks.stream()
.map(chunk -> new Document(chunk, Map.of("docId", docId)))
.toList();
vectorStore.add(docs);
}
}
Each chunk is stored with a docId so you can collapse results later and avoid returning dozens of fragments from the same document.
Implementing the RAG Query Flow
When a user asks a question, the system must:
- Embed the query
- Retrieve the most relevant chunks
- Build a structured prompt
- Ask the LLM to answer using only the retrieved context
The service below performs this end‑to‑end RAG flow.
RAG service that retrieves context and generates an AI‑powered answer
@Service
public class RagService {
private final VectorStore vectorStore;
private final ChatModel chatModel;
public RagService(VectorStore vectorStore, ChatModel chatModel) {
this.vectorStore = vectorStore;
this.chatModel = chatModel;
}
public String answer(String question) {
List<Document> results = vectorStore.similaritySearch(
SearchRequest.builder().query(question).topK(5).build()
);
String context = results.stream()
.map(Document::getText)
.collect(Collectors.joining("\n\n---\n\n"));
Prompt prompt = new Prompt(List.of(
new SystemMessage("Use ONLY the context to answer."),
new UserMessage("Context:\n" + context + "\n\nQuestion: " + question)
));
return chatModel.call(prompt).getResult().getOutput().getText();
}
}
This ensures the LLM remains grounded in the retrieved document content.
Hybrid Search for Better Accuracy
OpenSearch supports hybrid scoring, combining:
- BM25 keyword relevance
- Vector similarity
This is especially useful for technical, legal, and compliance documents where exact terminology matters.
Example hybrid search query in OpenSearch
{
"query": {
"hybrid": {
"queries": [
{ "match": { "content": "contract variation" }},
{ "knn": { "embedding": { "vector": [ ... ], "k": 10 }}}
]
}
}
}
Hybrid search often outperforms pure vector search in enterprise environments.
Document‑Level Result Collapsing
OpenSearch’s collapse feature can be used to avoid returning multiple chunks from the same document. This improves readability and user experience.
Collapse configuration to group results by document ID
"collapse": {
"field": "metadata.docId.keyword"
}
This ensures each document appears only once in the search results.
A reference implementation project
Below is a full working example and a reference implementation of an intelligent document management system. The project is built using open source technologies.
GitHub - kamranzafar/docman-ai: Reference implementation of an AI-enabled Document Management System
Final Thoughts
A Spring Boot application powered by OpenSearch and Ollama can deliver:
- Fast and accurate semantic search
- AI‑driven question answering
- Full control over data privacy
- A scalable foundation for enterprise document management
This architecture is ideal for industries like construction, engineering, legal, finance, and compliance — where documents are dense, technical, and mission‑critical.
If you want to extend this into a full document management platform, the next step is usually building a robust ingestion pipeline and workflows with PDF parsing, OCR, metadata extraction, and versioning.
Also published on Medium.