What Is a Vector Database? The Search Engine Behind AI Chatbots

By the LaunchLen Web Solutions Team · Kitchener, Ontario · Last updated: October 17, 2026

When an AI chatbot answers from your own documents, something has to find the right passage first. In many systems, that something is a vector database. So what is a vector database? It’s a database built to store and search information by meaning rather than by exact words.

If you’ve read our guide to retrieval-augmented generation (RAG), this is the “retrieval” part up close. Here’s how it works in plain English, and what to think about if you’re choosing one.

Why keyword search isn’t enough for AI

Traditional search matches words. Search a help centre for “refund” and it finds pages containing “refund”. That works well until people use different words for the same thing.

A customer might ask:

• “Can I get my money back?”
• “What’s your return policy?”
• “I want to cancel and be reimbursed.”

All three mean roughly the same thing, but they share few keywords. A keyword search might miss the right policy page entirely, and an AI chatbot is only as good as the passages it’s handed. AI systems need a way to find text that’s similar in meaning, even when the wording is different. That’s what embeddings and vector search provide.

Embeddings in plain English

An embedding is a list of numbers that represents the meaning of a piece of text. An AI model called an embedding model reads a sentence or paragraph and turns it into that list, which is called a vector.

The useful part is that texts with similar meanings end up with similar numbers. Picture a huge map where every passage of text is a dot. “Money back”, “refund” and “return policy” land close together. “Parking at our Kitchener office” lands far away from them.

Real embeddings have hundreds or thousands of dimensions, not two, but the idea is the same: closeness on the map means closeness in meaning. Embeddings can also represent images and other content, though text is the most common business use.

How vector search works

A vector search follows a simple pattern:

1. Prepare the content. Your documents are split into short passages, and each passage is turned into an embedding.
2. Store it. The embeddings are saved in a vector database, along with the original text and useful details such as the document title, date or who’s allowed to see it.
3. Embed the question. When someone asks a question, it’s turned into an embedding using the same model.
4. Find the nearest neighbours. The database finds the stored vectors closest to the question’s vector, meaning the passages most similar in meaning.
5. Return the results. It sends back the top matches, often with a similarity score.

With thousands or millions of passages, comparing a question against every one would be slow. Vector databases use specialized indexes that find close matches quickly, usually trading a small amount of precision for much faster results.

Many systems also combine vector search with traditional keyword search, sometimes called hybrid search. Keywords help with exact terms like product codes or names, while vectors handle meaning.

Where a vector database fits in a RAG system

In a RAG chatbot, the vector database sits between your content and the AI model:

• Your approved content is embedded and stored ahead of time.
• When a customer or staff member asks a question, the system searches the vector database for the most relevant passages.
• Those passages are passed to the AI model, which writes an answer based on them, ideally with a link to the source.

The vector database doesn’t write answers. It finds the right material. If it retrieves the wrong passage, even a strong AI model can give a wrong answer. That’s why content quality, sensible splitting of documents and testing with real questions matter so much.

Access control matters too. If some documents are for staff only, the search needs to respect that, so a public chatbot never retrieves internal material. Security guidance from OWASP lists weak access controls in vector and embedding systems as a risk to plan for.

Choosing one: managed, built-in or self-hosted

There are three broad approaches:

• Managed services. A provider runs the vector database for you as a cloud service. Less to maintain, but your data lives with that provider, so check their terms, security and data locations.
• Built-in options. Many existing databases and search platforms now offer vector search as a feature or extension, which can keep vectors alongside the rest of your data.
• Self-hosted. Open-source vector databases can run on infrastructure you control. More control, but more responsibility for updates, backups and security.

As industry examples only, not an endorsement or a description of what we use: pgvector adds vector search to PostgreSQL, Pinecone is a managed vector database service, and Qdrant is an open-source vector database.

Questions worth asking for any option:

• How much content will you store, and how often does it change?
• Do you need keyword and vector search together?
• Where is the data stored, and does that meet your obligations? Our guide to Canadian data residency covers what to ask.
• Can it filter results by permissions, so people only see what they’re allowed to?
• Who will maintain it?

When we help with AI projects, the work can include reviewing your content, agreeing the scope and deliverables with you, and testing retrieval with real questions before anything goes live.

FAQ

Do I need a vector database to use an AI chatbot?

Not necessarily. A chatbot with a small, fixed set of information may not need one. A vector database becomes useful when the AI has to search a larger or changing collection of documents to find the right answer.

Is a vector database the same as a regular database?

No. A regular database is designed for exact lookups, like finding an order by its number. A vector database is designed to find items that are similar in meaning. Some regular databases now include vector search as well.

Does a vector database store my documents?

Usually it stores the embeddings plus the original text passages and some details about them, so the right passage can be returned. That means it holds real business content and should be protected like any other system with sensitive data.

Let’s work together on your project

Thinking about an AI chatbot or assistant that answers from your own content? We’re LaunchLen Web Solutions Inc., based in Kitchener, serving Waterloo Region and businesses across Ontario. Our AI Solutions page explains how we approach AI projects, and we’re happy to talk through your goals with you. Book a free consultation to get started.

Sources

• Google for Developers, Machine Learning Crash Course: Embeddings: https://developers.google.com/machine-learning/crash-course/embeddings
• AWS, What is a vector database?: https://aws.amazon.com/what-is/vector-databases/
• IBM, What is a vector database?: https://www.ibm.com/think/topics/vector-database
• OWASP GenAI Security Project, LLM08:2025 Vector and Embedding Weaknesses: https://genai.owasp.org/llmrisk/llm082025-vector-and-embedding-weaknesses/
• Industry examples (labelled in copy, not endorsements): pgvector, “Open-source vector similarity search for Postgres”: https://github.com/pgvector/pgvector · Pinecone documentation: https://docs.pinecone.io/guides/get-started/overview · Qdrant (open-source repository, “Vector Database and Vector Search Engine”; also offered as a cloud service): https://github.com/qdrant/qdrant