Vector Databases Get Attention, and Amazon Follows

The year 2023 saw a burst of interest in vector databases. That included a cluster of startup funding rounds but also many "regular" databases, like Oracle's, adding support for vector search — crucial for AI training and retrieval-augmented generation (RAG).

Two years later, with inference dominating the enterprise AI discussion, vector databases could get another wave of attention, because they're critical elements of an enterprise's AI inference strategy. For the startups specializing in vector databases, however, the elapsed time has given more established data players—such as Databricks and Snowflake—a chance to "discover" the technology and fold it into their platform-level stories.

Just last week, AWS joined the crowd as well with the introduction of Amazon S3 Vectors, now in preview—not a good sign for smaller vector database contenders.

Here's an update on what's happening from the standpoint of the tech vendors. We discuss vector databases and vector search at more depth in our recent report, RAGs to Riches: The New Era of AI Inferencing.

Inference Gives Vectors Life

The overall conversation around enterprise AI has shifted toward inference—deriving more meaningful results out of foundation models. One crucial step there is to infuse the enterprise's own data into queries or specialized models.

Much of that data, though, is unstructured. It's in the form of emails, documents, meeting minutes, and so on—not in ready-made numbers already found in a database.

Vectors are numerical representations of unstructured data. They're not simple digits; vectors can have hundreds or thousands of dimensions. The magic behind vector databases lies in not only storing these vectors, but also indexing them for quick searching. Indexing lets an agent search the database without having to search through every vector; without it, vector search, and therefore RAG, is not feasible at scale.

Follow the Money

In 2022, Zilliz, which offers the Milvus database, raised a $60 million extension to its $43 million Series B. That presaged a burst of funding for vector databases in 2023: Pinecone (a Futuriom 50 company), Qdrant, TileDB and Weaviate.

Around the same time, established database vendors added vector indexing and search to their products. For PostGreSQL databases, there's an obvious path to doing this: an open source extension called pgvector. In contrast, Oracle developed vector similarity search capabilities in-house. At Oracle CloudWorld in 2023, the company announced a new vector data type, vector indexes, and vector search SQL operators.

The native vector database vendors would argue that their ground-up efforts provide better performance for AI workloads. Still, existing databases have their advantages; many enterprises are deeply dependent on Oracle, for example, making its vector capabilities inherently compelling.

There's a middle ground to be had. Yugabyte (a Futuriom 50 company) offers a Postgres database and announced in 2023 that it was working with pgvector. But Yugabyte has an additional mission: making Postgres work better for distributed, cloud-native environments. Performance is a big factor there, so Yugabyte integrated vector indexing libraries onto its distributed back-end and refined them for performance, using pgvector as the interface to them. It all results in a performance boost compared to simply adding pgvector atop a traditional Postgres database, Yugabyte says.

Big Data Platforms

Data platforms need to adjust as well. In one dramatic moment in June, Databricks and Snowflake both declared modernization efforts that included newly launched vector databases. Both companies developed those databases through acquisitions of PostgreSQL startups.

Both companies said their motivations ran deeper, however, targeting the need to modernize, preparing their platforms for for agentic AI.

That means supporting vectors, but Databricks went a step further by introducing a serverless vector database type called Lakebase. The serverless aspect is key, because it means the database can be spun up immediately and will autoscale to fit the job. In other words, Databricks is anticipating a world where agents spontaneously create databases on their own as an intermediate step in AI processing.

As Databricks and Yugabyte show, the magic isn't just in having vectors inside a database. Performance is a crucial aspect of fitting into the AI world. The real significance behind vector databases might be a need for databases to become more agile to match the dataflow needs of AI.

Amazon Dives Into Vectors

Finally, there's the case of AWS. Amazon S3 Vectors, released in preview last week, is described as durable vector storage, a long-term repository for data in vector form. But it also provides the indexing and querying capabilities that make a vector database valuable.

Notably, though, the querying is described as "subsecond," suggesting this is a far cry from the near real-time performance that vector database specialists tout. In fact, one use case suggested by AWS is to use S3 Vectors to store cold data (vectors that don't often get queried), moving them quickly into the Amazon OpenSearch Service as needed.

S3 Vectors is built for scale. AWS created a new vector type, vector buckets, that holds up to 10,000 vector indexes, with each index capable of holding tens of millions of vectors. It might not be a direct replacement for specialty vector databases in some cases, but it might prove suitable for many enterprises, especially those with large libraries of unstructured data.

Futuriom Take: Vector databases were a novel category in 2023 but are now everywhere, with well known players modernizing to catch up. Factors such as performance will dictate which products stand out, especially as larger vendors build their own in-house vector solutions.