Chasing Polaris-People 06 When Vector Databases Were Not Yet Mainstream, They Had Already Seen the Future
Interview with Xiaofan Luan, VP of technology at Zilliz (Vector Database Series B)
This writing is based on the Innovator Coffee Interview for the next two sessions, which have more insights and details. Please stay tuned….
Before generative AI became a defining keyword of the era, very few people regarded vector databases as an independent and critical layer of infrastructure. At the time, mainstream discussions still revolved around traditional relational databases, search engines, or big data processing frameworks, while vector search largely remained confined to academic papers or algorithm libraries.
Yet vector databases were never a sudden invention. They were a natural outcome of the evolution of data representation. As early as 2017–2018, the team at Zilliz had already observed a clear trend: enterprises and developers were increasingly eager to understand and process unstructured data-including text, images, audio, logs, and behavioral traces-while traditional relational databases and keyword-based search systems exhibited inherent limitations at the semantic level.
Vectors, as a way of mapping objects into high-dimensional space, enabled similarity to be computed and retrieved systematically for the first time. Once embedding models transformed text, images, or other objects into vectors, databases no longer merely stored records; they began to assume the role of semantic retrieval and approximate understanding.
Vector databases thus emerged as a critical intermediary layer connecting model capabilities with the real-world data landscape.
How did this technology evolve from algorithmic refinement to real-world deployment? What challenges had to be overcome along the way? And where is it ultimately heading?
From Academic Algorithms to Engineering Systems: The Development Paths of Vector Databases
Looking back at the early history of vector search, Xiaofan Luan points out that the field was initially driven primarily by research institutions and large technology companies. A representative example is FAISS, developed by Meta (Facebook) Research, which exemplified this early stage in which vector search existed mainly as algorithm libraries rather than complete database systems. Microsoft, Spotify, and others also developed internal vector search technologies for their own use cases, but these efforts largely remained at the tooling level.
True differentiation began when vector search started transitioning toward engineering and productization.
Some companies chose a managed-service approach, focusing on online inference scenarios that worked closely with large language models. Others attempted to integrate vector capabilities with data lakes or traditional databases, exploring broader forms of enterprise-grade infrastructure. In Luan’s view, this multi-path divergence was a natural phase preceding the stabilization of a new paradigm.
As large models matured and applications accelerated toward production, vector database use cases evolved significantly. Early applications were primarily concentrated in recommendation systems and search engines, such as similarity-based product recommendations, image retrieval, and content matching. Over the past two to three years, however, Retrieval-Augmented Generation (RAG) architectures have become mainstream, positioning vector databases as providers of trusted context for models, responsible for fact verification and knowledge retrieval.
Within this paradigm, vector databases not only reduce hallucinations in large models but also serve as long-term memory layers for agent systems, supporting multi-step reasoning, context compression, and multimodal retrieval.
When discussing agent accuracy, Luan emphasizes the principle of “less structure, more intelligence.” Over-reliance on predefined workflows and rigid tagging, he argues, can actually constrain a model’s reasoning capacity. As models become more capable, agents benefit from operating in relatively open contexts, autonomously determining how to retrieve, combine, and reason over information.
Equally critical is data governance. Vector databases are not magic; they amplify the quality of the data they ingest. Only high-quality, scenario-relevant, curated, and validated data can fundamentally improve accuracy. Finally, Luan advises developers to maintain continuous evaluation practices, as embedding models, re-ranking models, and search algorithms evolve at an extraordinary pace—falling behind by six months often means being left behind entirely.
He also notes that as multimodal models advance, vector databases increasingly serve not only inference workflows supporting RAG and agent memory, but also model training and construction, such as assisting AI infrastructure teams with training-data deduplication, cleaning, and fine-grained selection. At their core, these capabilities all revolve around semantic similarity detection and retrieval.
In the long run, these capabilities may converge with data lakes to form a new architecture known as the “Vector Lake,” bridging batch data processing with online inference systems.
The future of vector databases, therefore, extends beyond inference alone, encompassing the entire lifecycle of model training, inference, and long-term data governance.
Finding the Signal Amid Noise Before Vector Databases Became Mainstream
Xiaofan Luan’s technical career began in the world of traditional databases. Both he and Zilliz’s founder, Charles Xie, spent many years working on transactional database systems, developing a deep understanding of structured data, performance optimization, and system stability.
It was precisely this background that made the rapid growth of unstructured data in 2017–2018 especially striking. Images, text, and audio were becoming mainstream, yet traditional databases and keyword search systems were nearly powerless when it came to semantic understanding. Vectors, meanwhile, were beginning to reveal their potential as a next-generation data representation.
Should they seize this opportunity?
Luan recalls that Zilliz did not initially set out to build a vector database. The team first experimented with GPU-accelerated databases, aiming to leverage hardware acceleration for large-scale data processing. Reality, however, quickly intervened: under the cost structures of the time, GPUs failed to achieve sufficient product-market fit.
Through continuous experimentation, the team identified a neglected demand—users wanted similarity search over images and unstructured data, but existing solutions were limited to algorithm libraries, lacking scalable and operable database forms.
This realization marked a pivotal turning point. Rather than chasing hype, the decision to pursue vector databases stemmed from a database engineer’s intuition: if AI were to permeate every industry, and if vector search became a core AI capability, then it would inevitably require proper infrastructure, not fragmented algorithm calls.
History ultimately validated this judgment. Early vector search tools such as FAISS and others offered immense performance value, but they remained libraries, leaving developers to handle data management, scalability, stability, and operations themselves.
Milvus emerged against this backdrop, posing a more engineering-driven question: if vector search were to become foundational for AI applications, could it be delivered as a standardized, distributed, and scalable system, much like relational databases?
This philosophy led Milvus down the open-source path early on, with scalability and stability as long-term core objectives.
The choice was far from easy. Luan openly acknowledges that vector databases were not initially well understood by the market, and even the definition of “vector database” itself has evolved substantially over the past six years. Yet persistence stemmed from a conviction forged in database history: truly valuable infrastructure requires long-term investment before markets fully crystallize.
Stability as the Greatest Virtue of Vector Databases
From a technical perspective, Luan believes the most difficult leap, from vector search to enterprise-grade infrastructure, lies in stability.
Vector search itself is not new. Leveraging existing open-source tools, many teams can assemble a vector database prototype within six to twelve months.
But stability does not result from a single breakthrough. It emerges from completing hundreds or thousands of incremental improvements. Each contributes only marginally, but collectively they define the system’s reliability.
Performance optimization often yields immediate gratification, a few months of benchmarking can deliver 20–30% gains. Stability, by contrast, may improve SLA by only 0.1% at a time, barely perceptible in isolation. Yet only through sustained patience and focus does true stability materialize.
On Open Source: Wearing Two Hats
In recent years, beyond engineering and product development, Luan has spent significant time grappling with a persistent challenge: balancing the dual identities of open-source community stewardship and commercial viability.
A healthy open-source community demands openness and inclusivity, while a company must still meet revenue and growth targets. This tension has become increasingly visible, with some companies relegating open-source projects to maintenance mode after finding them misaligned with business goals.
For Luan, open source is fundamentally a go-to-market strategy, a form of free trial that enables awareness and trust. As an engineering-driven company, Zilliz chose GitHub as its primary channel, allowing developers to experience Milvus firsthand.
This approach proved effective: today, roughly 80% of Zilliz Cloud customers originated from open-source Milvus users.
Open source also serves as a powerful trust-building mechanism. Users who rely on an open-source product over time are more inclined to enter deeper commercial relationships.
Yet conversion requires more than a thin commercial layer. It demands real added value, whether through management capabilities, operational efficiency, or cost optimization. Notably, many users migrating to commercial offerings have seen overall costs decrease, driven by advances in indexing, quantization, and storage architectures that follow a Moore’s-law-like trajectory of cost reduction.
Where Zilliz Stands Among Vector Database Providers
As AI enthusiasm surged, the vector database market grew crowded, encompassing managed services and lightweight plugins alike. Yet Luan argues that differentiation lies not in feature lists but in engineering depth.
Zilliz’s strengths center on scalability, stability, and enterprise readiness. Designed from the outset as a distributed, Kubernetes-native system, it maintains reliability even as data scales by orders of magnitude. Multi-tier indexing strategies combine memory, disk, and object storage to balance performance and cost.
Equally critical are enterprise capabilities, access control, data isolation, BYOC deployment, encryption, and compliance, allowing vector databases to operate in regulated industries rather than remaining confined to experimentation.
Many teams underestimate vector database complexity. While initial systems are easy to build, production realities quickly expose challenges in stability, scalability, and operational cost. Enterprises increasingly prefer to offload this complexity, focusing instead on core business value.
This division of labor represents Zilliz’s enduring value proposition.
Looking Ahead: The Next Three to Five Years
Luan remains pragmatic about the future. While AI infrastructure will continue its rapid growth, the true challenge is shifting from feasibility to sustainability. As data volumes explode alongside model usage, companies that master cost, performance, and security trade-offs will endure.
Reducing costs by an order of magnitude while preserving accuracy will become the next decisive frontier, one where infrastructure companies ultimately build their strongest moats.