Embeddings

Embeddings are numerical vector representations of text (or images, audio, and other data) produced by neural networks. Each piece of text is converted into a list of numbers — typically 768 to 3,072 dimensions — positioned in a high-dimensional space where similar meanings cluster together. The sentence "web scraping tools for data collection" and "software for extracting information from websites" would produce embeddings that are close together in vector space, even though they share few words. Embeddings are the mathematical foundation of both semantic search and RAG. To build a semantic search system, you embed your document corpus and store the vectors in a vector database (Pinecone, Weaviate, Qdrant, Chroma, or pgvector). At query time, you embed the user's query, find the nearest document vectors, and return those documents. The quality of the embedding model determines how well "nearness in vector space" corresponds to "actually relevant to the query." The embedding model landscape has matured rapidly. OpenAI's text-embedding-3-large is widely used in production. Cohere's embed v3 supports over 100 languages. Open-source models like BGE-M3, E5-Mistral, and Nomic Embed offer competitive quality without per-token API costs. The MTEB benchmark provides standardized comparisons across models, though benchmark performance does not always predict real-world retrieval quality for specific domains. For AI product builders, embeddings are a core infrastructure decision. The embedding model you choose affects retrieval quality across your entire application. Switching models later requires re-embedding your entire corpus, which can be expensive and time-consuming. Key considerations include dimensionality (higher dimensions capture more nuance but cost more to store and search), multilingual support, maximum input length, and whether the model handles domain-specific vocabulary well. In the web data pipeline context, embeddings bridge the gap between raw web content and usable knowledge. Web scraping and AI search APIs fetch the content. Embedding models convert that content into vectors. Vector databases store and index those vectors. And retrieval systems query those vectors to power RAG, semantic search, and recommendation features.