Embeddings

Executive Definition: Embeddings — Dense numerical vectors that represent the semantic meaning of data, allowing AI models to measure relatedness.

Embeddings map data—such as text, images, or audio—into a continuous mathematical space where semantically similar items are located near each other. They serve as the core enabling technology for semantic search, recommendation systems, and Retrieval-Augmented Generation (RAG), allowing systems to retrieve information based on conceptual meaning rather than exact keyword matches.

Frequently Asked Questions

How are embeddings generated?

Specialized neural networks process input data and map it to a high-dimensional vector space (often 768 to 1536 dimensions). Models like OpenAI’s text-embedding-3 or open-source BGE are commonly used for text.

What is the difference between embeddings and tokens?

Tokens are discrete chunks of text used as input for models. Embeddings are the continuous mathematical representations of those tokens or broader text sequences, capturing their underlying semantic meaning rather than just their characters.

Can embeddings represent non-text data?

Yes. Embedding models exist for images, audio, and graphs. Multimodal models, such as CLIP, can map different data types (like text and images) into the same shared vector space, enabling cross-modal search.