Search for a command to run...
I'm the founder of CareerPod, a Software Quality Engineer at Red Hat, Python Developer, Cloud & DevOps Enthusiast, AI/ML Advocate, and Tech Enthusiast. I enjoy building projects, sharing valuable tips for new programmers, and connecting with the tech community. Check out my blog at Tech Journal.
No comments yet. Be the first to comment.
Most people think NVIDIA, AMD, or Intel won the AI race by building faster GPUs. That's only half the story. The real winner isn't just the company with the most powerful hardware it's the company wit
For a long time, I’ve been fascinated by Artificial Intelligence (AI) and how it’s shaping the world around us. But I’ve noticed that many people struggle with the basics AI/ML terms often feel full of jargon and hard to grasp. So, in this blog, I’ll...
Large language models (LLMs) like ChatGPT, LLaMA, and Mistral are incredibly powerful, but they're also resource-hungry. They need lots of memory and processing power to respond to a single prompt, let alone handle multiple users. So how do you run a...
When we browse the internet, a lot happens behind the scenes. One such hidden hero is the proxy. Whether it’s keeping your identity safe, speeding things up, or managing requests, proxies silently work in the background to make your online experience...
Introduction : I had an old laptop and a wild idea… Most GenAI tools need expensive GPUs or cloud credits. I had neither. So, I asked myself can I run a language model locally, without the cloud, and still make it accessible from anywhere? Turns out,...
Hey Everyone 👋🏻 !
In my previous blog, we explored how Transformers work and how they revolutionized modern AI, paving the way for major advancements.
Today, let's dive into embeddings—the foundation of Large Language Models (LLMs). We'll cover how embeddings work, different types of embeddings, and their applications. Let's get started!
An embedding is a numerical representation of data (words, sentences, images, videos, audio, documents) in a vector format within a multidimensional space. These representations capture meaning and relationships between data points.
It is also known as a vector embedding.
📌 Example:
Image sources : towardsdatascience.com
Words with similar meanings—like "king" and "queen"—will have embeddings that are closer together in vector space.
Represent individual words in a multi-dimensional space.
Capture relationships between words.
📌 Models: Word2Vec, GloVe, FastText (by Facebook)
📌 Use Cases: Machine translation, chatbots, search engines
📌 Models: BERT, Paragraph2Vec
📌 Use Cases: Text classification, sentiment analysis
Note:
Word embeddings focus on individual words.
Text embeddings capture the meaning of entire texts.
Represent entire sentences in vector form.
Capture both meaning and context.
Similar sentences have embeddings that are closer together.
📌 Models: Sentence-BERT (SBERT), Universal Sentence Encoder (USE) [by Google], Infersent
📌 Use Cases: Semantic search, text retrieval
Note: Sentence embeddings are a type of text embedding but specifically focus on entire sentences
Convert images into feature vectors.
Helps in image similarity searches & object recognition.
📌 Models: CNN (ResNet, VGG, CLIP)
📌 Use Cases: Image search, object detection
Represent nodes, edges, or entire graphs as vectors.
Used in social networks, fraud detection, recommendation systems.
📌 Models: Node2Vec, GraphSAGE
📌 Use Cases: Fraud detection, social network analysis
📌 Models: C3D, CLIP (for video)
📌 Use Cases: Video search, activity recognition
Convert sound waves into vector representations.
Capture pitch, tone, and speech meaning.
📌 Models: Wav2Vec, OpenL3
📌 Use Cases: Speech recognition, music classification
Note: All these embeddings fall under vector embeddings.
A shared embedding space is a common vector space where different types of data (e.g., text & images) are mapped close together if they are related.
Image sources : dailydoseofds.com
📌 Example: CLIP (Contrastive Language-Image Pretraining)
- Developed by OpenAI to create a shared embedding space for images & text.
- Allows images and textual descriptions to be compared directly.
Use Case: You can search for images using text descriptions!
Image sources : lyzr.ai
Large Language Models (LLMs): Convert input tokens into token embeddings.
Semantic Search: Retrieves similar sentences to improve search relevance.
RAG (Retrieval-Augmented Generation): Uses sentence embeddings to retrieve relevant text.
Recommendations: Finds similar products using vector search.
Anomaly Detection: Identifies unusual patterns in data.
Image sources : medium.com
Word2Vec → Predicts a word based on surrounding words (developed by Google).
GloVe → Similar to Word2Vec but with a different mathematical approach (by Stanford).
Image sources : edenai.co
A token is a small unit of text used in NLP models. It can be:
A word (e.g., "cat")
A subword (e.g., "play" and "ing" in "playing")
A character (e.g., "C", "a", "t")
A symbol or punctuation (e.g., "!")
Example:
- Sentence:"I love AI!"
- Tokens:["I", "love", "AI", "!"]
| Feature | Token | Embedding |
| Definition | A unit of text (word, subword, character) | A numeric vector representing meaning |
| Format | Text | Numbers (vector) |
| Example | "cat" → Token | "cat" → [0.23, 0.87, -0.45, ...] |
Tokens help models read text.
Embeddings help models understand meaning.
Example Code: How Embeddings Works : https://github.com/Raghul-M/GenAI/blob/main/Embedings/Token-Embeddings.ipynb
Embeddings have become a fundamental building block in modern AI, enabling machines to understand and represent complex data in a way that drives advancements in natural language processing, search, and more. Their ability to convert words, sentences, or documents into numerical vectors allows for more efficient and accurate tasks like similarity search and classification. As technology evolves, embeddings will continue to play a crucial role in shaping the future of AI.
Connect with me on Linkedin: Raghul M