AI Basics vocabulary
150 AI Basics terms, from Novice to Master. Start with the basics or jump to a level.
Updated
Novice
- AI Assistant
- An AI model packaged to help with tasks.
- Artificial Intelligence
- Software that does tasks we think need human intelligence.
- Chatbot
- A program you talk to in messages.
- Generative AI
- AI that creates new text, images, code or audio.
- Large Language Model
- An AI model trained on huge text to understand and write language.
- Machine Learning
- Programs that learn patterns from examples.
- Model
- The trained system that turns input into output.
- Prompt
- The instruction or question given to an AI model.
Apprentice
- Accuracy
- The share of a classifier's guesses that are right: right guesses divided by all guesses.
- AI API
- A web API for sending prompts to a model from code.
- AI Coding Assistant
- AI that suggests and writes code in your tools.
- Baseline
- The simplest honest guess, such as always guessing the average, that a model has to beat to be worth using.
- Binary Classification
- Classification with exactly two possible answers, such as spam or not spam, or fraud or not fraud.
- Categorical Feature
- A feature whose values are categories, not amounts, such as species or neighborhood; most models need it encoded as numbers first.
- Classification
- Machine learning that guesses a category, one of a fixed set of answers such as spam or not spam, from examples that come with their answers.
- Classifier
- A model that does classification: it guesses a category for each example, such as spam or not spam.
- Cluster
- A group of examples that clustering found to be similar to each other, such as customers who visit early and buy little.
- Context Window
- How much text a model can consider at once.
- Decision Tree
- A model that guesses by asking a chain of yes-or-no questions about the features, like a flowchart, ending at a leaf with the answer.
- False Negative
- A miss: the model said no, but the answer is yes, such as a fraud the check let through.
- False Positive
- A false alarm: the model said yes, but the answer is no, such as a normal payment flagged as fraud.
- Feature
- A column a model looks at to make its guess, such as the temperature when guessing a day's sales.
- Hallucination
- An AI confidently stating something false.
- Heuristic
- A hand-written rule of thumb, such as "Saturdays are busy, so bake more", used instead of a model learned from data.
- Inference
- Using a trained model to produce output.
- Intercept
- Where a line starts: its guess when every feature is 0, a starting amount the slopes build on.
- Label
- The column a model learns to guess, such as the cups sold; scikit-learn calls it the target.
- Majority Vote
- Picking the answer most of a group chose, such as the label most of a new example's nearest neighbors share.
- Multimodal Model
- A model that handles text plus images, audio or documents.
- Probability
- How likely something is, from 0 (it never happens) to 1 (it always does); 0.8 means about 8 times in 10.
- Prompt Engineering
- Writing prompts that get reliable results.
- Pythagorean Theorem
- The rule that a right triangle's long side, squared, equals the other two sides squared and added: 3² + 4² = 5².
- Regression
- Machine learning that guesses a number, such as tomorrow's loaves or a home's price, from examples that come with their answers.
- Slope
- How much a line's guess goes up for each one-unit step in a feature, in the label's units: dollars per square foot, say.
- System Prompt
- The standing instruction that sets an AI's role and rules.
- Temperature
- A setting for how random or creative output is.
- Test Data
- The examples held back from a model while it learns, so its guesses can be scored on examples it never saw.
- Token
- A chunk of text, about three-quarters of a word.
- Training Data
- The examples a model learned from.
- True Negative
- A no the model got right: it let through an example that really isn't the positive class, such as a normal payment.
- True Positive
- A yes the model got right: it flagged an example that really is the positive class, such as a fraud it caught.
Journeyman
- Algorithmic Bias
- When a model's decisions or mistakes fall unfairly on some groups of people, often learned from past decisions in its training data.
- Balanced Accuracy
- The average of each class's recall, so a rare class counts as much as a common one; always guessing one class scores 0.5.
- Base Rate
- How common the positive answer really is in a group, such as the share of its applicants who qualify, before any model is involved.
- Centroid
- The center of a cluster: the average of its examples, feature by feature; k-means moves each centroid to its group's average.
- Chain of Thought
- Having a model reason step by step before answering.
- Class Imbalance
- When one class is much rarer than another, such as 1 fraud in 100 payments, so accuracy looks high even for a model that never finds it.
- Clustering
- Machine learning that finds groups of similar examples on its own, with no answers given, such as customers who visit at the same times.
- Coefficient
- The number a linear model learns for one feature: how much its guess changes for one more unit of it, the other features held the same.
- Confusion Matrix
- A table counting a classifier's right and wrong guesses by kind: true positives, false positives, false negatives and true negatives.
- Cross-Validation
- Scoring a model several times, each time holding back a different part of the data for testing, then averaging, for a steadier score.
- Data Drift
- When the data a model sees in use changes from what it learned on, such as new kinds of customers, so its guesses may get worse.
- Data Leakage
- When information the model won't have in real use sneaks into training or testing, so its score looks better than it will be.
- Decision Path
- The questions one example answers on its way down a decision tree, from the top question to the leaf that gives its answer.
- Decision Threshold
- The probability at which a classifier switches its answer, 0.5 by default; lower it to catch more, raise it for fewer false alarms.
- Elbow Method
- Choosing k for k-means by plotting inertia for k = 1, 2, 3 and so on, and picking the k where the drop flattens, like a bent arm.
- Embedding
- Numbers capturing text meaning, used for semantic search.
- Ensemble Learning
- Combining several models' guesses, by voting or averaging, so their different mistakes cancel out, as a random forest does.
- Euclidean Distance
- The straight-line distance between two examples, from the Pythagorean theorem: the square root of their squared differences added up.
- Evaluation (Evals)
- Test sets that score AI output quality.
- Extrapolation
- Guessing for inputs outside the range a model learned from, such as a 110° day when training stopped at 80°; such guesses can't be trusted.
- F1 Score
- One number that combines precision and recall as their harmonic mean, so it's only high when both are.
- False Positive Rate
- Of the real negatives, the share the model wrongly said yes to, FP / (FP + TN), such as the unqualified applicants it invited.
- Feature Importance
- A score for how much a model relies on each feature; trees' built-in feature_importances_ and permutation importance are two ways.
- Feature Scaling
- Putting features on similar ranges, such as grams and a 1-10 score, so no feature counts more only because its numbers are bigger.
- Feature Selection
- Choosing which features a model gets, such as the ten most related to the label; done on every row before splitting, it leaks.
- Feedback Loop
- When a model's own decisions shape the data its next version learns from, so an early mistake can grow over time.
- Few-Shot Prompting
- Including examples in the prompt for the model to copy.
- Grid Search
- Trying every combination of a few hyperparameter values, scoring each with cross-validation, and keeping the best.
- Guardrails
- Checks that keep AI inputs and outputs safe.
- Harmonic Mean
- An average that leans toward the smaller numbers, 2 × a × b / (a + b) for two; the F1 score is precision and recall's harmonic mean.
- Hyperparameter
- A setting you choose before training, such as a tree's max_depth or the k in k-nearest neighbors, rather than something the model learns.
- Interpretability
- How easily a person can see why a model gave its answer, such as by reading a small tree's questions or a line's coefficients.
- K-Fold Cross-Validation
- Cross-validation that splits the rows into k equal parts and trains k times, each time testing on a different part.
- K-Means Clustering
- Clustering that splits examples into k groups by moving k centers until each example sits with the center nearest to it.
- k-Nearest Neighbors
- A model that guesses by finding the k training examples closest to a new one and taking their most common label, or their average.
- Linear Regression
- A model that guesses a number with the straight line that runs closest to its examples: a starting point plus a slope times each feature.
- Logistic Function
- The S-shaped curve that squeezes any number into one between 0 and 1: 0 becomes 0.5, big totals go near 1, very negative ones near 0.
- Logistic Regression
- A classification model that turns a weighted sum of the features into a probability between 0 and 1, then picks a class from it.
- Mean Absolute Error
- The average size of a model's misses, in the label's own units: "off by 8 cups, on average".
- Mean Squared Error
- The average of the squared misses; squaring makes big misses count much more, and the result is in squared units.
- Missing Indicator
- An extra 0-or-1 column that says whether a value was missing before it was filled, so the model can learn from the gap itself.
- Model Card
- A short document that comes with a model: what it's for, what it learned from, how well it scores for each group, and its limits.
- Model Persistence
- Saving a trained model to a file so another program can load it and predict without training again.
- Multiclass Classification
- Classification with three or more possible answers, such as apple, orange or pear, where each example gets exactly one.
- Neural Network
- Layers of adjustable weights that learn patterns.
- One-Hot Encoding
- Turning a category column into one 0-or-1 column per category, such as dog, cat and rabbit columns, so a model can use it.
- Ordinary Least Squares
- Fitting a line by choosing the one whose squared errors add up to the least; OLS for short, and what LinearRegression does.
- Overfitting
- When a model learns its examples' random wobble so closely that it does well on them and worse on new examples.
- Polynomial Features
- New columns made by raising a feature to powers, such as rainfall² beside rainfall, so a straight-line model can fit a curve.
- Polynomial Regression
- Linear regression on polynomial features, such as rainfall and rainfall², so its guesses follow a curve instead of a straight line.
- Positive Class
- In a yes-or-no classifier, the answer you're looking for, such as fraud or spam; precision and recall measure how well the model finds it.
- Precision
- Of the examples a model flagged, the share that really were positive: TP / (TP + FP). High precision means few false alarms.
- Proxy
- A feature that stands in for another, such as a ZIP code for neighborhood, so a model learns a sensitive feature it was never given.
- R Squared
- How much better a model does than always guessing the average: 1 is perfect, 0 is no better than the average, below 0 is worse.
- Random Forest
- A model made of many decision trees, each trained on a random sample of the rows, that takes their vote or their average.
- Rate Limits and Token Costs
- AI APIs charge per token and cap usage per minute.
- Recall
- Of the real positives, the share the model caught: TP / (TP + FN). Also called sensitivity or the true positive rate.
- Regression Tree
- A decision tree that guesses a number: each leaf guesses the average label of the training examples that ended there.
- Reinforcement Learning
- Machine learning where a program learns by trying actions and getting rewards or penalties, such as a game player that improves by playing.
- Residual
- One example's miss: the actual value minus the model's guess, positive when the guess was too low.
- Residual Plot
- A chart of each example's residual against the model's guess, so patterns in the misses, such as always too low on big values, stand out.
- Retrieval-Augmented Generation
- Finding relevant documents and giving them to the model with the question.
- Root Mean Squared Error
- The square root of the mean squared error: an average miss in the label's own units that counts big misses more than mean absolute error does.
- Selection Rate
- The share of a group a model says yes to, such as the invited applicants divided by all applicants from one district.
- Sensitive Feature
- A feature about a personal trait, such as race, sex, age or district, that a model's decisions shouldn't treat people unfairly by.
- Standardization
- Rescaling a feature so its mean is 0 and its standard deviation is 1: subtract the mean, then divide by the standard deviation.
- Stratified Split
- A split that keeps each class's share the same in the training and test parts, such as 20% spam in both; stratify=y does it.
- Streaming Responses
- Sending the answer piece by piece as it is generated.
- Structured Output
- Making a model reply in an exact format like JSON.
- Supervised vs Unsupervised Learning
- Learning from labelled examples versus finding patterns alone.
- Tool Use (Function Calling)
- Letting a model call your functions and use the results.
- Underfitting
- When a model is too simple to catch the pattern, so it does badly on its training examples and on new ones alike.
- Validation Set
- Rows held back to compare models and choose settings, kept apart from the test set, which is saved for one final, honest score.
Freelancer
- AI Agent
- A model that plans, uses tools and repeats toward a goal.
- Bias-Variance Tradeoff
- The balance between a model too simple to catch the pattern (high bias) and one so flexible it learns the noise (high variance).
- Bootstrap Sample
- A random sample the same size as the data, drawn with replacement, so some rows appear twice and others not at all.
- Calibration
- How well a model's probabilities match reality: of the cases it gives 0.7, about 7 in 10 should really be positive.
- Concept Drift
- When the link between the features and the answer changes, such as what makes a payment fraud, so a model's old lessons stop being true.
- Demographic Parity
- A fairness test that asks a model to say yes to the same share of each group, whatever their qualifications.
- Equalized Odds
- A fairness test that asks for the same recall (true positive rate) and the same false positive rate in every group.
- Fine-Tuning
- Further training a model on your own examples.
- Gini Impurity
- How mixed a group's classes are: 0 when all are one class, 0.5 for an even two-class mix; a tree picks the question that lowers it most.
- Gradient Boosting
- An ensemble that builds small trees one after another, each trained to fix the mistakes the trees before it still make.
- Grounding
- Tying answers to provided sources.
- Individual Conditional Expectation
- One line per example showing how the model's guess for it changes as one feature changes; their average is the partial dependence plot.
- Inertia
- The total of each example's squared distance to its cluster's center; lower means tighter clusters, and it always drops as k rises.
- Instance-Based Learning
- Learning that keeps the training examples themselves and compares each new one to them at guessing time, as k-nearest neighbors does.
- k-means++
- A way of choosing k-means' starting centers spread far apart, which makes a poor result less likely; scikit-learn's default.
- Learning Curve
- A chart of training and validation scores as the training set grows, showing whether more data would help.
- Local Minimum
- A result that's the best nearby but not the best overall; k-means can settle in one, so it's run from several starts.
- Model Context Protocol
- An open standard for connecting AI to tools and data.
- Multicollinearity
- Features that move together, such as a home's size and its bedrooms, so a model can't tell their effects apart and they share the credit.
- Open-Weight Model
- A model whose weights are published to run yourself.
- Partial Dependence Plot
- A chart of a model's average guess as one feature changes and the rest stay as they are, showing that feature's typical effect.
- Permutation Importance
- How much a model's score drops when one feature's values are shuffled; a big drop means the model leans on that feature.
- Precision-Recall Curve
- A chart of precision against recall at every decision threshold, so you can pick the trade-off between false alarms and misses.
- Prompt Caching
- Reusing a repeated prompt prefix to save time and cost.
- Prompt Injection
- An attack where input text tries to override the AI's instructions.
- Regularization
- A penalty on large coefficients while a model learns, so it can't lean too hard on any one feature; it helps against overfitting.
- Semantic Search
- Search by meaning rather than exact words.
- Silhouette Score
- A score from -1 to 1 for how well examples sit in their own cluster rather than the next nearest; higher means clearer clusters.
- Target Leakage
- Data leakage through a feature that's only known after the answer, such as a refund column in a fraud model: great test scores, failure in real use.
- Transformer
- The network design behind modern LLMs.
- Validation Curve
- A chart of training and validation scores as one setting changes, such as a tree's depth, showing where underfitting turns into overfitting.
Master
- Attention Mechanism
- How a model weighs which tokens matter for the next one.
- Mixture of Experts
- A model routing each token to a few specialist parts.
- Model Distillation
- Training a small model to copy a larger one.
- Quantization of Models
- Shrinking a model by storing weights with fewer bits.
- RLHF
- Training models on human ratings of their answers.