AI Basics vocabulary

150 AI Basics terms, from Novice to Master. Start with the basics or jump to a level.

Updated

Novice

AI Assistant
An AI model packaged to help with tasks.
Artificial Intelligence
Software that does tasks we think need human intelligence.
Chatbot
A program you talk to in messages.
Generative AI
AI that creates new text, images, code or audio.
Large Language Model
An AI model trained on huge text to understand and write language.
Machine Learning
Programs that learn patterns from examples.
Model
The trained system that turns input into output.
Prompt
The instruction or question given to an AI model.

Apprentice

Accuracy
The share of a classifier's guesses that are right: right guesses divided by all guesses.
AI API
A web API for sending prompts to a model from code.
AI Coding Assistant
AI that suggests and writes code in your tools.
Baseline
The simplest honest guess, such as always guessing the average, that a model has to beat to be worth using.
Binary Classification
Classification with exactly two possible answers, such as spam or not spam, or fraud or not fraud.
Categorical Feature
A feature whose values are categories, not amounts, such as species or neighborhood; most models need it encoded as numbers first.
Classification
Machine learning that guesses a category, one of a fixed set of answers such as spam or not spam, from examples that come with their answers.
Classifier
A model that does classification: it guesses a category for each example, such as spam or not spam.
Cluster
A group of examples that clustering found to be similar to each other, such as customers who visit early and buy little.
Context Window
How much text a model can consider at once.
Decision Tree
A model that guesses by asking a chain of yes-or-no questions about the features, like a flowchart, ending at a leaf with the answer.
False Negative
A miss: the model said no, but the answer is yes, such as a fraud the check let through.
False Positive
A false alarm: the model said yes, but the answer is no, such as a normal payment flagged as fraud.
Feature
A column a model looks at to make its guess, such as the temperature when guessing a day's sales.
Hallucination
An AI confidently stating something false.
Heuristic
A hand-written rule of thumb, such as "Saturdays are busy, so bake more", used instead of a model learned from data.
Inference
Using a trained model to produce output.
Intercept
Where a line starts: its guess when every feature is 0, a starting amount the slopes build on.
Label
The column a model learns to guess, such as the cups sold; scikit-learn calls it the target.
Majority Vote
Picking the answer most of a group chose, such as the label most of a new example's nearest neighbors share.
Multimodal Model
A model that handles text plus images, audio or documents.
Probability
How likely something is, from 0 (it never happens) to 1 (it always does); 0.8 means about 8 times in 10.
Prompt Engineering
Writing prompts that get reliable results.
Pythagorean Theorem
The rule that a right triangle's long side, squared, equals the other two sides squared and added: 3² + 4² = 5².
Regression
Machine learning that guesses a number, such as tomorrow's loaves or a home's price, from examples that come with their answers.
Slope
How much a line's guess goes up for each one-unit step in a feature, in the label's units: dollars per square foot, say.
System Prompt
The standing instruction that sets an AI's role and rules.
Temperature
A setting for how random or creative output is.
Test Data
The examples held back from a model while it learns, so its guesses can be scored on examples it never saw.
Token
A chunk of text, about three-quarters of a word.
Training Data
The examples a model learned from.
True Negative
A no the model got right: it let through an example that really isn't the positive class, such as a normal payment.
True Positive
A yes the model got right: it flagged an example that really is the positive class, such as a fraud it caught.

Journeyman

Algorithmic Bias
When a model's decisions or mistakes fall unfairly on some groups of people, often learned from past decisions in its training data.
Balanced Accuracy
The average of each class's recall, so a rare class counts as much as a common one; always guessing one class scores 0.5.
Base Rate
How common the positive answer really is in a group, such as the share of its applicants who qualify, before any model is involved.
Centroid
The center of a cluster: the average of its examples, feature by feature; k-means moves each centroid to its group's average.
Chain of Thought
Having a model reason step by step before answering.
Class Imbalance
When one class is much rarer than another, such as 1 fraud in 100 payments, so accuracy looks high even for a model that never finds it.
Clustering
Machine learning that finds groups of similar examples on its own, with no answers given, such as customers who visit at the same times.
Coefficient
The number a linear model learns for one feature: how much its guess changes for one more unit of it, the other features held the same.
Confusion Matrix
A table counting a classifier's right and wrong guesses by kind: true positives, false positives, false negatives and true negatives.
Cross-Validation
Scoring a model several times, each time holding back a different part of the data for testing, then averaging, for a steadier score.
Data Drift
When the data a model sees in use changes from what it learned on, such as new kinds of customers, so its guesses may get worse.
Data Leakage
When information the model won't have in real use sneaks into training or testing, so its score looks better than it will be.
Decision Path
The questions one example answers on its way down a decision tree, from the top question to the leaf that gives its answer.
Decision Threshold
The probability at which a classifier switches its answer, 0.5 by default; lower it to catch more, raise it for fewer false alarms.
Elbow Method
Choosing k for k-means by plotting inertia for k = 1, 2, 3 and so on, and picking the k where the drop flattens, like a bent arm.
Embedding
Numbers capturing text meaning, used for semantic search.
Ensemble Learning
Combining several models' guesses, by voting or averaging, so their different mistakes cancel out, as a random forest does.
Euclidean Distance
The straight-line distance between two examples, from the Pythagorean theorem: the square root of their squared differences added up.
Evaluation (Evals)
Test sets that score AI output quality.
Extrapolation
Guessing for inputs outside the range a model learned from, such as a 110° day when training stopped at 80°; such guesses can't be trusted.
F1 Score
One number that combines precision and recall as their harmonic mean, so it's only high when both are.
False Positive Rate
Of the real negatives, the share the model wrongly said yes to, FP / (FP + TN), such as the unqualified applicants it invited.
Feature Importance
A score for how much a model relies on each feature; trees' built-in feature_importances_ and permutation importance are two ways.
Feature Scaling
Putting features on similar ranges, such as grams and a 1-10 score, so no feature counts more only because its numbers are bigger.
Feature Selection
Choosing which features a model gets, such as the ten most related to the label; done on every row before splitting, it leaks.
Feedback Loop
When a model's own decisions shape the data its next version learns from, so an early mistake can grow over time.
Few-Shot Prompting
Including examples in the prompt for the model to copy.
Grid Search
Trying every combination of a few hyperparameter values, scoring each with cross-validation, and keeping the best.
Guardrails
Checks that keep AI inputs and outputs safe.
Harmonic Mean
An average that leans toward the smaller numbers, 2 × a × b / (a + b) for two; the F1 score is precision and recall's harmonic mean.
Hyperparameter
A setting you choose before training, such as a tree's max_depth or the k in k-nearest neighbors, rather than something the model learns.
Interpretability
How easily a person can see why a model gave its answer, such as by reading a small tree's questions or a line's coefficients.
K-Fold Cross-Validation
Cross-validation that splits the rows into k equal parts and trains k times, each time testing on a different part.
K-Means Clustering
Clustering that splits examples into k groups by moving k centers until each example sits with the center nearest to it.
k-Nearest Neighbors
A model that guesses by finding the k training examples closest to a new one and taking their most common label, or their average.
Linear Regression
A model that guesses a number with the straight line that runs closest to its examples: a starting point plus a slope times each feature.
Logistic Function
The S-shaped curve that squeezes any number into one between 0 and 1: 0 becomes 0.5, big totals go near 1, very negative ones near 0.
Logistic Regression
A classification model that turns a weighted sum of the features into a probability between 0 and 1, then picks a class from it.
Mean Absolute Error
The average size of a model's misses, in the label's own units: "off by 8 cups, on average".
Mean Squared Error
The average of the squared misses; squaring makes big misses count much more, and the result is in squared units.
Missing Indicator
An extra 0-or-1 column that says whether a value was missing before it was filled, so the model can learn from the gap itself.
Model Card
A short document that comes with a model: what it's for, what it learned from, how well it scores for each group, and its limits.
Model Persistence
Saving a trained model to a file so another program can load it and predict without training again.
Multiclass Classification
Classification with three or more possible answers, such as apple, orange or pear, where each example gets exactly one.
Neural Network
Layers of adjustable weights that learn patterns.
One-Hot Encoding
Turning a category column into one 0-or-1 column per category, such as dog, cat and rabbit columns, so a model can use it.
Ordinary Least Squares
Fitting a line by choosing the one whose squared errors add up to the least; OLS for short, and what LinearRegression does.
Overfitting
When a model learns its examples' random wobble so closely that it does well on them and worse on new examples.
Polynomial Features
New columns made by raising a feature to powers, such as rainfall² beside rainfall, so a straight-line model can fit a curve.
Polynomial Regression
Linear regression on polynomial features, such as rainfall and rainfall², so its guesses follow a curve instead of a straight line.
Positive Class
In a yes-or-no classifier, the answer you're looking for, such as fraud or spam; precision and recall measure how well the model finds it.
Precision
Of the examples a model flagged, the share that really were positive: TP / (TP + FP). High precision means few false alarms.
Proxy
A feature that stands in for another, such as a ZIP code for neighborhood, so a model learns a sensitive feature it was never given.
R Squared
How much better a model does than always guessing the average: 1 is perfect, 0 is no better than the average, below 0 is worse.
Random Forest
A model made of many decision trees, each trained on a random sample of the rows, that takes their vote or their average.
Rate Limits and Token Costs
AI APIs charge per token and cap usage per minute.
Recall
Of the real positives, the share the model caught: TP / (TP + FN). Also called sensitivity or the true positive rate.
Regression Tree
A decision tree that guesses a number: each leaf guesses the average label of the training examples that ended there.
Reinforcement Learning
Machine learning where a program learns by trying actions and getting rewards or penalties, such as a game player that improves by playing.
Residual
One example's miss: the actual value minus the model's guess, positive when the guess was too low.
Residual Plot
A chart of each example's residual against the model's guess, so patterns in the misses, such as always too low on big values, stand out.
Retrieval-Augmented Generation
Finding relevant documents and giving them to the model with the question.
Root Mean Squared Error
The square root of the mean squared error: an average miss in the label's own units that counts big misses more than mean absolute error does.
Selection Rate
The share of a group a model says yes to, such as the invited applicants divided by all applicants from one district.
Sensitive Feature
A feature about a personal trait, such as race, sex, age or district, that a model's decisions shouldn't treat people unfairly by.
Standardization
Rescaling a feature so its mean is 0 and its standard deviation is 1: subtract the mean, then divide by the standard deviation.
Stratified Split
A split that keeps each class's share the same in the training and test parts, such as 20% spam in both; stratify=y does it.
Streaming Responses
Sending the answer piece by piece as it is generated.
Structured Output
Making a model reply in an exact format like JSON.
Supervised vs Unsupervised Learning
Learning from labelled examples versus finding patterns alone.
Tool Use (Function Calling)
Letting a model call your functions and use the results.
Underfitting
When a model is too simple to catch the pattern, so it does badly on its training examples and on new ones alike.
Validation Set
Rows held back to compare models and choose settings, kept apart from the test set, which is saved for one final, honest score.

Freelancer

AI Agent
A model that plans, uses tools and repeats toward a goal.
Bias-Variance Tradeoff
The balance between a model too simple to catch the pattern (high bias) and one so flexible it learns the noise (high variance).
Bootstrap Sample
A random sample the same size as the data, drawn with replacement, so some rows appear twice and others not at all.
Calibration
How well a model's probabilities match reality: of the cases it gives 0.7, about 7 in 10 should really be positive.
Concept Drift
When the link between the features and the answer changes, such as what makes a payment fraud, so a model's old lessons stop being true.
Demographic Parity
A fairness test that asks a model to say yes to the same share of each group, whatever their qualifications.
Equalized Odds
A fairness test that asks for the same recall (true positive rate) and the same false positive rate in every group.
Fine-Tuning
Further training a model on your own examples.
Gini Impurity
How mixed a group's classes are: 0 when all are one class, 0.5 for an even two-class mix; a tree picks the question that lowers it most.
Gradient Boosting
An ensemble that builds small trees one after another, each trained to fix the mistakes the trees before it still make.
Grounding
Tying answers to provided sources.
Individual Conditional Expectation
One line per example showing how the model's guess for it changes as one feature changes; their average is the partial dependence plot.
Inertia
The total of each example's squared distance to its cluster's center; lower means tighter clusters, and it always drops as k rises.
Instance-Based Learning
Learning that keeps the training examples themselves and compares each new one to them at guessing time, as k-nearest neighbors does.
k-means++
A way of choosing k-means' starting centers spread far apart, which makes a poor result less likely; scikit-learn's default.
Learning Curve
A chart of training and validation scores as the training set grows, showing whether more data would help.
Local Minimum
A result that's the best nearby but not the best overall; k-means can settle in one, so it's run from several starts.
Model Context Protocol
An open standard for connecting AI to tools and data.
Multicollinearity
Features that move together, such as a home's size and its bedrooms, so a model can't tell their effects apart and they share the credit.
Open-Weight Model
A model whose weights are published to run yourself.
Partial Dependence Plot
A chart of a model's average guess as one feature changes and the rest stay as they are, showing that feature's typical effect.
Permutation Importance
How much a model's score drops when one feature's values are shuffled; a big drop means the model leans on that feature.
Precision-Recall Curve
A chart of precision against recall at every decision threshold, so you can pick the trade-off between false alarms and misses.
Prompt Caching
Reusing a repeated prompt prefix to save time and cost.
Prompt Injection
An attack where input text tries to override the AI's instructions.
Regularization
A penalty on large coefficients while a model learns, so it can't lean too hard on any one feature; it helps against overfitting.
Semantic Search
Search by meaning rather than exact words.
Silhouette Score
A score from -1 to 1 for how well examples sit in their own cluster rather than the next nearest; higher means clearer clusters.
Target Leakage
Data leakage through a feature that's only known after the answer, such as a refund column in a fraud model: great test scores, failure in real use.
Transformer
The network design behind modern LLMs.
Validation Curve
A chart of training and validation scores as one setting changes, such as a tree's depth, showing where underfitting turns into overfitting.

Master

Attention Mechanism
How a model weighs which tokens matter for the next one.
Mixture of Experts
A model routing each token to a few specialist parts.
Model Distillation
Training a small model to copy a larger one.
Quantization of Models
Shrinking a model by storing weights with fewer bits.
RLHF
Training models on human ratings of their answers.

More subjects in AI