...predict categories

↺ Start over · Full treeYou are here: AI Solver › Predict and analyze › ...predict categories

AI Solver map

Your current page is in bold 0 · Start Do I need AI? Full decision tree 1 · Predict & analyze tables, numbers, events Predict values Predict categories Forecast time series Cluster / segment Find outliers Make recommendations Find key drivers 2 · Understand unstructured data text, documents, images, audio Understand language Images & video Speech & audio 3 · Generate & converse Generate content Answer questions from my data Build an agent 4 · Decide & act Optimize decisions 5 · Build it right applies to every path Choose a model Customize a model Deploy AI Evaluate & trust AI New architectures Products & platforms

YouTube ... Quora ...Google search ...Google News ...Bing News

• AI Solver ... Algorithms ... Administration ... Model Search ... Discriminative vs. Generative ... Train, Validate, and Test

Use this page when each item should get a label: yes/no (churn, fraud, spam), one of several classes (product type, diagnosis), or several tags at once. Classifiers are everywhere: they flag customers likely to churn, predict whether it will rain, and filter spam.

Start here

• Is the input text, images or audio?
  • Text → ...understand language (prompted LLM, embeddings + classifier, or a fine-tuned small model)
  • Images → ...understand images and video (Image Classification)
• Are positives rare (fraud, defects, disease: under ~5%)?
  • Yes → plan for imbalance: class weights, tune the decision threshold to the business cost, evaluate with precision–recall. With almost no labeled positives, treat it as anomaly detection: ...find outliers
• How many labeled examples do you have?
  • None or a handful → describe the classes to an LLM and classify zero/few-shot; or let an LLM label a sample, have people check it, then train a small, cheap model on those labels
  • Hundreds or more → train a supervised model (below)
• Two-class classification; two predicting categories?
  • Do you need the results to be explainable?
    • Yes, and fast training, accurate, can have a large footprint → (Boosted) Decision Tree (gradient boosting: XGBoost, LightGBM, CatBoost), explained with SHAP values
    • Yes, and simple and auditable → Logistic Regression (LR), a shallow decision tree, or an Explainable Boosting Machine (EBM)
    • Linear, greater than 100 features → Support Vector Machine (SVM)
  • No →
    • Fast training, linear, features roughly independent → two-class Naive Bayes
    • Best accuracy → gradient-boosted trees; on small tables also try TabPFN
• Multi-class classification; three or more categories?
  • Do you need the results to be explainable?
    • Yes, fast training, linear → Logistic Regression (LR) (multinomial)
    • Yes, accurate → Decision Jungle (Azure ML Studio classic; today use a random forest or boosting)
    • Yes, fast training, accurate → Random Forest (or) Random Decision Forest
  • No →
    • Linear → Naive Bayes
    • Best accuracy on tables → gradient-boosted trees; TabPFN for small data
    • Accurate, can allow long training times, unstructured inputs → Deep Neural Network (DNN), e.g. Image Classification
      • which is a type of its predecessors... Feed Forward Neural Network (FF or FFNN) and Neural Network
• Several labels per item (multi-label)? → one model per label (one-vs-rest), or an LLM given a fixed tag list and a structured (JSON) output format

Quick picks

If you need...	Try first	Then compare
A transparent baseline	Logistic Regression (LR)	Shallow decision tree, EBM
Best accuracy on a table	LightGBM or CatBoost	XGBoost, Random Forest (or) Random Decision Forest, TabPFN (small data)
Text, very fast and cheap	TF-IDF + logistic regression or Naive Bayes	Embeddings + classifier
Text, few or changing labels	Prompted LLM with label definitions	LLM-labeled data → small fine-tuned model
Rare positives	Gradient boosting with class weights + tuned threshold	Anomaly detection (...find outliers)
Images	Fine-tuned pretrained network	Zero-shot image-text model (CLIP-style)

Real-world examples

• Spam filtering — Paul Graham's 2002 essay showed that a simple statistical (Bayesian) filter on word frequencies could catch most spam; spam filters became one of the most widespread classifiers. A Plan for Spam | Paul Graham
• Payment fraud — Stripe trained a transformer "payments foundation model" on tens of billions of transactions; it reported a 64% jump in detecting card-testing attacks on large businesses, compared with two years of gradual gains from its earlier specialized models. Stripe unveils AI foundation model for payments | TechCrunch

What's new

• LLMs as zero-shot classifiers and labelers remove the cold-start problem: you can classify on day one, then distill into a small model when volume grows.
• Tabular foundation models (TabPFN, TabICL) are strong first tries on small and medium tables.
• Foundation models trained on your own event data (as Stripe did) are replacing many hand-built features at large companies.

Watch out

• Accuracy can mislead: with 1% fraud, "never fraud" is 99% accurate. Use precision, recall and PR-AUC.
• The threshold is a business decision: set it from the cost of a false alarm versus a missed case.
• Fairness: check error rates per group (age, region, gender) before deployment.
• Calibration: if people act on the score as a probability, calibrate it.

Measure success

Precision, recall, F1-score, ROC-AUC, PR-AUC, confusion matrix, and calibration (Brier score). More: ...evaluate and trust AI

Learn more

• Data Augmentation, Data Labeling, and Auto-Tagging
• Machine Learning: Trying to classify your data | Stacey Ronaghan - Medium
• Choosing the right estimator | scikit-learn
• Evaluating Machine Learning Models
  • Feature Engineering and Selection: A Practical Approach for Predictive Models - 5 Encoding Categorical Predictors | Max Kuhn and Kjell Johnson
  • Binary Model Insights | Amazon Web Services
  • Multiclass Model Insights | Amazon Web Services

Classification problems are sometimes divided into binary (yes or no) and multi-category problems (animal, vegetable, or mineral). Classifiers are ubiquitous in data science. The world around is full of classifiers. Classifiers help in identifying customers who may churn. Classifiers help in predicting whether it will rain or not. Classifiers help in preventing spam e-mails. If the targets are designed to be binary (two-class classification) then a binary classifier is used, the target will only take a 0 or 1 value. Machine learning algorithms explained | Martin Heller - InfoWorld

---

Where next? ◀ Predict & analyze · ...predict values · ...find outliers · ...understand language · ▶ ...evaluate and trust AI