What classification is
Until now every model answered with a number — a price. Classification answers with a class: cheap, mid, or expensive. Same HOUSES features; new output shape.
You still use layers, gradient descent, and a ReLU hidden layer. The new pieces are softmax (turn scores into probabilities) and a label that is a category, not dollars.
- Class
- One bucket the model may choose. Here: 0 cheap, 1 mid, 2 expensive.
- Softmax
- Turns three raw scores into three probabilities that sum to 1.
- One-hot label
- The true class as a vector of 0s with a single 1 — mid is
[0, 1, 0]. - Cross-entropy (idea)
- Loss that gets large when the model puts little probability on the correct class. Training pushes that probability up.
Data
Same houses, tier labels
Cut prices into three bands. The network never sees the dollar amount as the target — only the tier index.
| sqft | beds | age | price | tier | |
|---|---|---|---|---|---|
| H1 | 1200 | 2 | 15 | $245k | 0 cheap |
| H2 | 1800 | 3 | 8 | $310k | 1 mid |
| H3 | 2200 | 4 | 3 | $420k | 2 expensive |
| H4 | 900 | 2 | 40 | $180k | 0 cheap |
| H5 | 1500 | 3 | 10 | $295k | 1 mid |
[sqft/1000, beds, age/10]. That is the same scale lesson that forced tiny learning rates in linear regression — named this time.Build model
Three outputs, then softmax
- in play
Still [sqft, beds, age] — the running example from Vectors through Neural Networks.
Softmax
From scores to a distribution
Walk one toy logit vector. This is the classification-specific step — regression never needed it.
The last layer still outputs ordinary numbers — one per class. They are not probabilities yet.
What happens in this step
logits = [2.1, 0.3, −1.0] cheap mid expensive
Solution in TypeScript
Labels come from priceToTier. scale keeps features on similar magnitudes. softmax + argmax produce the class. Training (same backprop spine as the XOR network) uses probs − oneHot as the output gradient — the multi-class version of “how wrong.”
type Vector = number[];
function relu(x: number): number {
return Math.max(0, x);
}
function softmax(logits: Vector): Vector {
const max = Math.max(...logits);
const exps = logits.map((x) => Math.exp(x - max));
const sum = exps.reduce((a, b) => a + b, 0);
return exps.map((e) => e / sum);
}
function argmax(v: Vector): number {
return v.reduce((best, x, i, arr) => (x > arr[best] ? i : best), 0);
}
function priceToTier(price: number): number {
if (price < 250_000) return 0; // cheap
if (price <= 350_000) return 1; // mid
return 2; // expensive
}
function oneHot(tier: number, classes = 3): Vector {
const v = Array(classes).fill(0);
v[tier] = 1;
return v;
}
/** Scale features so sqft does not dwarf beds/age (same lesson as tiny learning rates). */
function scale(features: Vector): Vector {
return [features[0] / 1000, features[1], features[2] / 10];
}
const HOUSES: [Vector, number][] = [
[[1200, 2, 15], 245_000],
[[1800, 3, 8], 310_000],
[[2200, 4, 3], 420_000],
[[ 900, 2, 40], 180_000],
[[1500, 3, 10], 295_000],
];
const samples = HOUSES.map(([features, price]) => ({
features,
tier: priceToTier(price),
}));
// 3 → 8 ReLU → 3 logits → softmax (train loop omitted for length;
// same backprop idea as Neural Networks, with dL/dlogit = probs − oneHot)
function predictTier(features: Vector, logits: Vector): { probs: Vector; tier: number } {
const probs = softmax(logits);
return { probs, tier: argmax(probs) };
}
// Example forward on H5 after training (seed 1, scaled features):
// probs ≈ [0, 1, 0] → mid
console.log(samples.map((s) => ({ house: s.features, tier: s.tier })));Next
Accuracy on the training set is not the end
You can fit these five tiers perfectly and still fail on a new listing. Generalization is the habit of checking a house the model did not train on.
Trail: Neural Networks → Price Tier Classifier → Generalization.