Inference and hold-out
After
Train the Deep House Model, the network’s dials have moved.
In production you do inference only — predict / forward, weights frozen.
No sale price is required, and nothing backpropagates until you choose to train again.
This lesson also brings the generalization habit into the deep net: train on four houses, score the fifth you never trained on. Square footage is thousands; we scale inputs so training does not explode.
- Inference
- Using a finished network on new data. Forward only — no label required, no weights moving.
- Hold-out / train-test split
- Train on some rows; score others only with
predict. If youtrainOneon the exam, the exam is leaked. - Overfitting
- Train loss looks great; held-out error is poor. The net memorized the homework instead of a pattern that transfers. Deeper nets have more capacity, so this risk rises.
Reminder
Architecture you already built
Same stack as Lessons 1–4: 3 → 4 (ReLU) → 4 (ReLU) → 1 (linear). Inference does not change the shape — it only freezes the weights and stops the backward pass.
- in play
Lesson 4 moved every weight with trainOne (seed 37, scale, lr 1e-9). Those numbers are now the model.
Inference
New listing → forward only → estimate
A listing that never appeared in HOUSES still gets a price: run the forward stack and stop.
After training on the first four houses (5000 epochs, seed 37), [2000, 3, 5] infers ≈ 339983.
- current hop
[2000, 3, 5] is not one of the five training rows. You may not have a sale price at all.
A broker sends [2000, 3, 5] — 2000 sqft, 3 beds, 5 years old. That row is not in HOUSES. There is no sale price yet.
What happens in this step
listing = [2000, 3, 5] not in HOUSES no label attached
forward. Improving it means a separate training run on labeled sales — then you ship the new weights.Hold-out
Train on four, check the fifth
Same honest protocol as
Generalization, now on DeepHouseNet:
trainSet = HOUSES.slice(0, 4), heldOut = HOUSES[4].
Never call trainOne on the held-out house — score it with predict only.
After 5000 epochs: held-out pred ≈ 311758 vs actual 295000.
Lesson 4 trained on every HOUSES row. Loss going down felt like success — but the model had already seen every exam question.
What happens in this step
H1 H2 H3 H4 H5 all in the training loop
| Split | Rows | Role |
|---|---|---|
| Train | H1–H4 | Weights may look at these sale prices via trainOne |
| Held-out | H5 [1500, 3, 10] → $295,000 | Exam only — predict ≈ 311758, never train |
Solution in TypeScript
Self-contained copy of the deep house net (abbreviated training from Lesson 4), then the two habits of this lesson:
predict for frozen forward, and a hold-out split so train MSE is never the only score.
Same contract as the rest of the trail: seed 37, scale, lr = 1e-9.
type Vector = number[];
type Matrix = number[][]; // rows = neurons, cols = inputs
const HOUSES: [Vector, number][] = [
[[1200, 2, 15], 245_000],
[[1800, 3, 8], 310_000],
[[2200, 4, 3], 420_000],
[[ 900, 2, 40], 180_000],
[[1500, 3, 10], 295_000],
];
function relu(x: number): number {
return Math.max(0, x);
}
function reluDeriv(x: number): number {
return x > 0 ? 1 : 0;
}
/** Square footage is thousands; we scale inputs so training does not explode. */
function scale(features: Vector): Vector {
return [features[0] / 1000, features[1], features[2] / 10];
}
/** Mulberry32 — fixed seed so this lesson’s numbers are reproducible. */
function mulberry32(seed: number): () => number {
return () => {
seed |= 0;
seed = (seed + 0x6d2b79f5) | 0;
let t = Math.imul(seed ^ (seed >>> 15), 1 | seed);
t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
};
}
class Layer {
weights: Matrix;
biases: Vector;
activation: "relu" | "linear";
constructor(
inputSize: number,
outputSize: number,
activation: "relu" | "linear",
rnd: () => number,
) {
this.activation = activation;
this.weights = Array.from({ length: outputSize }, () =>
Array.from({ length: inputSize }, () => rnd() * 0.5 - 0.25),
);
this.biases = Array(outputSize).fill(0);
}
forward(inputs: Vector): { outputs: Vector; preActivations: Vector } {
const preActivations: Vector = [];
const outputs: Vector = [];
for (let j = 0; j < this.weights.length; j++) {
let sum = this.biases[j];
for (let i = 0; i < inputs.length; i++) {
sum += inputs[i] * this.weights[j][i];
}
preActivations.push(sum);
outputs.push(this.activation === "relu" ? relu(sum) : sum);
}
return { outputs, preActivations };
}
}
/** 3 → 4 ReLU → 4 ReLU → 1 linear — same net as Lessons 1–4. */
class DeepHouseNet {
h1: Layer;
h2: Layer;
out: Layer;
constructor(seed = 37) {
const rnd = mulberry32(seed);
this.h1 = new Layer(3, 4, "relu", rnd);
this.h2 = new Layer(4, 4, "relu", rnd);
this.out = new Layer(4, 1, "linear", rnd);
}
forward(features: Vector): number {
const x = scale(features);
const a = this.h1.forward(x);
const b = this.h2.forward(a.outputs);
const y = this.out.forward(b.outputs);
return y.outputs[0];
}
}
/** One labeled example: forward → backprop → one GD step (Lesson 3–4). */
function trainOne(
net: DeepHouseNet,
features: Vector,
target: number,
lr: number,
): number {
const x = scale(features); // Square footage is thousands; we scale inputs so training does not explode.
const h1 = net.h1.forward(x);
const h2 = net.h2.forward(h1.outputs);
const y = net.out.forward(h2.outputs);
const prediction = y.outputs[0];
const error = target - prediction;
for (let j = 0; j < net.out.weights.length; j++) {
for (let i = 0; i < h2.outputs.length; i++) {
net.out.weights[j][i] += error * h2.outputs[i] * lr;
}
net.out.biases[j] += error * lr;
}
const h2Error: Vector = [];
for (let i = 0; i < h2.outputs.length; i++) {
let e = 0;
for (let j = 0; j < net.out.weights.length; j++) {
e += error * net.out.weights[j][i];
}
h2Error.push(e * reluDeriv(h2.preActivations[i]));
}
for (let j = 0; j < net.h2.weights.length; j++) {
for (let i = 0; i < h1.outputs.length; i++) {
net.h2.weights[j][i] += h2Error[j] * h1.outputs[i] * lr;
}
net.h2.biases[j] += h2Error[j] * lr;
}
const h1Error: Vector = [];
for (let i = 0; i < h1.outputs.length; i++) {
let e = 0;
for (let j = 0; j < net.h2.weights.length; j++) {
e += h2Error[j] * net.h2.weights[j][i];
}
h1Error.push(e * reluDeriv(h1.preActivations[i]));
}
for (let j = 0; j < net.h1.weights.length; j++) {
for (let i = 0; i < x.length; i++) {
net.h1.weights[j][i] += h1Error[j] * x[i] * lr;
}
net.h1.biases[j] += h1Error[j] * lr;
}
return prediction;
}
function mse(net: DeepHouseNet, data: [Vector, number][]): number {
let total = 0;
for (const [x, y] of data) {
const err = y - net.forward(x);
total += err * err;
}
return total / data.length;
}
function train(
net: DeepHouseNet,
data: [Vector, number][],
epochs: number,
lr: number,
): void {
for (let epoch = 0; epoch < epochs; epoch++) {
for (const [features, price] of data) {
trainOne(net, features, price, lr);
}
}
}
/** Inference — forward only. No label, no weight updates. */
function predict(net: DeepHouseNet, features: Vector): number {
return net.forward(features);
}
// --- Hold-out: train on four, exam on the fifth ---
const trainSet = HOUSES.slice(0, 4);
const heldOut = HOUSES[4]; // [[1500, 3, 10], 295_000]
const net = new DeepHouseNet();
train(net, trainSet, 5_000, 1e-9); // never trainOne on heldOut
const trainLoss = mse(net, trainSet);
const heldPred = predict(net, heldOut[0]);
const heldLoss = (heldOut[1] - heldPred) ** 2;
console.log({ trainLoss, heldLoss });
console.log("held-out prediction", heldPred.toFixed(0)); // ≈ 311758
console.log("held-out actual ", heldOut[1]); // 295000
// If trainLoss looks great but heldLoss is large → overfitting.
// --- Production-style inference: a listing never seen in HOUSES ---
const listing: Vector = [2000, 3, 5];
const estimate = predict(net, listing);
console.log("inferred price", estimate.toFixed(0)); // ≈ 339983
// Weights stay frozen. No sale price required.