Forward Pass Through Depth

Take one house — [1500, 3, 10] — and walk it slowly through the deep house net: features → scale → H1 ReLU → H2 ReLU → linear price.

Every number below uses seed 37 (same DeepHouseNet as the architecture lesson). Square footage is thousands; we scale inputs so training does not explode. Predictions are still garbage — we are learning how depth transforms a vector, not training yet.

Pre-activation: The weighted sum W · inputs + b before ReLU. Can be negative.
ReLU gate: max(0, pre). Negative → 0 (“off”). Positive → pass through (“on”).
Linear output: Last hop skips ReLU so the price is not forced ≥ 0 by the activation — markets need unbounded dollars.

Forward

One house traveling through depth

Step the same listing hop by hop. Watch which H1 neurons turn off before H2 even runs.

[1500, 3, 10] through 3 → 4 → 4 → 1

current hop

StatusStart with features

Want ≈ 295_000 someday. Fresh net (seed 37) scales to [1.5, 3, 1] — expectation: nonsense price.

Step 1 of 4

Numbers

Features → H1 → H2 → price

Rounded values for [1500, 3, 10] with seed 37. Square footage is thousands; we scale inputs so training does not explode. “Pre” is before ReLU; “out” is after.

Stage	Neuron 1	Neuron 2	Neuron 3	Neuron 4	Notes
Features	1500	3	10	—	sqft, beds, age
Scaled	1.5	3	1	—	sqft/1000, age/10
H1 pre	0.521	−0.229	0.515	0.294	weighted sums
H1 ReLU	0.521	0	0.515	0.294	off on neuron 2
H2 pre	0.094	0.183	−0.167	−0.075	sees H1 vector
H2 ReLU	0.094	0.183	0	0	on, on, off, off
Price	≈ 0.006				linear — no ReLU

Why no ReLU on the output? Sale price is not a non-negative activation — it is an unbounded regression target. If the last layer used ReLU, every negative pre-activation would clamp to 0 and the model could not freely sit above or below a target while learning. Hidden layers bend; the head reports a number.

Solution in TypeScript

forwardLogged is the architecture forward with a console.log after each layer. Run both houses with the same net — weights do not change between calls.

deep-forward-pass.tsTypeScript

type Vector = number[];
type Matrix = number[][];

function relu(x: number): number {
  return Math.max(0, x);
}

/** Square footage is thousands; we scale inputs so training does not explode. */
function scale(features: Vector): Vector {
  return [features[0] / 1000, features[1], features[2] / 10];
}

function mulberry32(seed: number): () => number {
  return () => {
    seed |= 0;
    seed = (seed + 0x6d2b79f5) | 0;
    let t = Math.imul(seed ^ (seed >>> 15), 1 | seed);
    t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}

class Layer {
  weights: Matrix;
  biases: Vector;
  activation: "relu" | "linear";

  constructor(
    inputSize: number,
    outputSize: number,
    activation: "relu" | "linear",
    rnd: () => number,
  ) {
    this.activation = activation;
    this.weights = Array.from({ length: outputSize }, () =>
      Array.from({ length: inputSize }, () => rnd() * 0.5 - 0.25),
    );
    this.biases = Array(outputSize).fill(0);
  }

  forward(inputs: Vector): { outputs: Vector; preActivations: Vector } {
    const preActivations: Vector = [];
    const outputs: Vector = [];
    for (let j = 0; j < this.weights.length; j++) {
      let sum = this.biases[j];
      for (let i = 0; i < inputs.length; i++) {
        sum += inputs[i] * this.weights[j][i];
      }
      preActivations.push(sum);
      outputs.push(this.activation === "relu" ? relu(sum) : sum);
    }
    return { outputs, preActivations };
  }
}

class DeepHouseNet {
  h1: Layer;
  h2: Layer;
  out: Layer;

  constructor(seed = 37) {
    const rnd = mulberry32(seed);
    this.h1 = new Layer(3, 4, "relu", rnd);
    this.h2 = new Layer(4, 4, "relu", rnd);
    this.out = new Layer(4, 1, "linear", rnd);
  }

  /** Same forward as architecture lesson — with a print at each hop. */
  forwardLogged(features: Vector): number {
    const x = scale(features);
    console.log("features", features, "→ scaled", x);

    const a = this.h1.forward(x);
    console.log("H1 pre ", a.preActivations.map((n) => n.toFixed(3)));
    console.log("H1 ReLU", a.outputs.map((n) => n.toFixed(3)));

    const b = this.h2.forward(a.outputs);
    console.log("H2 pre ", b.preActivations.map((n) => n.toFixed(3)));
    console.log("H2 ReLU", b.outputs.map((n) => n.toFixed(3)));

    const o = this.out.forward(b.outputs);
    const pred = o.outputs[0];
    console.log("price  ", pred.toFixed(3), "(linear — no ReLU)");
    return pred;
  }
}

const net = new DeepHouseNet();

net.forwardLogged([1500, 3, 10]);
// scaled [1.5, 3, 1]; H1 on/off: on, off, on, on  →  price ≈ 0.006

net.forwardLogged([900, 2, 40]);
// scaled [0.9, 2, 4]; H1 on/off: off, off, on, on  →  price ≈ 0.001
// Same weights. Different house → different ReLU gates.

Compare

Second house — same weights, different gates

Feed [900, 2, 40] through the identical DeepHouseNet. ReLU’s on/off pattern at H1 flips. That is the point of depth: the same W matrices carve different active paths for different inputs.

House	Scaled	H1 on/off	H2 on/off	Price guess	True price
`[1500, 3, 10]`	`[1.5, 3, 1]`	on, off, on, on	on, on, off, off	≈ 0.006	295_000
`[900, 2, 40]`	`[0.9, 2, 4]`	off, off, on, on	on, on, off, on	≈ 0.001	180_000

Same W — different H1 gates

focus

StatusListing A

Larger, newer-ish house. H1 neurons 1, 3, 4 fire; neuron 2 stays off.

Step 1 of 3

Still untrained. Both guesses are absurd versus true prices. Forward pass shows structure; backprop will assign blame and take a downhill step.

Trail

Deep House Architecture → Forward Pass Through Depth → Backprop Through Depth.

Topic	Description
Deep Learning	Machine learning with stacked neurons and ReLU layers: how a deep network builds a price guess from house features in TypeScript.
Deep House Architecture	Build a 3→4→4→1 ReLU network and run one forward pass on a house — no training yet.
Backprop Through Depth	Send price error backward through two ReLU layers and take one gradient-descent step.
Train the Deep House Model	Epochs over all five HOUSES: forward, backprop, update — watch loss fall.
Inference and Hold-Out	Freeze the net, price a new listing, and check a held-out house so train loss is not the whole story.