Neural Network Layer As Matrix Transformation

Definition

A neural network layer can be understood as a transformation of a vector.

At the simplest level:

input vector -> matrix transformation -> output vector

Example:

output = W * input

where W is a matrix of weights.

A layer can start as a matrix acting on an input vector.

---

Input Vector

The input to a layer is a vector.

Example:

input = [x1, x2, x3]

Each component is one input value.

The layer transforms these values into new values.

A layer receives structured numerical data.

---

Weight Matrix

The matrix contains weights.

Example:

W =
[w11 w12 w13]
[w21 w22 w23]

This matrix has:

2 rows
3 columns

It can transform a 3-component input into a 2-component output.

The weight matrix controls how input components combine.

---

Output Components

Each output component comes from one row of the matrix.

Example:

input = [x1, x2, x3]

W =
[w11 w12 w13]
[w21 w22 w23]

Output:

first output  = w11*x1 + w12*x2 + w13*x3
second output = w21*x1 + w22*x2 + w23*x3

Each row defines one weighted combination of the input.

---

Weighted Sum

Each output component is a weighted sum.

Example:

input = [2, 3]
weights = [4, 5]

Weighted sum:

4*2 + 5*3 = 23

This is just a dot product.

A layer output component is often built from a dot product.

---

Bias Term

Many neural network layers also add a bias.

Basic form:

output = W * input + b

The matrix part transforms the input.

The bias shifts the result.

Important distinction:

W * input is linear
W * input + b is affine

We do not need the full term yet.

The key idea is that the matrix does the structured mixing.

Bias adds a shift after the matrix transformation.

---

Why This Matters

This connects matrices to learning systems.

A layer can learn weights that decide:

which input components matter
how strongly they matter
how components are combined

The matrix packages many weighted sums into one operation.

Neural network layers use matrix structure to transform representations.

---

Do Not Add Mystery Yet

At this level, a layer is not magic.

It is:

numbers in
weighted combinations
numbers out

Later chapters will add activation functions, loss, and training.

For now, the important foundation is:

matrix transformation creates new vector representation

A neural layer starts as structured numerical transformation.

---

Final Summary

• A layer can take a vector as input.
• A weight matrix transforms the input vector.
• Each matrix row creates one output component.
• Each output component is a weighted sum.
• Bias can shift the output after the matrix transformation.
• The matrix part is the structured transformation.
• This prepares the move toward perceptrons and neural networks.