Why One Perceptron Fails

Definition

One perceptron fails on XOR because one perceptron can only create one linear decision boundary.

Perceptron score:

score = dot(w, x) + b

Decision boundary:

dot(w, x) + b = 0

In 2D, that boundary is one line.

One perceptron means one linear separator.

---

XOR Requires More Than One Linear Region

XOR classes:

class 0: [0,0], [1,1]
class 1: [0,1], [1,0]

Class 1 appears in two separated corners.

Class 0 appears in the other two separated corners.

One line cannot isolate both positive corners from both negative corners.

XOR needs a decision structure that is not one straight split.

---

Weight Choice Cannot Fix It

Changing weights changes the line.

Example:

w = [1, 0]
w = [0, 1]
w = [1, -1]

Each weight vector gives a different line orientation.

Changing bias moves the line.

But no orientation and position solves XOR perfectly.

Weights and bias can move the line, but they cannot make it non-linear.

---

The Perceptron Is Doing What It Can

The perceptron is not broken.

It is following its structure:

input -> weighted sum -> threshold -> class

That structure creates a linear boundary.

XOR needs a different representational structure.

The perceptron fails because the task is outside its representational capacity.

---

Capacity Limit

Model capacity means what kinds of patterns a model can represent.

A single perceptron can represent:

linearly separable patterns

It cannot represent:

XOR in the original 2D input space

XOR is beyond the capacity of one perceptron.

---

What Would Help

To solve XOR, we need to change the representation or use more structure.

Options:

• add a useful feature
• use multiple linear boundaries
• use a hidden representation
• use a multi-layer network

The key idea is not just more arithmetic.

It is a better representation.

XOR motivates hidden representation.

---

Why This Matters

This is the first major reason to move beyond a single linear classifier.

The problem is small and clear.

But it reveals a deep idea:

learning is not only about fitting weights
learning is also about finding useful representations

XOR shows why representation is central.

---

Final Summary

• One perceptron creates one linear decision boundary.
• XOR is not linearly separable in the original input space.
• Changing weights and bias only moves or rotates the line.
• It cannot create the needed nonlinear separation.
• The failure is about representational capacity.
• Solving XOR requires a changed representation or more model structure.