Perceptron

A perceptron is the fundamental building block of neural networks, originally proposed by Frank Rosenblatt in 1958. It takes multiple inputs, multiplies each by a learnable weight, sums them together with a bias term, and passes the result through a step function (or other activation) to produce a binary output.

The perceptron can learn to classify linearly separable data by adjusting its weights through a simple learning rule. However, it famously cannot solve problems like XOR that require non-linear decision boundaries. This limitation was highlighted by Minsky and Papert in 1969 and led to the first AI winter. The solution came decades later with multi-layer perceptrons (MLPs) -- stacking multiple layers of perceptrons with non-linear activations, which can approximate any continuous function.

Modern neural networks are essentially vast networks of perceptron-like units. Each neuron in a neural network performs the same fundamental operation: weighted sum plus bias, followed by a non-linear activation. The key insight is that while a single perceptron is limited, layers of them create the representational power needed for deep learning.