ReLU

The simplest rule for deciding how much signal a neuron passes on

A neural network is built out of many small units called neurons, and each one does a little bit of math and then has to decide how much of its answer to pass along to the next unit. ReLU, short for "rectified linear unit," is the simplest common rule for that decision: if the number is positive, pass it along exactly as it is; if the number is zero or negative, pass along zero instead.

def relu(x):
    """One number in, one number out. Never negative."""
    return max(0.0, x)


def relu_layer(values):
    """Apply ReLU to every number in a layer at once."""
    return [relu(v) for v in values]

What each part does

relu(x) takes one number, x, and returns whichever is bigger: x itself, or zero. That single comparison is the entire rule — there is nothing hidden inside it.

relu_layer(values) is the same rule applied to a whole list of numbers at once, because a real layer of a neural network is doing this to many neurons at the same time, not just one.

Why this shape

Any accepted submission in this category takes a list of numbers in and returns a list of the same length back out, with no number in the result ever below zero. That fixed shape is what lets a Forward Pass piece built by someone else plug straight into this one without either person ever meeting.