How would you define a Neural Network and describe how it functions?

Question

Anonymous · Accepted Answer

A neural network is a function approximation algorithm inspired by the brain. It
is a dense network of nodes, which store values, and edges, which connect nodes.
The simplest model is a dense sequential network which has layers of nodes. This
network would be called dense when all nodes of one layer are connected to the
nodes of the next layer. The network is called sequential because each layer
activates one after the other.

The value of a node is determined by all of the nodes that are connected to it
in the previous layers. Each of these values will be multiplied by a weight, and
then a bias will be added. This value is passed through an activation function
and then result is then the value of the node.

Typical activation functions are sigmoid, which is shaped like a very wide S,
and a relu (rectified linear unit), which is like a hockey stick. The sigmoid
activation function is tied to the way that axons in the brain have threshold
response functions.

Neural networks are trained by using the gradient between the output value in
the dataset and predicted value. This update is propagated through the network
using backpropagation.

Neural networks are useful because they are universal approximators. This means
that an infinitely wide network with one layer input input and output can
represent an arbitrary function. This contributes to them being adept at so many
tasks.

How would you define a Neural Network and describe how it functions?

Answers