Expert Answer
Anonymous
A neural network is a function approximation algorithm inspired by the brain. It is a dense network of nodes, which store values, and edges, which connect nodes. The simplest model is a dense sequential network which has layers of nodes. This network would be called dense when all nodes of one layer are connected to the nodes of the next layer. The network is called sequential because each layer activates one after the other.
The value of a node is determined by all of the nodes that are connected to it in the previous layers. Each of these values will be multiplied by a weight, and then a bias will be added. This value is passed through an activation function and then result is then the value of the node.
Typical activation functions are sigmoid, which is shaped like a very wide S, and a relu (rectified linear unit), which is like a hockey stick. The sigmoid activation function is tied to the way that axons in the brain have threshold response functions.
Neural networks are trained by using the gradient between the output value in the dataset and predicted value. This update is propagated through the network using backpropagation.
Neural networks are useful because they are universal approximators. This means that an infinitely wide network with one layer input input and output can represent an arbitrary function. This contributes to them being adept at so many tasks.
