28 Neural Networks
Neural networks (NN): a brain metaphor for information processing
Artificial neural network (ANN)
Many uses for ANN for such as pattern recognition, forecasting, prediction, and classification
Many application areas such as finance, marketing, manufacturing, operations, information systems, and so on.
Neural Network Architectures
- Several ANN architectures exist
- Feedforward
- Recurrent
- Associative memory
- Probabilistic
- Self-organizing feature maps
- Hopfield networks
Architecture of a neural network is driven by the task it is intended to address classification, regression, clustering, general optimization, association etc.
Most popular architecture: Feedforward, multi-layered perceptron with backpropagation learning algorithm
This architecture is used for both classification and regression type problems
Learning in ANN
A process by which a neural network learns the underlying relationship between input and outputs, or just among the inputs
Supervised learning
For prediction type problems.
Unsupervised learning
For clustering type problems.
Self-organizing
Backpropagation Learning
The learning algorithm procedure:
1.Initialize weights with random values and set other network parameters
2.Read in the inputs and the desired outputs
3.Compute the actual output (by working forward through the layers)
4.Compute the error (difference between the actual and desired output)
5.Change the weights by working backward through the hidden layers
6.Repeat steps 2-5 until weights stabilize
Testing a Trained ANN Model
Data is split into three parts
- Training (~60%)
- Validation (~20%)
- Testing (~20%)
nk-fold cross validation
- Less bias
- Time consuming
Application Types of ANN
Classification: Feedforward networks (MLP), radial basis function, and probabilistic NN
Regression: Feedforward networks (MLP), radial basis function
Clustering: Adaptive Resonance Theory (ART) and SOM
Association: Hopfield networks
Advantages of ANN
- Able to deal with (identify/model) highly nonlinear relationships
- Not prone to restricting normality and/or independence assumptions
- Can handle variety of problem types
- Usually provides better results (prediction and/or clustering) compared to its statistical counterparts
- Handles both numerical and categorical variables (transformation needed!)
Disadvantages of ANN
- They are deemed to be black-box solutions, lacking expandability
- It is hard to find optimal values for large number of network parameters
- Optimal design is still an art: requires expertise and extensive experimentation
- It is hard to handle large number of variables (especially the rich nominal attributes)
- Training may take a long time for large datasets; which may require case sampling
Python 3 Example: Please click here to see the Python3 Example.