28 Neural Networks

Neural networks (NN): a brain metaphor for information processing

Artificial neural network (ANN)

Many uses for ANN for such as pattern recognition, forecasting, prediction, and classification
Many application areas such as finance, marketing, manufacturing, operations, information systems, and so on.

Neural Network Architectures

  • Several ANN architectures exist
  • Feedforward
  • Recurrent
  • Associative memory
  • Probabilistic
  • Self-organizing feature maps
  • Hopfield networks

 

Architecture of a neural network is driven by the task it is intended to address classification, regression, clustering, general optimization, association etc.
Most popular architecture: Feedforward, multi-layered perceptron with backpropagation learning algorithm
This architecture is used for both classification and regression type problems

Learning in ANN

A process by which a neural network learns the underlying relationship between input and outputs, or just among the inputs

Supervised learning

For prediction type problems.

Unsupervised learning

For clustering type problems.

Self-organizing

 

 

Backpropagation Learning

The learning algorithm procedure:
1.Initialize weights with random values and set other network parameters
2.Read in the inputs and the desired outputs
3.Compute the actual output (by working forward through the layers)
4.Compute the error (difference between the actual and desired output)
5.Change the weights by working backward through the hidden layers
6.Repeat steps 2-5 until weights stabilize

Testing a Trained ANN Model

Data is split into three parts
  1. Training (~60%)
  2. Validation (~20%)
  3. Testing (~20%)
nk-fold cross validation
  • Less bias
  • Time consuming

Application Types of ANN

Classification: Feedforward networks (MLP), radial basis function, and probabilistic NN
Regression: Feedforward networks (MLP), radial basis function
Clustering: Adaptive Resonance Theory (ART) and SOM
Association: Hopfield networks

Advantages of ANN

  1. Able to deal with (identify/model) highly nonlinear relationships
  2. Not prone to restricting normality and/or independence assumptions
  3. Can handle variety of problem types
  4. Usually provides better results (prediction and/or clustering) compared to its statistical counterparts
  5. Handles both numerical and categorical variables (transformation needed!)

Disadvantages of ANN

  1. They are deemed to be black-box solutions, lacking expandability
  2. It is hard to find optimal values for large number of network parameters
  3. Optimal design is still an art: requires expertise and extensive experimentation
  4. It is hard to handle large number of variables (especially the rich nominal attributes)
  5. Training may take a long time for large datasets; which may require case sampling

Python 3 Example: Please click here to see the Python3 Example.

License

Building Skills for Data Science Copyright © by Dr. Nouhad Rizk. All Rights Reserved.

Share This Book