Introduction to¶

NBIS Neural Nets & Deep Learning workshop¶

Who we are¶

Claudio	Bengt	Christophe	Dan	Per

Marcin	Nikolay	Carl	Olga	Eva

Who are you?¶

You're our Guinea pigs!¶

guinea Pigs

(taken from https://www.marinij.com/2018/03/05/guinea-pigs-have-lots-of-love-and-fun-to-share/)

Who are you?¶

Feedback session on Friday¶

We are counting on your feedback!

Are we ready to deliver this workshop externally?
Does the schedule work the way it's laid out?
What did you think of the lectures and labs?
Too simple/hard?
Write down one thing you liked and one you didn't at the end of each session, let us know at the end

Online classroom¶

[https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/classroom](https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/classroom)

Course website:
[https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/](https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/)
- schedule with links to lectures and exercises
The Zoom Main room for:
[https://uu-se.zoom.us/j/63739918797](https://uu-se.zoom.us/j/63739918797)
- lectures
- questions during exercises
Zoom breakout rooms will be used for discussions in smaller groups
Student slack channel:
https://nbisweden.slack.com/app_redirect?channel=nn_dl_students
- questions during lectures or exercises
- links to zoom and hacmd
HackMD:
[https://hackmd.io/fZc9DFyDSOO5Kd4RORb4Uw](https://hackmd.io/fZc9DFyDSOO5Kd4RORb4Uw)
- working area for exercises, etc.

Course website¶

[https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/](https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/)

Online classroom¶

[https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/classroom](https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/classroom)

Schedule¶

[https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/schedule](https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/schedule)

Learning outcomes¶

[https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/learningOutcomes](https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/learningOutcomes)

Prerequisites¶

[https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/prerequisites](https://nbisweden.github.io/workshop-neural-nets-and-deep-learning/prerequisites)

HackMD¶

[https://hackmd.io/fZc9DFyDSOO5Kd4RORb4Uw](https://hackmd.io/fZc9DFyDSOO5Kd4RORb4Uw?view)

Practical issues¶

Try to keep your cameras on, but microphone muted
Lectures with many breaks
Take lots of small breaks also when working with the exercises
We will try to stick to the schedule, but it's only preliminary until it's happened
If you have any questions during the lecture, feel free to unmute and ask. If you don't want to ask in the Zoom meeting, write the question in the chat or on slack

So, this course is about Artificial Neural networks (Anns)¶

.... What is that?¶

AI vs ML vs ANN vs DL¶

ai_ml_ann_dl

(taken from https://medium.com/ai-in-plain-english/artificial-intelligence-vs-machine-learning-vs-deep-learning-whats-the-difference-dccce18efe7f)

AI -- Artificial Intelligence¶

The science and engineering of creating intelligent machines

or machines that behave as they were intelligent

ML -- Machine learning¶

Subset of AI
Computer algorithms that learn/improve from experience

ANN -- Artificial Neural Networks¶

Subset of ML
Model of human neural networks
Networks built up by artificial neurons

DL -- Deep Learning¶

Subset of ANN (? -- depend on definition of ANN)
ANN with more than one hidden layer

So where does statistics fit in?¶

Statistics and ML -- parallel and overlapping?¶

methodological elements/framework
are both data-dependent
both have "models"
- Stat: explicit probabilistic model
- ML: implicit "graphical" model
  - (samples from array of models)
problems
- over-fitting
- over-parameterization

Typically differ in¶

optimization
- Stat: analytically under the probabilistic model
- ML: learns from training data, evaluates on test data
main aims
- Stat: infer relationships between variables
- ML: predictions about unseen data
size of data
- stat: small vs ML: big data
  - variables and samples

... and what about Data Science?¶

Neuron

(taken from https://www.javatpoint.com/data-science-vs-machine-learning)

Neuron

(taken from https://crate.io/a/machine-learning-cratedb-jupyter/)

... ~Don't know~ I mean, outside the scope of this study¶

Where does ANN come from?¶

History¶

First wave of ANN (funding)¶

1943 First model of biological neruon (McCulloch & Pitts)
1949 Hebbian Learning rules for ANN, (Donald Hebb)
1951 First ANN (Minsky)
1958 The Perceptron -- linear classification
1969 Limitations of the perceptron (Minsky & Papert)

Neuron

(taken from https://www.sciencedirect.com/topics/neuroscience/perceptron)

Neuron

(taken from https://news.cornell.edu/stories/2019/09/professors-perceptron-paved-way-ai-60-years-too-soon)

History¶

Second wave of ANN (funding)¶

(1975 Back propagation algorithm (Werbos))
1986 Back propagation rediscovered and sigmoid activation -- ANNs in vogue again
1985-87 ANN conferences, societies started (IEEE, INNS)
Vanishing gradient problem
SVM outcompetes ANNs, ANNs survives in protein structure prediction (e.g., CASP)

Neuron

(taken from https://www.kdnuggets.com/2019/01/backpropagation-algorithm-demystified.html)

Neuron

(taken from https://predictioncenter.org)

History¶

Third wave of ANN (funding)¶

GPUs and parallelization
"Deep Learning" buzzword and performing well in contests
1997 Recursive Neural Nets. LSTMs (Smidhuber & Hochreiter)
2000 ReLU (Hahnloser et al.?)
2011 CNNs
ANNs are everywhere

Neuron

(taken from https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53)

Applications -- general¶

Written and spoke language recognition mnist

(taken from Mouret, Jean-Baptiste & Doncieux, Stéphane. (2008). Evolutionary Intelligence

(Autoencoder) showing recogniytion of written text. ALso mention sentences more general langauge grammar and grammatics -- NLPN

Risk predictions mnist

(taken from https://spectrum.ieee.org/computing/embedded-systems/bringing-big-neural-networks-to-selfdriving-cars-smartphones-and-drones)

Self-driving cars, risk estimation

Forecasting, weather, business

Neuron

(taken from https://www.gjesm.net/article_23079.html)

Forecasting, e.g, weather

Face recognition and generation mnist

(taken from https://medium.com/syncedreview/gan-2-0-nvidias-hyperrealistic-face-generator-e3439d33ebaf)

Applications -- Biosciences¶

Protein structure prediction Neuron

(taken from https://www.theverge.com/2020/12/1/21754310/deepmind-alphafold-ai-protein-folding-casp-competition)

Alphafold ooutcompetes other methods in CASP 2020

Bioimaging diagnostics

(taken from https://peerj.com/articles/6201/)

Figure 3: An example of image enhancement. (A) Original malignant mass case extracted from DDSM, (B) Enhanced image using CLAHE, and (C) Histogram representation of the image.

Spatial transcriptomics Neuron

(taken from https://www.uu.se/en/news-media/press-releases/press-release/?id=5226&typ=pm&lang=en)

Decrypting spatial transcriptomics heterogeneity (spage2vec) in complex tissues

resolution
balance between intrinsic regul network and extrinsic subcell processes

Time for a break¶

pattern recognition
non-linear classification/regression?
- signature verification
- cybersecurity
image/character recognition
- handwriting (mnist)
- bioimages
- faces
- 3D object recognition
forecasting/prediction
- business
- medical diagnosis
- generate images (faces)

dimensional reduction
- bar-coding
sequence recognition
- speech
- text
- NLP
data mining
time series
protein structure prediciton

Introduction to¶

NBIS Neural Nets & Deep Learning workshop¶

Who we are¶

Who are you?¶

You're our Guinea pigs!¶

Who are you?¶

Who are you?¶

Feedback session on Friday¶

Online classroom¶

Course website¶

Online classroom¶

Schedule¶

Learning outcomes¶

Prerequisites¶

HackMD¶

Practical issues¶

So, this course is about Artificial Neural networks (Anns)¶

.... What is that?¶

AI vs ML vs ANN vs DL¶

AI -- Artificial Intelligence¶

ML -- Machine learning¶

ANN -- Artificial Neural Networks¶

DL -- Deep Learning¶

So where does statistics fit in?¶

Statistics and ML -- parallel and overlapping?¶

Share¶

Typically differ in¶

... and what about Data Science?¶

... ~Don't know~ I mean, outside the scope of this study¶

Where does ANN come from?¶

History¶

First wave of ANN (funding)¶

History¶

Second wave of ANN (funding)¶

History¶

Third wave of ANN (funding)¶

Applications -- general¶

Applications -- Biosciences¶

Time for a break¶