Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minor text formatting on resnet notebook #137

Merged
merged 1 commit into from
Sep 4, 2023
Merged

minor text formatting on resnet notebook #137

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 16 additions & 29 deletions notebooks/L96_ResNet_RNN.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,17 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Machine Learning with different neural networks"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "eIo6nnRzXDCR",
"id": "3Uai4mkDXGYx",
"tags": []
},
"outputs": [],
Expand All @@ -26,28 +33,10 @@
"\n",
"# from torch_lr_finder import LRFinder\n",
"import torch.nn.functional as F\n",
"from torch import nn"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "3Uai4mkDXGYx",
"tags": []
},
"outputs": [],
"source": [
"from torch import nn\n",
"from L96_model import L96, L96_eq1_xdot, integrate_L96_2t, EulerFwd, RK2, RK4"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Machine Learning with different neural networks "
]
},
{
"cell_type": "markdown",
"metadata": {
Expand Down Expand Up @@ -83,7 +72,6 @@
"source": [
"```{figure} https://www.researchgate.net/publication/319201436/figure/fig1/AS:869115023589376@1584224577926/Visualisation-of-a-two-scale-Lorenz-96-system-with-J-8-and-K-6-Global-scale-values.png\n",
":width: 400px\n",
":name: l96-equation-figure\n",
"\n",
"*Visualisation of a two-scale Lorenz '96 system with J = 8 and K = 6. Global-scale variables ($X_k$) are updated based on neighbouring variables and on the local-scale variables ($Y_{j,k}$) associated with the corresponding global-scale variable. Local-scale variabless are updated based on neighbouring variables and the associated global-scale variable. The neighbourhood topology of both local and global-scale variables is circular. Image from [Exploiting the chaotic behaviour of atmospheric models with reconfigurable architectures - Scientific Figure on ResearchGate.](https://www.researchgate.net/figure/Visualisation-of-a-two-scale-Lorenz-96-system-with-J-8-and-K-6-Global-scale-values_fig1_319201436)*.\n",
"\n",
Expand Down Expand Up @@ -208,8 +196,7 @@
"user_expressions": []
},
"source": [
"## **Janni's fully connected, 3-layer Artificial Neural Network (ANN)**\n",
"From meeting #4 on 28 May 2021"
"## Janni's fully connected, 3-layer Artificial Neural Network (ANN)"
]
},
{
Expand Down Expand Up @@ -425,7 +412,7 @@
"user_expressions": []
},
"source": [
"## **ResNet (sorta)**\n",
"## ResNet\n",
"\n",
"ResNet is based on residual blocks, depicted below \n",
"Residual blocks are based on skip connections \n",
Expand Down Expand Up @@ -822,11 +809,11 @@
"user_expressions": []
},
"source": [
"**Main points** \n",
"Skip connections are easy to program in pytorch \n",
"Deeper isn't always better \n",
"Residual structure can lower loss, and lead to faster training \n",
"Even shallow NN can benefit from using a residual block"
"### Main points\n",
" - Skip connections are easy to program in pytorch \n",
" - Deeper isn't always better \n",
" - Residual structure can lower loss, and lead to faster training \n",
" - Even shallow NN can benefit from using a residual block"
]
},
{
Expand Down