Commit 751f8453 authored by Carsten Eie Frigaard's avatar Carsten Eie Frigaard
Browse files

finalized_keras_installation_demo

parent 031364ef
...@@ -217,7 +217,7 @@ ...@@ -217,7 +217,7 @@
], ],
"metadata": { "metadata": {
"kernelspec": { "kernelspec": {
"display_name": "Python 3", "display_name": "Python 3 (ipykernel)",
"language": "python", "language": "python",
"name": "python3" "name": "python3"
}, },
...@@ -231,7 +231,7 @@ ...@@ -231,7 +231,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.8.5" "version": "3.9.7"
}, },
"varInspector": { "varInspector": {
"cols": { "cols": {
......
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# ITMAL Demo # SWMAL Demo
## Installing Keras and Tensorflow for Anaconda 2021.11 ## Installing Keras and Tensorflow for Anaconda 2021.11
REVISIONS| | REVISIONS| |
---------| | ---------| |
2018-03-25| CEF, initial. 2018-03-25| CEF, initial.
2020-03-05| CEF, F20 ITMAL update. 2020-03-05| CEF, F20 ITMAL update.
2020-03-06| CEF, investigated Anaconda 2019.10 on Windows and updated GPU server notes. 2020-03-06| CEF, investigated Anaconda 2019.10 on Windows and updated GPU server notes.
2021-10-12| CEF, updated for ITMAL E21. 2021-10-12| CEF, updated for ITMAL E21.
2022-03-23| CEF, updated for SWMAL F22, rewrote install method for Keras via environments. 2022-03-23| CEF, updated for SWMAL F22, rewrote install method for Keras via environments.
2022-03-25| CEF, elaboreated on conda and pip, and added pip install and screenshots. 2022-03-25| CEF, elaborated on conda and pip, and added pip install and screenshots.
### About Anaconda Package Managers ### About Anaconda Package Managers
Anaconda comes with two different package managers: `conda` and `pip` Anaconda comes with two different package managers: `conda` and `pip`
* `conda` is the prefered package manager, that will check new package installation for compability with existing packages. While this pre-compability-check is nice in theory, it often turns out, that it becomes impossible to install a given package due to incompabilities with one or more other packages. Even more annoing, the simple install of a package may take hours to complete. * `conda` is the prefered package manager, that will check new package installation for compatibility with existing packages. While this pre-compatibility-check is nice in theory, it often turns out, that it becomes impossible to install a given package due to incompatibilities with one or more other packages. Even more annoying, the simple installation of a package may take hours to complete.
* `pip` is a simpler package mangager, that comes with most Python distributions. It does none of the compability-checks when installing; it just installs whatever you ask it to (and incompabilites may then creep in in the run-time enviroment). * `pip` is a simpler package manager, that comes with most Python distributions. It does none of the compatibility-checks when installing; it just installs whatever you ask it to (and incompatibility may then creep in in the run-time environment).
Normally you would try to install whatever you need via `conda` and when it breaks down due to to many broken dependencies (trying with 'flexible solve' etc. for hours), you just continue with `pip`. Normally you would try to install whatever you need via `conda` and when it breaks down due to to many broken dependencies (trying with 'flexible solve' etc. for hours), you just continue with `pip`.
If you just want a quick(-and-dirty?) install of `keras` and `tensorflow` goto the "Install via PIP" cell, otherwise try the "Install via CONDA".. If you just want a quick(-and-dirty?) install of `keras` and `tensorflow` goto the "Install via PIP" cell, otherwise try the "Install via CONDA"..
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Installing via PIP ## Installing via PIP
If `conda` fails, or you just want to proceede quickly, then install everything in the base environment via pip If `conda` fails, or you just want to proceed quickly, then install everything in the base environment via pip
```bash ```bash
> pip install keras tensorflow > pip install keras tensorflow
``` ```
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_pip_install_tensorflow.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_pip_install_tensorflow.png" alt="WARNING: could not get image from server." style="width:800px">
and then just launch the jupyter-notebook in the default (base) environment. and then just launch the jupyter-notebook in the default (base) environment.
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_pip_install_run_notebook.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_pip_install_run_notebook.png" alt="WARNING: could not get image from server." style="width:800px">
and test the installed versions via the `Version()` function in `libitmal` and test the installed versions via the `Version()` function in `libitmal`
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from libitmal import versions from libitmal import versions
versions.Versions() versions.Versions()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
I got the output, with a `keras` and `tensorflow` version at 2.8,0 (instead of 2.6.0 from a `conda` install): I got the output, with a `keras` and `tensorflow` version at 2.8,0 (instead of 2.6.0 from a `conda` install):
``` ```
Python version: 3.9.7. Python version: 3.9.7.
Scikit-learn version: 0.24.2. Scikit-learn version: 0.24.2.
Keras version: 2.8.0 Keras version: 2.8.0
Tensorflow version: 2.8.0 Tensorflow version: 2.8.0
Tensorflow.keras version: 2.8.0 Tensorflow.keras version: 2.8.0
OK OK
``` ```
NOTE: this pip install needs testing and verification --- please report if it does not work on your PC! NOTE: this pip install needs testing and verification --- please report if it does not work on your PC!
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## installing via CONDA ## installing via CONDA
Keras will not install under Anaconda version 2021.11. It ends up in a endless package-conflict, when installing under `conda`. Keras will not install under Anaconda version 2021.11.It ends up in an endless package conflict when installing under `conda`.
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_conflicts.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_conflicts.png" alt="WARNING: could not get image from server." style="width:800px">
(This is a new finding for the particular version 2021.11, and previous version did not have this tensorflow install problem, but typically only a Keras install problem). (This is a new finding for the particular version 2021.11, and the previous version did not have this tensorflow install problem, but typically only a Keras install problem).
The root-cause of the problem must be missing check when packageing the Anaconda, failing to find the set of conflicts we see, when installing `tensorflow` in the set of default installe packages that comes with the Anaconda distribution. The root cause of the problem must be missing checks when packaging the Anaconda, failing to find the set of conflicts we see, when installing `tensorflow` in the set of default installed packages that come with the Anaconda distribution.
#### 1) Prepare and Ceate a new Environment
#### 1) Prepare and Create a new Environment
So, one solution is to create a new conda environment, and from this install `scikit-learn` and `tensorflow` and `keras`, So, one solution is to create a new conda environment, and from this install `scikit-learn` and `tensorflow` and `keras`,
Later we need a package called `nb_conda_kernels`, let us install this before we create and activate the new enviroment Later we need a package called `nb_conda_kernels`, let us install this before we create and activate the new environment
```bash ```bash
(base)> conda install nb_conda_kernels (base)> conda install nb_conda_kernels
``` ```
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_preinstall.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_preinstall.png" alt="WARNING: could not get image from server." style="width:800px">
Now, let us call our enviroment `swmal` and create it by running Now, let us call our environment `swmal` and create it by running
```bash ```bash
(base)> conda create --name swmal (base)> conda create --name swmal
``` ```
Activate the newly created enviroment it via Activate the newly created environment via
```bash ```bash
(base)> conda activate swmal (base)> conda activate swmal
``` ```
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_create_env.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_create_env.png" alt="WARNING: could not get image from server." style="width:800px">
#### 2) Install Needed Packages #### 2) Install Needed Packages
Now we got a clean state enviroment and we need to install the packages needed for `scikit-learn` and `tensorflow`, but this is as easy as Now we got a clean state environment and we need to install the packages needed for `scikit-learn` and `tensorflow`, but this is as easy as
```bash ```bash
(swmal)> conda install scikit-learn tensorflow keras nb_conda_kernels (swmal)> conda install scikit-learn tensorflow keras nb_conda_kernels
``` ```
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_installing.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_installing.png" alt="WARNING: could not get image from server." style="width:800px">
Notice that the package `nb_conda_kernels` also needs to be installed in this new enviroment, and it actually also installes the jupyter notebook packages.
#### 3) Launching Jupyter-Notebooks in the new Enviroment <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_installing_done.png" alt="WARNING: could not get image from server." style="width:800px">
Notice that the package `nb_conda_kernels` also needs to be installed in this new environment, and it actually also installs the jupyter notebook packages.
#### 3) Launching Jupyter-Notebooks in the new Environment
There are two `keras` interfaces, the stand-alone `keras`, and then a similar interface already build into `tensorflow.keras`. You could use both, but direct access via the stand-alone inteface may be the most obvious. There are two `keras` interfaces, the stand-alone `keras`, and then a similar interface already build into `tensorflow.keras`. You could use both, but direct access via the stand-alone interface may be the most obvious.
```bash ```bash
(swmal)> jupyter-notebook (swmal)> jupyter-notebook
``` ```
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_run_notebook.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_run_notebook.png" alt="WARNING: could not get image from server." style="width:800px">
For some reason there now is a missing function in the `pythoncom39.dll` reported via the "Entry Point not Found" dialog,, but just ignore this warning for now (or find a fix and share it). For some reason there now is a missing function in the `pythoncom39.dll` reported via the "Entry Point Not Found" dialog, but just ignore this warning for now (or find a fix and share it).
#### 4) Testing the New Enviroment Setup You can change the environment in the Jupyter-Notebook via the menu "Kernel | Change Kernel" and this screendump shows how the environment is set to "swmal" when launching the Notebook in the activated environment above
Lets see the version installed in the new `swmal` environment via the `Versions()` function found in the `itmallib` <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_run_notebook2.png" alt="WARNING: could not get image from server." style="width:800px">
#### 4) Testing the New Environment Setup
Let us see the version installed in the new `swmal` environment via the `Versions()` function found in the `itmallib`
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from libitmal import versions from libitmal import versions
versions.Versions() versions.Versions()
``` ```
%% Output %% Output
Python version: 3.9.7. Python version: 3.9.7.
Scikit-learn version: 1.0.2. Scikit-learn version: 1.0.2.
Keras version: 2.6.0 Keras version: 2.6.0
Tensorflow version: 2.6.0 Tensorflow version: 2.6.0
Tensorflow.keras version: 2.6.0 Tensorflow.keras version: 2.6.0
Opencv2 version: 4.5.5 Opencv2 version: 4.5.5
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The `Versions()` function should print whatever version you installed, or produce a warning, if the package is not installed at all. The `Versions()` function should print whatever version you installed, or produce a warning, if the package is not installed at all.
For my current Windows/Anaconda setup I got the versions, Yours may differ slightly For my current Windows/Anaconda setup I got the versions, Yours may differ slightly
``` ```
Python version: 3.9.7. Python version: 3.9.7.
Scikit-learn version: 1.0.2. Scikit-learn version: 1.0.2.
Keras version: 2.6.0 Keras version: 2.6.0
Tensorflow version: 2.6.0 Tensorflow version: 2.6.0
Tensorflow.keras version: 2.6.0 Tensorflow.keras version: 2.6.0
``` ```
#### 5) Wrapping It All Up in a BAT File #### 5) Wrapping It All Up in a BAT File
To make development easy, a BAT (Windows batch or script file) shold be created. This should ease the launch of Jupyter-Notebooks and the BAT file could be put in a icon placed on the taskbar or similar. To make development easy, a BAT (Windows batch or script file) should be created. This should ease the launch of Jupyter-Notebooks and the BAT file could be put in an icon placed on the taskbar or similar.
The BAT file should contain the following text lines, and you can place this in an icon in the taskbar allowing for easy launch of future notebooks.
The BAT-file should contain the followin text lines, and you can place this in an icon in the taskbar allowing for easy launch of future notebooks.
``` ```
@ECHO OFF @ECHO OFF
REM my-jupyter-notebook REM my-jupyter-notebook
REM Version: 0.1 REM Version: 0.1
REM 2022-03-23: CEF, inital version REM 2022-03-23: CEF, inital version
echo MY-JUPYTER-NOTEBOOK launcher.. echo MY-JUPYTER-NOTEBOOK launcher..
REM %windir%\System32\cmd.exe "/K" %HOMEPATH%\Anaconda3\Sc2 ripts\activate.bat %HOMEPATH%\Anaconda3 swmal REM %windir%\System32\cmd.exe "/K" %HOMEPATH%\Anaconda3\Sc2 ripts\activate.bat %HOMEPATH%\Anaconda3 swmal
@CALL "%HOMEPATH%\Anaconda3\condabin\conda.bat" activate swmal %* @CALL "%HOMEPATH%\Anaconda3\condabin\conda.bat" activate swmal %*
REM note book start in this directory, you may change it: REM note book start in this directory, you may change it:
cd \ cd \
jupyter-notebook jupyter-notebook
echo DONE echo DONE
``` ```
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment