Commit 031364ef authored by Carsten Eie Frigaard's avatar Carsten Eie Frigaard
Browse files

update

parent 1f262449
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# ITMAL Demo # ITMAL Demo
## Installing Keras and Tensorflow for Anaconda 2021.11 ## Installing Keras and Tensorflow for Anaconda 2021.11
REVISIONS| | REVISIONS| |
---------| | ---------| |
2018-03-25| CEF, initial. 2018-03-25| CEF, initial.
2020-03-05| CEF, F20 ITMAL update. 2020-03-05| CEF, F20 ITMAL update.
2020-03-06| CEF, investigated Anaconda 2019.10 on Windows and updated GPU server notes. 2020-03-06| CEF, investigated Anaconda 2019.10 on Windows and updated GPU server notes.
2021-10-12| CEF, updated for ITMAL E21. 2021-10-12| CEF, updated for ITMAL E21.
2022-03-23| CEF, updated for SWMAL F22, rewrote install method for Keras via environments. 2022-03-23| CEF, updated for SWMAL F22, rewrote install method for Keras via environments.
2022-03-25| CEF, elaboreated on conda and pip, and added pip install and screenshots. 2022-03-25| CEF, elaboreated on conda and pip, and added pip install and screenshots.
### About Anaconda Package Managers ### About Anaconda Package Managers
Anaconda comes with two different package managers: `conda` and `pip` Anaconda comes with two different package managers: `conda` and `pip`
* `conda` is the prefered package manager, that will check new package installation for compability with existing packages. While this pre-compability-check is nice in theory, it often turns out, that it becomes impossible to install a given package due to incompabilities with one or more other packages. Even more annoing, the simple install of a package may take hours to complete. * `conda` is the prefered package manager, that will check new package installation for compability with existing packages. While this pre-compability-check is nice in theory, it often turns out, that it becomes impossible to install a given package due to incompabilities with one or more other packages. Even more annoing, the simple install of a package may take hours to complete.
* `pip` is a simpler package mangager, that comes with most Python distributions. It does none of the compability-checks when installing; it just installs whatever you ask it to (and incompabilites may then creep in in the run-time enviroment). * `pip` is a simpler package mangager, that comes with most Python distributions. It does none of the compability-checks when installing; it just installs whatever you ask it to (and incompabilites may then creep in in the run-time enviroment).
Normally you would try to install whatever you need via `conda` and when it breaks down due to to many broken dependencies (trying with 'flexible solve' etc. for hours), you just continue with `pip`. Normally you would try to install whatever you need via `conda` and when it breaks down due to to many broken dependencies (trying with 'flexible solve' etc. for hours), you just continue with `pip`.
If you just want a quick(-and-dirty?) install of `keras` and `tensorflow` goto the "Install via PIP" cell, otherwise try the "Install via CONDA".. If you just want a quick(-and-dirty?) install of `keras` and `tensorflow` goto the "Install via PIP" cell, otherwise try the "Install via CONDA"..
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Installing via PIP ## Installing via PIP
If `conda` fails, or you just want to proceede quickly, then install everything in the base environment via pip If `conda` fails, or you just want to proceede quickly, then install everything in the base environment via pip
```bash ```bash
> pip install keras tensorflow > pip install keras tensorflow
``` ```
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_pip_install_tensorflow.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_pip_install_tensorflow.png" alt="WARNING: could not get image from server." style="width:800px">
and then just launch the jupyter-notebook in the default (base) environment. and then just launch the jupyter-notebook in the default (base) environment.
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_pip_install_run_notebook.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_pip_install_run_notebook.png" alt="WARNING: could not get image from server." style="width:800px">
and test the installed versions via the `Version()` function in `libitmal` and test the installed versions via the `Version()` function in `libitmal`
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from libitmal import versions from libitmal import versions
versions.Versions() versions.Versions()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
I got the output, with a `keras` and `tensorflow` version at 2.8,0 (instead of 2.6.0 from a `conda` install): I got the output, with a `keras` and `tensorflow` version at 2.8,0 (instead of 2.6.0 from a `conda` install):
``` ```
Python version: 3.9.7. Python version: 3.9.7.
Scikit-learn version: 0.24.2. Scikit-learn version: 0.24.2.
Keras version: 2.8.0 Keras version: 2.8.0
Tensorflow version: 2.8.0 Tensorflow version: 2.8.0
Tensorflow.keras version: 2.8.0 Tensorflow.keras version: 2.8.0
OK OK
``` ```
NOTE: this pip install needs testing and verification --- please report if it does not work on your PC! NOTE: this pip install needs testing and verification --- please report if it does not work on your PC!
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## installing via CONDA ## installing via CONDA
Keras will not install under Anaconda version 2021.11. It ends up in a endless package-conflict, when installing under `conda`. Keras will not install under Anaconda version 2021.11. It ends up in a endless package-conflict, when installing under `conda`.
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_conflicts.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_conflicts.png" alt="WARNING: could not get image from server." style="width:800px">
(This is a new finding for the particular version 2021.11, and previous version did not have this tensorflow install problem, but typically only a Keras install problem). (This is a new finding for the particular version 2021.11, and previous version did not have this tensorflow install problem, but typically only a Keras install problem).
The root-cause of the problem must be missing check when packageing the Anaconda, failing to find the set of conflicts we see, when installing `tensorflow` in the set of default installe packages that comes with the Anaconda distribution. The root-cause of the problem must be missing check when packageing the Anaconda, failing to find the set of conflicts we see, when installing `tensorflow` in the set of default installe packages that comes with the Anaconda distribution.
#### 1) Prepare and Ceate a new Environment #### 1) Prepare and Ceate a new Environment
So, one solution is to create a new conda environment, and from this install `scikit-learn` and `tensorflow` and `keras`, So, one solution is to create a new conda environment, and from this install `scikit-learn` and `tensorflow` and `keras`,
Later we need a package called `nb_conda_kernels`, let us install this before we create and activate the new enviroment Later we need a package called `nb_conda_kernels`, let us install this before we create and activate the new enviroment
```bash ```bash
(base)> conda install nb_conda_kernels (base)> conda install nb_conda_kernels
``` ```
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_preinstall.png" alt="WARNING: could not get image from server." style="width:800px">
Now, let us call our enviroment `swmal` and create it by running Now, let us call our enviroment `swmal` and create it by running
```bash ```bash
(base)> conda create --name swmal (base)> conda create --name swmal
``` ```
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_create_env.png" alt="WARNING: could not get image from server." style="width:800px">
(your install drive is probally `C:\`, I used `Z:\` due to my virtual machine environment.)
Activate the newly created enviroment it via Activate the newly created enviroment it via
```bash ```bash
(base)> conda activate swmal (base)> conda activate swmal
``` ```
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_activate_env.png" alt="WARNING: could not get image from server." style="width:600px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_create_env.png" alt="WARNING: could not get image from server." style="width:800px">
#### 2) Install Needed Packages #### 2) Install Needed Packages
Now we got a clean state enviroment and we need to install the packages needed for `scikit-learn` and `tensorflow`, but this is as easy as Now we got a clean state enviroment and we need to install the packages needed for `scikit-learn` and `tensorflow`, but this is as easy as
```bash ```bash
(swmal)> conda install scikit-learn tensorflow keras nb_conda_kernels (swmal)> conda install scikit-learn tensorflow keras nb_conda_kernels
``` ```
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_installing.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_installing.png" alt="WARNING: could not get image from server." style="width:800px">
Notice that the package `nb_conda_kernels` also needs to be installed in this new enviroment, and it actually also installes the jupyter notebook packages. Notice that the package `nb_conda_kernels` also needs to be installed in this new enviroment, and it actually also installes the jupyter notebook packages.
#### 3) Launching Jupyter-Notebooks in the new Enviroment #### 3) Launching Jupyter-Notebooks in the new Enviroment
There are two `keras` interfaces, the stand-alone `keras`, and then a similar interface already build into `tensorflow.keras`. You could use both, but direct access via the stand-alone inteface may be the most obvious. There are two `keras` interfaces, the stand-alone `keras`, and then a similar interface already build into `tensorflow.keras`. You could use both, but direct access via the stand-alone inteface may be the most obvious.
```bash ```bash
(swmal)> jupyter-notebook (swmal)> jupyter-notebook
``` ```
<img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_notebook.png" alt="WARNING: could not get image from server." style="width:800px"> <img src="https://itundervisning.ase.au.dk/SWMAL/L06/Figs/Screenshot_conda_install_keras_tensorflow_run_notebook.png" alt="WARNING: could not get image from server." style="width:800px">
For some reason there now is a missing function in the `pythoncom39.dll` reported via the "Entry Point not Found" dialog,, but just ignore this warning for now (or find a fix and share it). For some reason there now is a missing function in the `pythoncom39.dll` reported via the "Entry Point not Found" dialog,, but just ignore this warning for now (or find a fix and share it).
#### 4) Testing the New Enviroment Setup #### 4) Testing the New Enviroment Setup
Lets see the version installed in the new `swmal` environment via the `Versions()` function found in the `itmallib` Lets see the version installed in the new `swmal` environment via the `Versions()` function found in the `itmallib`
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from libitmal import versions from libitmal import versions
versions.Versions() versions.Versions()
``` ```
%% Output %% Output
Python version: 3.9.7. Python version: 3.9.7.
Scikit-learn version: 1.0.2. Scikit-learn version: 1.0.2.
Keras version: 2.6.0 Keras version: 2.6.0
Tensorflow version: 2.6.0 Tensorflow version: 2.6.0
Tensorflow.keras version: 2.6.0 Tensorflow.keras version: 2.6.0
Opencv2 version: 4.5.5 Opencv2 version: 4.5.5
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The `Versions()` function should print whatever version you installed, or produce a warning, if the package is not installed at all. The `Versions()` function should print whatever version you installed, or produce a warning, if the package is not installed at all.
For my current Windows/Anaconda setup I got the versions, Yours may differ slightly For my current Windows/Anaconda setup I got the versions, Yours may differ slightly
``` ```
Python version: 3.9.7. Python version: 3.9.7.
Scikit-learn version: 1.0.2. Scikit-learn version: 1.0.2.
Keras version: 2.6.0 Keras version: 2.6.0
Tensorflow version: 2.6.0 Tensorflow version: 2.6.0
Tensorflow.keras version: 2.6.0 Tensorflow.keras version: 2.6.0
``` ```
#### 5) Wrapping It All Up in a BAT File #### 5) Wrapping It All Up in a BAT File
To make development easy, a BAT (Windows batch or script file) shold be created. This should ease the launch of Jupyter-Notebooks and the BAT file could be put in a icon placed on the taskbar or similar. To make development easy, a BAT (Windows batch or script file) shold be created. This should ease the launch of Jupyter-Notebooks and the BAT file could be put in a icon placed on the taskbar or similar.
The BAT-file should contain the followin text lines, and you can place this in an icon in the taskbar allowing for easy launch of future notebooks. The BAT-file should contain the followin text lines, and you can place this in an icon in the taskbar allowing for easy launch of future notebooks.
``` ```
@ECHO OFF @ECHO OFF
REM my-jupyter-notebook REM my-jupyter-notebook
REM Version: 0.1 REM Version: 0.1
REM 2022-03-23: CEF, inital version REM 2022-03-23: CEF, inital version
echo MY-JUPYTER-NOTEBOOK launcher.. echo MY-JUPYTER-NOTEBOOK launcher..
REM %windir%\System32\cmd.exe "/K" %HOMEPATH%\Anaconda3\Sc2 ripts\activate.bat %HOMEPATH%\Anaconda3 swmal REM %windir%\System32\cmd.exe "/K" %HOMEPATH%\Anaconda3\Sc2 ripts\activate.bat %HOMEPATH%\Anaconda3 swmal
@CALL "%HOMEPATH%\Anaconda3\condabin\conda.bat" activate swmal %* @CALL "%HOMEPATH%\Anaconda3\condabin\conda.bat" activate swmal %*
REM note book start in this directory, you may change it: REM note book start in this directory, you may change it:
cd \ cd \
jupyter-notebook jupyter-notebook
echo DONE echo DONE
``` ```
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment