Authors: Francesco Pugliese & Matteo Testi
In this post, we are going to tackle the tough issue of the installation, on Windows, of the popular framework for Deep Learning “Keras” and all the backend stack “Tensorflow / Theano“.
Installation starts from the need to download the Python 3 package. Let us choose Miniconda and download it at the following link: https://conda.io/miniconda.html that will show the following screen:
Select Python 3.6 and the operating system version: Windows a 64-bit or 32-bit. Click on the downloaded package and install it with the default settings. In the end of the installation accept the system reboot.
Once the PC is rebooted, from the Windows’ search box, digit cmd.exe and run the prompt. Then run the script c:\Users\-user-\Miniconda3\Scripts\activate.bat which will launch the Anaconda’s prompt (change -user- with the current account name).
Therefore, digit: conda install numpy scipy mkl-service m2w64-toolchain in order to install:
- “numpy” Python library which is very useful for the matrices and arrays management.
- “scipy” that is a scientific computing library for python.
- “mkl-service” optimization library with vectorial maths routines to speed-up mathematical functions and applications.
- “libpython” library for Python 3 for Machine Learning and effective code development.
- “m2w64-toolchain” providing a GCC compatible version, so it is strongly recommended.
Other optional libraries are:
- “nose” library for programs testing in Python.
- “nose-parameterized” for the parametric testing.
- “sphinx” library for building program’s stylish documentation in diverse formats (HTML, PDF, ePyub, etc.).
- “pydot-ng” interface for the graphic rendering language Graphviz’s Dot.
Once the environment settings are finished, at this stage you are able to install Cuda drivers from the following link:
This will open the following view with different operating systen options and things:
Download the local version (recommended) of the installation file and proceed with the Cuda drivers installation. Cudas are parallel programming libraries of the Nvidia GPU (Graphic Processing Unit) which is part of the video card. It might be necessary to install the card drivers as well in case it is not updated or it is not working properly.
When the Cuda driver installation process finally ends (and all the possible graphic card drivers) run the Theano installation plus the additional supporting library “libgpuarray” which is required to handle tensors on GPU, with the command:
conda install theano pygpu
Theano NOTE 1: In order to install Theano we suggest to always use at least 1 point version less of Cuda with regard to the current version. This is due to uneffective maintenance of Theano which is not rapidly up-to-dated and this leads to compilation errors after the installation with the current version of Cuda. For instance, at this time, the most stable version of Theano is 0.9.0, for which we suggest to use Cuda 8.0 instead of Cuda 9.0. There might exist some tricks online to make things working between Cuda 9 and Theano 0.9 but they turn out a little bit tricky and take time, and it will not be worh the risk eventually. Our hint is to handle a steady Cuda-Theano configuration such as the ones recommended.
Now you need to install Visual Studio providing to Theano the C++ compiler for Windows (indeed the previously installed GCC refers to the only C compiler). In order to do this, download Visual Studio Community from the link: https://www.visualstudio.com/it/downloads/ and follow all the required steps, trying to install only the basic components for C++.
Theano NOTE 2: Seemingly, after the next release, Theano will be dismisses, Bengio himself explains it in this: link There are multiple reasons for this choice, we believe essentially due to the latest massive competition from the other Deep Learning frameworks (mxnet, tensorflow, deeplearning4j, gluon, to name a few) which are more mantained. As we just showed, Theano constantly suffers updating problems from the MILA team. However we believe Theano is still a milestone for Deep Learning, the first that introduced the automatic differentiation, clear and effective parallelization of matrix operations on GPU that enabled the spread of GPU deep neural networks. Hence we consider the need of giving the right prestige to this brilliant framework, and after all it still confers its upside in terms of versatility and speed when used as backend of Keras.
Visual Studio NOTE: Also Visual Studio is affected by compatibility problems with Theano. Basically Visual Studio 2017 will return an exception during the import of Theano both with Cuda 9 and Cuda 8. Therefore we suggest to install a stable preceding version like Visual Studio 2013.
Once you have installed Visual Studio you need to fill the .theanorc, which is the Theano configuration file, you can find it within Miniconda3 at the path: c:\Users\-user-\.theanorc
Fill .theanorc as follows, given that you have decided to install Cuda 8 e Visual Studio 2013 :
device = gpu
floatX = float32
root = C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0
compiler_bindir = C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin
cnmem = 0
Let us pay attention to these parameters: “device” of [global] section which define whether you want to use CPU or GPU, “root” of [cuda] section needed to set Cuda libraries path, whereas “compiler_bindir” of [nvcc] needed to define the C++ compiler path of Visual Studio which is critical to compile Theano programs. CNMeM refers to a library (built-in in Theano) allowing you to set (by means of a value between 0 and 1) the way Deep Learning framework is capable to handle the GPU shared memory, and the way to speed-up neural networks computation on Theano. For example, video cards shared with the monitor we suggest a parameter around 0.8 whereas stand-alone graphic cards work with a cnmem equal to 1.
Another very important parameter for boosting the computation, especially for the convolution, is the setting “enabled” of [dnn] section which allows to enable or disable Nvidia CuDNN libraries. This is basically a library supplying optimized primitives for deep neural networks leading to the speed-up of th training stage, testing and to energy saving.
In order to install CuDNN you need to go to this link: https://developer.nvidia.com/cudnn and click on the download button. Proceed with the download (Nvidia membership registration might be necessary), the following screed should pop up:
cuDNN NOTE: also in this case as previously stated, we advise not to download the last version of cuDNN but one of the two preceding version as it may not be “seen” neither by Cuda 8 nor Theano 0.9, in this case we recommend cuDNN 6.0 version. Anyway youa warning may arise with Theano 0.9 indicating that cuDNN version is too much new and could generate possible problems. We noticed incompatibility problems between cuDNN and TensorFlow as well.
Extracting the downloaded file, you will obtain 3 folders: bin, lib and include. All you need to do is copying the content of these folders into the folders of the same name withing the Cuda directory, namely inside: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0
Hence copy cudnn64_6.dll into bin of Cuda path, copy cudnn.h into include and finally copy cudnn.lib into lib.
Once you have installed cuDNN, go with the installation of Keras by means of pip:
pip install keras
The instruction will install all the dependencies and also the last version (currently Keras 2.0.9). To set Theano as Keras backend, go into the folder: c:\users\-user-\.keras and edit the file keras.json as follows, namely setting “theano” as “backend” item.
To check everything is going the right way, launch the anaconda prompt and then launch python. From the python prompt digit: import keras. If everything went well the following screen will appear:
Notice the warning we mentioned earlier speaking of cuDNN and showed by Theano it self: if you meet all the listed problems downgrade cuDNN to the 5.1 version as adived by the team itself. When a stable version of Theano 0.10 will come out, it will probably solve all these compatibility problems.
Anyway we know that the the environment configured with Keras and Theano according these modalities perfectly works on a diverse models that we previously trained and tested. We decided to use Theano as backed because it turns out faster than TensorFlow with some Computer Vision trainings very often.
In any case, if you want to use TensorFlow as backend you need to install it. To install tensorflow for GPU you need to do the following command:
pip install –upgrade tensorflow-gpu
This instruction will install the last version (1.4.0) of Tensorflow-gpu. To try it with Keras change “theano” with the string “tensorflow” withing the file keras.json, reboot the anaconda prompt and re-digit import keras.
TensorFlow NOTE: it is not supported on 32 bit platforms, installation program will download only the wheel related to the 64 bit framework. Furthermore, in order to download the cpu version you just need to specify the following command (without gpu): pip install –upgrade tensorflow.
If everything went fine, you will see TensorFlow appearing as keras backend this time:
Other useful packages to work with Keras are:
- scikit-image: A library very useful for image processing in python, that allow us to save matrices and tensors onto jpeg pictures or many other supported formats. Installable with: conda install scikit-image.
- gensim: The word embeddings library implementing word2vec algorithm, among other things. Installable with: conda install gensim.
- h5py: The library interfacing with the format HDF5 from Pythonic, this is necessary to save models trained on disk in Keras. Installable with pip install h5py.
At this point, the environment Keras/Tf-Th on Windows is ready to go, to test you code and your models natively harnessing the GPU.
See you at the next tutorial.
Greetings from Deep Learning Italia.
For any information or clarification here you have our emails: