Overview of Deep Learning Frameworks
Anaconda is an open source distribution of the Python and R programming languages for large-scale data processing, predictive analytics, and scientific computing, that aims to simplify package management and deployment. Apache MXNet is a flexible open source deep learning framework for training and deploy Deep Learning Neural Network. It allows both symbolic and imperative programming to increase the efficiency. It is highly scalable for running multiple GPU's and multiple machines. It has efficient memory and makes symbolic execution fast. Caffe is one of the deep learning framework. It is written in C++ and highly compatible with Python and Matlab. It is easy to define models and optimisation without hard coding using configuration. It can process over 60M images per day with a single NVIDIA K40 GPU. PyTorch is one of the python package. It provides two highly important features.They are -
- Computation of Tensor with strong GPU acceleration
- Deep neural networks built on a tape-based auto grad system
Guide to Installation of Anaconda The best step is to download the latest version of Anaconda for Python3 from Anaconda Downloads page Move to the /tmp directory on your server to download the Anaconda bash script file cd /tmp Use curl to download Anaconda from the link that you have copied from Anaconda website sudo curl -0 https://repo.continum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh Verify the data integrity of the installer with cryptographic hash verification through the SHA-256 checksum. sha256sum Anaconda3-5.0.1-Linux-x86_64.sh The output is checked against the hashes using link - https://docs.anaconda.com/anaconda/install/hashes/Anaconda3-5.0.1-Linux-x86_64.sh-hash After matching the obtained hash run the script for installation bash Anaconda3-5.0.1-Linux-x86_64.sh Press Enter to continue the installation and type yes to approve the License terms.
- You will be asked to enter the location of installation. Press Enter to choose default location.
- The installation will continue and will take some time to complete it.
- After completion, type yes to use the conda command.
To activate the installation, source the ~/.bashrc file source ~/.bashrc Verify the installation by making use of the conda command conda list After obtaining the above results, the installation of Anaconda is completed
Installation of MXNet (CPU Enabled)
Before installing the MXnet, it is better to build a virtual environment and then install the framework in virtual environment. Create the virtual environment using anaconda conda create --name mxnet_env python=3.6.3 Here, mxnet_env is the name of virtual environment Press yes to complete the creation of virtual environment Check whether the virtual environment is created or not - ls ~/anaconda3/envs conda info --envs Install the MXNet framework using conda conda install --name mxnet_env -c pjmtdw mxnet Check the working of MXNet framework by activating the virtual environment and run python
Installation OF MXNet (GPU Enabled)
Before installing the MXnet, it is better to build a virtual environment and then install the framework in virtual environment. Create the virtual environment using anaconda conda create --name gpu_mxnet_env python=3.6.3 Here, gpu_mxnet_env is the name of virtual environment Press yes to complete the creation of virtual environment. Check whether the virtual environment is created or not - ls ~/anaconda3/envs conda info --envs Install the mxnet-gpu framework using conda - conda install --name gpu_mxnet_env -c pjmtdw mxnet-cudnn Check the working of MXNet framework by activating the virtual environment and run python
Installation of Caffe (CPU Enabled)
Before installing the Caffe, it is better to build a virtual environment and then install the framework in virtual environment. Create the virtual environment using anaconda conda create --name intel_caffe_env python=3.6.3 Here, intel_caffe_env is the name of virtual environment Press yes to complete the creation of virtual environment. Check whether the virtual environment is created or not - ls ~/anaconda3/envs conda info --envs Install the Caffe framework using conda - conda install --name intel_caffe_env -c intel caffe Check the working of Caffe framework by activating the virtual environment and run python.
Installation of Caffe (GPU Enabled)
Before installing the Caffe, it is better to build a virtual environment and then install the framework in virtual environment. Create the virtual environment using anaconda conda create --name gpu_caffe_env python=3.6.3 Here, gpu_caffe_env is the name of virtual environment Press yes to complete the creation of virtual environment. Check whether the virtual environment is created or not - ls ~/anaconda3/envs conda info --envs Install the Caffe framework using conda - conda install --name gpu_caffe_env -c anaconda caffe-gpu But there is a need to install updated protobuf to resolve the issue of compatibility. The command to install protobuf using conda is - conda install protobuf=3.2 Press yes to continue the installation. Check the working of Caffe framework by activating the virtual environment and run python.
Installation of PyTorch (CPU Enabled)
Before installing the PyTorch, it is better to build a virtual environment and then install the framework in virtual environment. Create the virtual environment using anaconda conda create --name pytorch_env python=3.6.3 Here, pytorch_env is the name of virtual environment Press yes to complete the creation of virtual environment Check whether the virtual environment is created or not - ls ~/anaconda3/envs conda info --envs Install the PyTorch using conda command - conda install --name pytorch_env -c soumith pytorch Check the working of Caffe framework by activating the virtual environment and run python.
Installation of PyTorch (GPU Enabled)
Before installing the PyTorch, it is better to build a virtual environment and then install the framework in virtual environment. Create the virtual environment using anaconda conda create --name gpu_pytorch_env python=3.6.3 Here, gpu_pytorch_env is the name of virtual environment Press yes to complete the creation of virtual environment. Check whether the virtual environment is created or not -
ls ~/anaconda3/envs
conda info --envs
Install the PyTorch using conda command - conda install --name gpu_pytorch_env -c anaconda pytorch-gpu Check the working of Caffe framework by activating the virtual environment and run python. Check the usage of GPU using command - nvidia-smi
A Comprehensive Approach
To know more about Development and training of machine learning and deep learning models, we advise taking the subsequent steps -
