Performance evaluation of deep learning frameworks on computer vision problems

Abstract

Deep Learning (DL) applications have skyrocketed in recent years and are being applied in various domains. There has been a tremendous surge in the development of DL frameworks to make implementation easier. In this paper, we aim to make a comparative study of GPU-accelerated deep learning software frameworks such as Torch and TenserFlow (with Keras API). We attempt to benchmark the performance of these frameworks by implementing three different neural networks, each designed for a popular Computer Vision problem (MNIST, CIFAR10, Fashion MNIST). We performed this experiment on both CPU and GPU(Nvidia GeForce GTX 960M) settings. The performance metrics used here include evaluation time, training time, and accuracy. This paper aims to act as a guide to selecting the most suitable framework for a particular problem. The special interest of the paper is to evaluate the performance lost due to the utility of an API like Keras and a comparative study of the performance over a user-defined neural network and a standard network. Our interest also lies in their performance when subjected to networks of different sizes. �2019 IEEE.