Ankivil

Machine Learning Experiments

Software

Choosing a Deep Learning Software

Introduction

To be able to create deep learning networks and other related models the first thing to it to choose a framework, library or a program to design them. This choice is not trivial as there are a lot of softwares available and they each have their strengths and weaknesses.

You could also design and code your own deep learning library, but it shouldn’t be easy as it involves tricky maths, clever optimization and low level code for GPUs. So only if you are an expert (if so, you should not read this article), you should stick with the available softwares.

Criteria

In order to  determine if each feature is considered as a strength or a weakness, criteria must be defined. These should differ depending on needs, especially the first criterion:

  • Works on Windows 10 to avoid dual boot or virtual machine
  • Can be used by a neophyte
  • Uses GPU for faster training
  • Can model different types of networks to do experiments
  • Uses an “easy” programming language to create prototypes faster
  • Has pretrained models to avoid wasting time training on classic datasets
  • Free

Softwares

Caffe (site)

  • (+) Very good for convnets
  • (-) Not good for other types of networks like recurrent networks for example
  • (-) Not very easy to use

CNTK (site)

  • (-) Only usable through command line. Support for C++, Python and .NET should be coming

Deeplearning4j (site)

  • (+) Very good overall, nice architecture, can model lots of network types, fast
  • (+/-) Seems to be oriented toward business
  • (+/-) Use of Java great idea but it makes prototyping slower

Neural designer (site)

  • (-) Not free
  • (-) No API only GUI
  • (-) Can only model few network types

OpenNN (site)

  • (-) C++ API
  • (-) Can only model few network types

Nervana neon (site)

  • (+) Very fast, maybe faster than cuDNN
  • (-) Does not work on Windows

Tensorflow (site)

  • (+) Nice architecture
  • (-) Not mature: buggy, slow, few pretrained models and cannot model some network types
  • (-) Does not support GPU on Windows

Theano & wrappers (site)

  • (+) Very good overall, nice architecture, can model lots of network types
  • (-) Slow on start
  • (-) Not easy to use
  • (+) Wrappers (Pylearn2, Blocks, Lasagne, Keras) make it easy to use

Torch (site)

  • (+) Very good overall, nice architecture, can model lots of network types, fast
  • (+/-) Uses Lua which is a nice language but not widly used
  • (-) Does not work on Windows

Conclusion

According to the criteria there are two softwares that stand out from the others: Deeplearning4j and Theano combined with a wrapper. Even though I love Java, I found it ill-suited for rapidly testing new ideas, Python is way better for that. For now Theano with Keras seems the best solution. Keras is a good choice because it is also compatible with Tensorflow which may become very good as it matures.

If you are using Linux, Torch seems to be the way to go but you should consider Deeplearning4j, Theano and maybe Tensorflow as well.

References

2 Comments

  1. Nice comparison! However I too have found Tensorflow still not that mature to be used.Also you might want to add mxnet ,h2o and keras in your list as well.

    • Fabien Tencé

      Hi Tanay,

      Keras is only listed as a Theano wrapper. I think it will change as it is becoming more popular and mature.
      Thanks for your suggestion, I’ll review mxnet and h2o in the future.

Leave a Reply

Theme by Anders Norén