Multi-GPU Parallel DNN training Using CNTK
CNTK is the deep learning toolkit built by Microsoft. Like many deep learning toolkit, CNTK can support the effective training of various deep learning model. Besides of these basic functions on model training, we have to highlight the scale out capability of CNTK. CNTK has many advanced techniques like 1-Bit SGD, Block Momentum, and parameter server based asynchronous training. All these techniques helps CNTK scale out to multiple GPU and multiple node with almost linear speed up in many deep learning tasks. I will talk about these techniques in detail in this talk, and it will be particularly interesting to users who wants to run large scale deep learning.