New dependency in CNTK

Oct 14, 2015 at 10:22 PM
Edited Oct 14, 2015 at 10:23 PM
Hi CNTK contributors,
We are adding a dependency to CNTK which will be mandatory in order to build CUDA-enabled version. Below is the short version of the changes followed by more detailed explanation.

What: NVIDIA CUB library will be required to build CNTK CUDA code.
Why: CNTK currently contains a lot of duplicated and suboptimal/inefficient GPU kernel implementations that perform reductions and scans. CUB provides extremely well optimized routines that will replace all of that code as a part of ongoing refactoring efforts.
How: CUB is a simple, open-source (New BSD license) library distributed as .zip file and does not require any special build/pre-build steps. Download it from: https://github.com/NVlabs/cub/archive/1.4.1.zip
  • Windows:
    o Unzip the package to any directory.
    o Set environment variable CUB_PATH to CUB folder, e.g.: CUB_PATH=c:\src\cub-1.4.1
  • Linux:
    o Just unzip the package to: /usr/local directory.
    o Alternatively, unzip the package to any directory you like and run: ./configure --with-cub=<full path to CUB directory>
Documentation page on CodePlex has been updated with these steps. Also, if you try building the solution/project without CUB installed, you should receive a meaningful error message which will tell you what’s missing and how to fix it.

You might have questions on why we need this dependency, here is some more information.
NVIDIA CUB (https://nvlabs.github.io/cub/ ) is an open source, production quality library fully supported and developed by NVIDIA.
It provides a set of highly optimized routines that perform reductions, scans, sorts etc. These routines are optimized by NVIDIA engineers for every GPU architecture (Tesla, Fermi, Kepler, Maxwell) and new versions will be available immediately after new architecture is released.
The main reason to use the library is to remove old (and duplicated) reduction/scan code from CNTK. This code was written using now-outdated (more than 5 years old) reduction algorithm which worked fine on old Fermi architecture but can be much more efficiently implemented on newer architectures like Kepler and Maxwell.

CUB is currently not a part of CUDA Toolkit as it’s being actively developed (similar to another popular NVIDIA library, cuDNN). It might be included in CUDA Toolkit sometime in the future.

Let me know if you have any questions or problems building the code.
Thanks.

Alexey
Nov 18, 2015 at 2:21 AM
Hi Alexey,

After the new CUB library is added in CNTK, every time I run CNN on GPU, there is a Error says : Can't resize a externally managed matrix. Would you help me solve this problem?
This problem only happens when run CNN on GPU. Use CPU or run other DNN won't give this problem.

Shawn
Nov 18, 2015 at 6:36 AM
Edited Nov 18, 2015 at 6:36 AM
Hi Shawn,
The CUB dependency was added over a month ago and I don't remember having the problem you described.
Are you using latest version from master branch?
I might have an idea what went wrong (and it's not related to CUB). While I'm doing some verification, you can try using any version prior to Nov 4. For example, try resetting to commit 3ca402f94f843cd9b920ffd908769e23c902f296 and see if it helps.
I'll let you know once I find and fix the problem.
Thanks.

Alexey
Nov 18, 2015 at 7:12 PM
Hi Alexey,

Thank you so much for answering my question!
Yes, I was using Nov 17's master branch version.
I have tried your solution and changed back to Nov 4's version and it works! Problem doesn't show up anymore.
Please let me know if you fix the problem.

Shawn
Nov 23, 2015 at 5:05 PM
Hi Shawn,
Sorry for delay with reply. I checked in the fix last week and also added a test to our internal build system to avoid such errors in the future.
Let me know if the fix works for you.
Thanks.

Alexey
Marked as answer by MarkHillebrand on 12/9/2015 at 3:57 PM
Nov 24, 2015 at 8:17 PM
Hi Alexey,

The fix works! Error doesn't show up.
Thank you very much!

Shawn