1
Vote

EXCEPTION occurred: Can't resize a externally managed matrix

description

While trying to run the MNIST example setup using the second network structure:

cntk configFile=02_Conv.config configName=02_Conv

the end of the log shows the following error:
[CALL STACK]
>Microsoft::MSR::CNTK::ThrowFormatted<std::invalid_argument>
-Microsoft::MSR::CNTK::GPUMatrix<float>::Resize
-Microsoft::MSR::CNTK::Matrix<float>::Resize
-Microsoft::MSR::CNTK::ConvolutionNode<float>::EvaluateThisNodeS
-Microsoft::MSR::CNTK::ConvolutionNode<float>::EvaluateThisNode
-Microsoft::MSR::CNTK::ComputationNetwork::PARTraversalFlowControlNode::EvaluateThisNode
-Microsoft::MSR::CNTK::ComputationNetwork::Evaluate
-Microsoft::MSR::CNTK::SGD<float>::TrainOneEpoch
-Microsoft::MSR::CNTK::SGD<float>::TrainOrAdaptModel
-Microsoft::MSR::CNTK::SGD<float>::Train
-DoTrain<float>
-DoCommand<float>
-wmain1
-wmain
-__tmainCRTStartup
-BaseThreadInitThunk
-RtlUserThreadStart
EXCEPTION occurred: Can't resize a externally managed matrix

Notes:
The first topology run successfully and took < 1 min
Final Results: Minibatch[1-1]: Samples Seen = 10000 Err: ErrorPrediction/Sample = 0.0239 CE: CrossEntropyWithSoftmax/Sample = 0.076812531 Perplexity = 1.0798396
COMPLETED

MY laptop GPU device query is:

devicequery Starting...

CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "GeForce GTX 960M"
CUDA Driver Version / Runtime Version 7.5 / 7.5
CUDA Capability Major/Minor version number: 5.0
Total amount of global memory: 4096 MBytes (4294967296 bytes)
Total amount of CONSTANT memory: 0 MBytes (65536 bytes)
( 5) Multiprocessors, (128) CUDA Cores/MP: 640 CUDA Cores
GPU Max Clock rate: 1176 MHz (1.18 GHz)
Memory Clock rate: 2505 Mhz
Memory Bus Width: 128-bit
L2 Cache Size: 2097152 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 1 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
CUDA Device Driver Mode (TCC or WDDM): WDDM (Windows Display Driver Model)
Device supports Unified Addressing (UVA): Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0
Compute Mode:
 < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 7.5, CUDA Runtime Version = 7.5, NumDevs = 1, Device0 = GeForce GTX 960M
Result = PASS

file attachments

comments

dongyu wrote Dec 17, 2015 at 5:14 PM

this has been fixed long time ago. Please get the most recent master branch.

AlexeyK wrote Dec 17, 2015 at 5:29 PM

Hello,
Are you using the latest version from master branch? May I ask you to pull the latest code from master and try again? As the folder structure and samples have been changed a bit, please use the following commands to run the convo sample:
  1. Navigate to: cntk\Examples\Image\MNIST folder
  2. Run the following: <path_to_CNTK.exe>\CNTK.exe configFile=Config\02_Convolution.config RootDir=.
    Note that the data must be in cntk\Examples\Image\MNIST\Data folder.
    Let me know if that works for you.
    Thanks.
    Alexey

mfouad_thabet wrote Dec 17, 2015 at 11:00 PM

It worked after a pull and a re-build, thanks for your support.

Note:
As I have Cuda 7.5 I changed CNTKMathCUDA project from 7.0 to 7.5 in two locations.