Abstract : Recently, we have many research works on the neural networks and their related issues, even for network communications and information exchange. For exchangeability of neural network frameworks, the Neural Network Exchange Format (NNEF) specification is now widely used. Due to very large size of these neural networks, their accelerations are actively explored, and can be achieved through massively parallel processing techniques. In this paper, we present a prototype implementation of C++ code generator with parallel-processing accelerations with CUDA and underlying GPUs, for the NNEF specification files. We show the feasibility and also technical issues including C++ template uses with CUDA kernels.