CUDA and C++, in Alex Krizhevsky's cuda-convnet code, with Python scripts driving training.
Proposed by @GokuMohandas
Card 1 of 276. Answer: CUDA and C++, in Alex Krizhevsky's cuda-convnet code, with Python scripts driving training.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
AlexNet was written in CUDA and C++. Alex Krizhevsky built it on cuda-convnet, his own library, which its project page calls a "high-performance C++/CUDA implementation of convolutional neural networks". CUDA is NVIDIA's C/C++-based platform for programming its GPUs. The compiled C++ and CUDA core was loaded as a module by Python scripts that configured and ran training.
Hand-written GPU code was the point. The 2012 paper says the authors "wrote a highly-optimized GPU implementation of 2D convolution and all the other operations inherent in training convolutional neural networks". That made a network with 60 million parameters, five convolutional and three fully connected layers trainable in five to six days on two NVIDIA GTX 580 GPUs with 3 GB each; the network was split across the cards because it did not fit on one.
The earlier cuda-convnet, tuned on CIFAR-10, was public, but the exact 2012 code went to Google when it bought the team's company, DNNresearch. The Computer History Museum released it with Google in March 2025.
