For those who wish more material on pre-CUDA 4.0, following are my CUDA tutorials on the Doctor Dobbs website: 

·         CUDA, Supercomputing for the Masses: Part 1
CUDA lets you work with familiar programming concepts while developing software that can run on a GPU
·         CUDA, Supercomputing for the Masses: Part 2
A first kernel  
·         CUDA, Supercomputing for the Masses: Part 3
Error handling and global memory performance limitations
·         CUDA, Supercomputing for the Masses: Part 4
Understanding and using shared memory (1)
·         CUDA, Supercomputing for the Masses: Part 5
Understanding and using shared memory (2)
·         CUDA, Supercomputing for the Masses: Part 6
Global memory and the CUDA profiler
·         CUDA, Supercomputing for the Masses: Part 7
Double the fun with next-generation CUDA hardware
·         CUDA, Supercomputing for the Masses: Part 8
Using libraries with CUDA
·         CUDA, Supercomputing for the Masses: Part 9
Extending High-level Languages with CUDA
·         CUDA, Supercomputing for the Masses: Part 10
CUDPP, a powerful data-parallel CUDA library  
·         CUDA, Supercomputing for the Masses: Part 11
Revisiting CUDA memory spaces
·         CUDA, Supercomputing for the Masses: Part 12
CUDA 2.2 Changes the Data Movement Paradigm
·         CUDA, Supercomputing for the Masses: Part 13
Using texture memory in CUDA
·         CUDA, Supercomputing for the Masses: Part 14
Debugging CUDA and using CUDA-GDB
·         CUDA, Supercomputing for the Masses: Part 15
Using Pixel Buffer Objects with CUDA and OpenGL
·         CUDA, Supercomputing for the Masses: Part 16
CUDA 3.0 provides expanded capabilities (1)
·         CUDA, Supercomputing for the Masses: Part 17
CUDA 3.0 provides expanded capabilities and makes development easier (2)
·         CUDA, Supercomputing for the Masses: Part 18
Using Vertex Buffer Objects with CUDA and OpenGL
·         CUDA, Supercomputing for the Masses: Part 19
Parallel Nsight Part 1: Configuring and Debugging Applications
·         CUDA, Supercomputing for the Masses: Part 20
Parallel Nsight Part 2: Using the Parallel Nsight Analysis capabilities
·         CUDA, Supercomputing for the Masses: Part 21
The Fermi architecture and CUDA
·         CUDA, Supercomputing for the Masses: Part 22
CUDA for x86 architecture processors