CUDA by Example: An Introduction to General-Purpose GPU Programming

A great deal has been written about the various visions regarding parallel processing. In this dazzle it is often lost to the uninitiated, that parallel computing is not some distant promise, but a revolution that is well in the progress of happening. NVIDIA's CUDA platform is one, and perhaps the easiest and most affordable, of the ways to catch up and to join the front wave of this revolution. CUDA has several components from a hardware architecture for graphics processors to a high level programming interface, implemented as a few extensions to the C language, called CUDA C. One of the main features of the CUDA project is that it makes a systematic effort to separate the programming layer from the chip architecture. "CUDA by Example" by Sanders and Kandrot is the first book to make full use of this abstraction and to concentrate solely on the software side. As a result, it is the first text eminently suitable as a basis for an introductory course on CUDA C for students of software engineering or scientific computing. Working through the book the student, or reader, get acquinted step-by-step with most important distinguishing features of parallel programming, like need for memory sharing, event sychronization, atomic operations and isolated processing streams. All this is taught through sofware examples without the need to dwelve into the details of chip architecture. All that is required is some experience in basic C programming and an optional $200-300 "gamer" graphics board to demonstrate the real-life performance gains (no graphics programming experience is required). The authors put a lot of thought into designing each chapter and the corresponding illustrating exaples in such a way that the reader can concentrate on just one new feature at any one time. Every chapter has typically two examples: A very basic one focusing on the newly introduced feature and a more exciting one illustrating some of the power of the new feature. E.g. in the chapter on synchronization, the basic example is provided by the scalar product of two vectors and the need to complete all termwise multiplications before adding up their products. The more dazzling example is a little graphics program, where the lack of proper synchronization manifests itself in the same scrambled screen patterns as what old CRT TV's displayed, when they had synchronization problems. My personal favorite is the very simple heat conduction example, that visually demonstrates the potential of the use of texture memory for spatially related data. It is a gem of simplicity and clarity. (It also captured my fascination because previously I was not able to figure out how to benefit from texture memory in general-purpose applications.) The conscientious choice by the authors to write a software book for software developers implied that hardware dependent tricks had to be omitted. Specifically, code optimizations, that depend on specifics of various generations of chip designs, are not covered. This, as every compromise, has two sides. The positive one is that the knowledge acquired by working through the book will not become outdated, when a new generation of graphics boards hits the market. It also leaves the door open for other potential authors to write a second book on CUDA optimizations. In summary, "CUDA by Example" is an excellent and very welcome introductory text to parallel programming for non-ECE majors. It is very systematic, well tought-out and gradual. It goes beyond demonstrating the ease-of-use and the power of CUDA C; it also introduces the reader to the features and benefits of parallel computing in general. Perhaps a more fitting title could have been "An Introduction to Parallel Programming through CUDA-C Examples". I plan to use this book as the text for the first half of a graduate course on parallel computing for data analysis and quantitative finance.

I read this book a few months ago along with these 2 other books - "Programming Massively Parallel Processors" and "CUDA Application Design and Development" so maybe my review is a bit biased. Pro: (i) Overall, it is a good introductory book. Easy to read. (ii) Good examples, guide the readers step-by-step of what is going on. (iii) Good coverage of some important topics. Con: (i) I agree with another reader here that since this is an introductory text, some instructions of naming the file as *.cu and compile it with nvcc, etc. should be given. Sometimes setting up a development environment could be difficult and/or time-consuming, I for one don't really like to do it. (ii) Snippets of code were introduced first and then the whole program is then reprinted again. IMHO, a bit of re-organization can avoid such problem plus it's a waste of paper. (iii) The pace is a bit slow for me and I think the book could have been shorter. All in all, I would recommend this book to CUDA newbie but keep in mind that some of the topics are already obsolete.

This is an excellent introduction to CUDA. The prose and content are excellent: I read it cover-to-cover in a single sitting and enjoyed every page. The authors clearly explain the basic CUDA paradigm starting with very simple code and working up to progressively more complex examples. The authors spend a considerable amount of time discussing different memory types and memory access styles, motivating when each style is appropriate. The code snippets are clean, clear and concise, providing a minimal yet complete introduction to each new language feature. Highly recommended! The book does not provide an HTML pointer to the source code used in the book. Edward Kandrot writes: "The Kindle version shipped a week too soon, it was supposed to ship next week when the physical book ships. Because of this, the website at NVIDIA wasn't done yet. Jason just spent the day making the website happen! [...] is where the source code is currently located. I hope this helps. I wrote the examples to be specific for what is being covered, putting extras in the header files so as not to distract from the topic at hand. Only really works if the reader has the header files as well..."

One of the best resources to start with CUDA. Positive aspects from a starting-out perspective: -- a small, very readable book focusing on the important parts to start with; comprehensiveness has it's merits but not when we are just starting out -- brevity does not mean superficialness; the book is very methodical, has examples when you start doing some technique, but alas, it does not improve performance; then it explains more background and help to fix the issue; sticks much better than feeding the conclusion up front -- complete working samples -- enjoyable style; one of the very very small number of books where I am able to appreciate all of the humor (and I'm fussy about this topic; have other reviews triggered by the contrived tiresome attempts to be humorous, which is the case in most tech books I've seen) Will need to read other book(s) after this one but a very good one to start with. Anyone criticizing because of the size is either missing the point or did not get the right book for his own need (through no fault of his own).

This is a very well done introductory textbook for CUDA programming. The examples are very well explained, and are general enough that you really learn the broader concepts, not just how to do the what the example does. It is written in the standard C CUDA, but does mention that there are other language implementations of CUDA. It does *not* explain how to do anything in them, or the names of any of the libraries are anything, so if you are looking to do pyCuda or MatCuda, this is not the book for you. It does a good enough job of explaining that it is easy to understand even if you have no background in parallel computing or in C. Each example is built modularly, so you can see how each section works and why they do it that way. This really is a fantastic book to begin your journey into GPU programming with NVIDIA.

I don't regret buying this book because the contents are a very good tutorial on CUDA. That said, this is a unnecessarily hard to read book, and here's why: Pros: - Nice practical examples, most of them using visual representations of the data being processed; - Topics are introduced in an order that makes it easier for the reader to go from the basics to more advanced topics without feeling overwhelmed. Cons: - As other readers already mentioned, I often felt that the author was too verbose and trying to fill pages with [non-useful] repetition and/or random jokes; - Very often the code presented in the book is missing important parts, cluttered with errors that make it very hard to understand what's going and a very annoying lack of coding style (e.g. in the same example the author uses two or three different naming styles for variables). The source codes that can be downloaded from the website are usually correct (and different from the book) but that still makes the book nearly useless since the reader needs to constantly go back and forth between book and code in order to fully understand the concepts. This could very easily have been a 5 star book if it weren't for the above mentioned issues. Hopefully this will be fixed in future editions.

CUDA programming is often recommended as the best place to start out when learning about programming GPU's. The learning curve concerning the framework is less steep than say in OpenCL, and then you can learn about OpenCL quite easily because the concepts transfer quite easily. This book is aimed at a beginner in CUDA and the level of the explanations clearly shows that the authors are aware that it is so new to the reader that he/she will need a lot of explanations. However there are times in which I feel the style of this book is insufferable. For example there is a joke of a program which prints Hello World. This can be annoying to someone keen to see the first real example of parallel programming. The authors dont give you much help with regards to installation. Yet, I cant think of any book that can really replace this book. If you are starting out, you pretty much have to have it. It covers certain subjects that Wen-mei Hwu does not cover in his book. In fact in the latter's Coursera course he suggests that you read Sander's book for certain topics.

I loved reading this book. It helped me in getting my concepts sorted.

Une bonne publication sur un sujet qui n'est pas facile par des experts venus de chez Nvidia On peut juste regretter que les sources comporte des spécificités liées au livre. On aurait préféré une approche plus générale dans un premier temps

Very well written keeping in view the requirements of beginners in CUDA programming. However, new update or revision of this book as of 2017 is badly needed.

Lo pedí un sábado por la noche y me llego al día siguiente en la mañana. En cuanto al libro, es muy bueno con ejemplos explicados. Te recomiendo que veas un par de vídeos en otros sitios antes de leer el libro para que tengas una idea de lo que es CUDA y depués el libro te explica a fondo.

Um eines vorwegzunehmen: Ich würde mich im Bereich der Programmierung von C/C++ und mit Grafik-APIs wie OpenGL und DirectX als fortgeschrittenen Entwickler einstufen. Über den Umweg des Raytracing Frameworks OptiX von Nvidia bekam ich dann ersten Kontakt mit CUDA und wollte eigentlich erstmal nur wissen, wie das so funktioniert und was das genau ist. Daher griff ich zu diesem Buch, wohlwissen, dass es als absolutes Grundlagenwerk ohne "Tiefenwirkung" gebrandmarkt ist. Was soll ich sagen, ich wurde absout nicht enttäuscht. Ich habe das Buch mit in den Uraub genommen und es quasi in den Kaffeepausen bzw. bei nebenbei tobenden Kindern gelesen. Das beschreibt in etwa den Anspruch bzw. den Aufmerksamkeitsgrad den das Buch an einen erfahrenen Programmierer stellt. Und das ist nun absolut nicht negativ gemeint. Im Gegenteil. Ich fand es sehr erfrischend, dass man das Buch einfach mal so nebenbei konsumieren kann. Der didaktische Aufbau des Buches ist sehr gut. Man kommt ohne viel Umschweife direkt in die Materie, die einem aber Stück für Stück nähergebracht wird. Das Buch startet mit sehr einfachen Beispielen die dann erstmal sehr einfach in CUDA umgesetzt werden. Danach wird immer etwas weiter optimiert, um die verschiedenen Möglichkeiten von Parallelität CUDA aufzuzeigen oder verschiedene Speichertypen zu verwenden. Man sollte nicht erwarten, dass man aus dem Buch allzu tiefe technische Details ziehen kann. Auch wird man kaum den optimalsten Code für CUDA schreiben. Aber das ist auch gar nicht der Anspruch des Buches. Das Buch erklärt an sehr schön kurz abgeschlossenen Beispielen wie CUDA funktioniert, und dass CUDA nicht gleich CUDA ist da man zum einen verschiedene Speichertypen und zum anderen verschiedene Parallelisierungsebenen (Cores, Theads) zur Auswahl hat. All dies wird einem sehr schön nahegebracht ohne jedoch tiefer in die Optimierung des Codes einzusteigen. Einzig und allein das wiederholte Abdrucken kompletter Programmteile würde ich zur Diskussion stellen wollen. Das hat zwar den Vorteil, dass man als Anfänger nicht erst schauen muss, wo man diverse Code-Schnippsel einfügen muss. Aber mal ehrlich: Greift jemand, dem so etwas Probleme bereiten würde weil er kaum Programmiererfahrung hat, zu einem Buch über CUDA? Egal, das trübt das positive Gesamtbild nicht nachhaltig. Von daher kann ich mich den vorherigen Rezensionen nur anschließen. Ein tolles Buch. Für einen "echten" Entwickler ist es super nebenbei zu lesen, aber zu viel Details darf man hier nicht erwarten. Für diesen Interessentenkreis ist es eher ein Einstieg den man locker in ein paar Stunden durchgearbeitet hat, aber man sollte danach noch mehr Material in die Hand nehmen um effizienten CUDA Code schreiben zu können. Für den Laien (denke ich) hingegen ist das Buch ein guter Einstieg in die Programmierung mit CUDA, so dass man gleich loslegen kann. Ich bekam also genau das, was ich erwartet habe. Allerdings in deutlich besserer Qualität als erhofft. Von daher fünf Daumen hoch :)

Un ottimo libro per iniziare la tecnologia CUDA: semplice e con esercizi basilari, per imparare la basi del linguaggio di programmazione.