Generative AI with Python and TensorFlow 2: Create images, text, and music with VAEs, GANs, LSTMs, Transformer models

This is a very nice book. I am looking at it to learn what generative models are, and I am specifically interested in the use case of predicting protein structure. The book hits the spot in introducing me to both of these topics.The first chapter explains the difference between generative models and models that are more common such as linear and logistic regression. In the case of generative models, rather than using a fitted formula to predict an output, we either 1) sample from (x, y) or 2) we start with a target y, and sample the x. There are several types of generative models: Naive Bayes Classifiers, Gaussian Mixture Models, Latent Dirichlet Allocation (LDA) models, Hidden Markov Models, Deep Boltzmann Machines, Variational Autoencoders (VAEs), and Generative Adversarial Networks (GANs). The last three of these types are deep neural network models.The second chapter goes through, in careful detail, setting up a programming environment which provides a foundation for reproducible applications. TensorFlow and Keras are used due to their widespread usage. Interesting background on TensorFlow is provided. The setup for programming includes: open source code editor, VSCode, as IDE, Docker for creating a reproducible Python "execution environment" (Python language version and libraries), Kubernetes for managing applications with multiple containers (each container could be responsible for a different task), Kubeflow built on top of Kubernetes has several components useful in the "end-to-end process of managing ML applications", Katib - a utility built into Kubeflow enables optimizing models for parameter grids, Jupyter Notebooks can be started in a namespace using Kubeflow, Kubeflow can be deployed on AWS, Azure, and GCP, and Terraform is a template language whose instructions can be used to install Kubeflow on any of the cloud platforms.It is a long list, but each piece provides something useful.Chapter 3 covers the "building blocks of deep neural networks" to help understand the theory behind the code. This is a valuable chapter to build an understanding of the progression from perceptrons to multi-layer perceptrons and backpropagation. Backpropagation is described, including with math formulas.Different types of networks are described: convolution, recursive, and networks for sequence data. Also, there is a section on "building a better optimizer". There is a long list of useful references with links, at the end of the chapter, including some papers by Hinton.The next 10 chapters cover use cases, with code blocks: generating digits, painting pictures, generating images, creating deep fakes and transferring styles (like replacing one person's face with another's, converting a horse into a zebra, creating a painting from a photo), generating text, composing music, playing video games, and emerging applications, such as finding new drugs, predicting protein structure, solving PDEs, creating videos from images, and generating food recipes.For the two drug applications described in the last chapter, it is a short section with no code, though, the reader is referred back to earlier chapters where there is code for a model type. The writing is very to the point, with excellent diagrams to provide an understanding of the problem and the approach. The list of references at the end of the last chapter, Emerging Applications, includes a link to the DeepMind blog entry for AlphaFold: Using AI for scientific discovery (i.e. protein folding).The book clearly and carefully covers a lot.

This book covers the most advanced explicit density based generative models (RBM, VAE, etc.) and implicit density based generative models (GAN, etc.). It also gives detailed tutorials on using Keras, the most popular deep learning tool, to build these models. In particular, it also shows how to distribute large-scale deep learning models on multi-core CPUs, GPUs, and cloud servers (e.g., AWS). This is very essential for model training such as GAN that has huge number of parameters as well as a large number of image samples.In the first chapter, the authors introduced the basic concepts of generative models and discriminative models. They clarified the difference by introducing the Bayes probability model behind them. It helped reader sort out the general discriminant models (linear regression, logistic regression, SVM, etc.), and know how generative model works. They also introduced the different types of generative models and their applications. This chapter is very important to the beginners.In the second chapter, the authors gave a detailed tutorial to build the development environment. However, the build environment of this book is based on MacOS (this should have been mentioned), but according to my experience, many people in industry and especially academic use Linux.In the third chapter, the authors briefly reviewed the build blocks of the deep learning models including, perceptrons, hidden layers, loss function, activate function,optmizer, etc. These are all essential components for a deep neural network. Even most of them are build-in components in the recent AI packages, but understanding them can help the developers know deeper about their networks.The rest of the chapters talked about different types of generative models from Restricted Boltzmann Machine (RBM), Variational Autoencoders (VAE), to NLP and reinforce learning generative models. For each model, the author always first gave the mathematical description, and then showed the implementation process. I think understanding the theory behind each model is critical for practical using. There is a little thing can be improved when describing the math explanation. Some symbols in the formula and text are inconsistent, for example the epsilon and e.Among all the generative models in the book, the GAN family attracts me the most. The author introduced the development process from vanilla GAN, conditional GAN ​​to progressive GAN. They also taught two of the most exciting applications of GAN: style transfer and face transfer (deep fake). At the end of each part, they also discussed the challenges and potential research direction of the each generative model. In summary, this is a good manual for the people of deep learning in industry and academia.