Before we dive deeper into GenAI concepts, let’s discuss the differences between Artificial Intelligence (AI), Machine Learning (ML), Deep Learning (DL), and GenAI, as these terms are often used interchangeably.

Figure 1.1 shows the relationships between these concepts.

Figure 1.1 – Relationships between AI, ML, DL, and GenAI

Let’s learn more about these relationships:

• AI: AI refers to a system or algorithm that is capable of performing tasks that would otherwise typically require human intelligence. These tasks include reasoning, learning, problem-solving, perception, and language understanding. AI is a broad category and can include rule-based systems, expert systems, neural networks, and GenAI algorithms. The evolution of AI algorithms has provided machines with human-like senses and capabilities, such as vision to analyze the world around them, listening and speaking to understand...

Since this book is about GenAI, what could be a better way to start it than asking ChatGPT to summarize the evolution of AI and ML over the last decade?

Prompt: "Why did the chicken cross the road?" Describe how that question's answer evolved using AI/ML over the last decade.

ChatGPT Response (ChatGPT-4o, June 16th, 2024):

The evolution of AI/ML responses to the question "Why did the chicken cross the road?" over the past decade reflects significant advancements in language processing and understanding. Here's a detailed description of how these responses have evolved:
Early 2010s: Rule-Based Systems
Response Type: Traditional, rule-based
Example Answer: "To get to the other side."
Description: In the early 2010s, AI systems were primarily rule-based. These systems relied on pre-defined patterns and rules coded by developers. When asked "Why did the chicken cross the road?" the AI would output...

A transformer model uses an encoder-decoder architecture, where the encoder maps the input sequences/tokens through a self-attention mechanism. This mapped data is used by the decoder to generate the output sequence. The mapping of input tokens retains not only their intrinsic values but also their context and weight in the original sequence. Let’s go through some key aspects of the encoder architecture in the following figure:

Figure 1.3 – Transformer architecture from the Attention Is All You Need paper

Here is a breakdown of the concepts highlighted in Figure 1.3:

• Input embeddings: Marked as 1 in the figure, this is a key part of the transformer model, which converts input sequences/tokens into high-dimensional vector embeddings. In real-world applications, output embeddings from a trained model may be stored in high-dimensional vector databases, such as Elasticsearch, Milvus, or PineCone. Vector databases...

Enterprise spending on GenAI projects has been growing exponentially since 2023, with c-suite executives planning to spend even more on GenAI projects (https://www.gartner.com/en/newsroom/press-releases/2023-10-11-gartner-says-more-than-80-percent-of-enterprises-will-have-used-generative-ai-apis-or-deployed-generative-ai-enabled-applications-by-2026). However, there is growing concern about how to quantify the Return on Investment (ROI) of these efforts, such as revenue impact, efficiency, and accuracy gains. Moving forward, ROI will become a critical part of the conversation, as enterprises look for new GenAI projects. So, before starting a new GenAI project, it is recommended to think about the entire project life cycle. In this section, we will be covering the project life cycle.

Let’s first look at the following figure, which outlines the end-to-end GenAI project life cycle, starting from defining business objectives, or KPIs. This is followed...

As we discuss GenAI application development and deployment over Kubernetes, it is a good idea to understand the entire deployment stack, which can help us to think about the right infrastructure, orchestration platform, and libraries. The following figure shows the various layers of the GenAI deployment stack, from the foundational infrastructure layer comprising compute, storage, and networking through to the orchestration, tools, and deployment layers.

Figure1.6 – Deployment stack for GenAI applications

Let’s a closer look at each of these layers:

• Infrastructure layer: We will start from the foundation layer of the stack and move upward. The foundation of this stack is the infrastructure layer, which covers compute, networking, and storage options:
  • Compute: For compute, we can use options such as CPUs, GPUs, custom accelerators, or a combination of these. As explained previously, LLMs are very computationally...

GenAI is transforming all industries. As per McKinsey (https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-economic-potential-of-generative-ai-the-next-productivity-frontier#key-insights), it is expected to add trillions of dollars to the economy by 2030. The following are some of the industry verticals affected and use cases. It is by no stretch a comprehensive list, as the list of applications is growing very rapidly.

However, learning about some of these will give you an idea of the potential GenAI carries:

• Retail/e-commerce:
  • Product design: GenAI can be used with every stage of product design, such as summarizing market and user research, creating visuals/animations of possible product options, refining concepts based on user feedback, and creating product descriptions and marketing campaigns.
  • Personalized recommendation: GenAI can enable more natural engagement for e-commerce websites. Users can ask in natural language for product...

In this chapter, we covered the differences between AI and GenAI. AI is a very broad term that refers to technologies that enable machines to emulate human intelligence and encompasses a broad range of applications, including GenAI, whereas GenAI specifically focuses on creating new content, such as text, images, and videos.

We then looked at the evolution of ML, understanding its progression from CNNs/RNNs to the transformer architecture introduced in 2017. Transformers have revolutionized AI with their ability to process sequences of data efficiently, making them fundamental to many GenAI applications, particularly in NLP.

The chapter also outlined the life cycle of a GenAI project, which includes business objectives and KPIs, foundational model selection, model training, evaluation, and deployment. Each stage is critical, with continuous iterations based on performance feedback.

Finally, the chapter covered various use cases of GenAI across different sectors, including...

In this section, we will provide a basic overview of how the transformer model works, including a mathematical explanation of its functionality. We discussed the concepts of queues, keys, and values as part of transformer analysis earlier in this chapter. For a given attention head i, the following are the query, key, and value vectors:

Q= X* WiQ

K= X* WiK

V= X* WiV

Where WiQ, WiK, and WiV are the weight vectors for the attention head i for the query, key, and values. These weights are the parameters that we optimize as we train the model.

To understand the computational complexity of these calculations, let’s look over the dimensionality of these vectors:

• X= [n, dmodel], where n is the number of tokens in the input sequence and dmodel is the dimensionality of the multi-dimensional space.
• Weight vectors: WiQ Wik Wiv = [dmodel ,dk ], where dk =dmodel / # of attention...