2.3 GenAI and AI Project

 

2.3 AI Literacy

Transformer Architecture Overview

• Revolutionized AI through parallel processing vs sequential processing
• Three key processing steps:
  1. Tokenization - breaking input into smaller chunks (words, sub-words, or characters)
  2. Word embedding - converting tokens to vector representations with semantic meaning
  3. Positional encoding - maintaining word order despite parallel processing
• Enables faster, more robust text processing than traditional neural networks

BERT (Encoder-Only Models)

• Bidirectional Encoder Representations from Transformers
• Uses masked language modeling for training:
  • Randomly masks 15% of input tokens
  • Model predicts masked words using contextual understanding
  • Learns context rather than sequence
• Key use cases:
  • Text classification and fraud detection
  • Search engine optimization
  • Question answering systems

GPT Models (Decoder-Only)

• Generative Pre-trained Transformers for content generation
• Core capabilities:
  • Zero-shot learning - tasks without examples
  • Few-shot learning - tasks with example demonstrations
  • Prompt engineering - iterative instruction refinement
• Applications: chatbots, content generation, AI tutoring

Natural Language Processing (NLP)

• Two main components:
  1. Natural Language Understanding (NLU) - interpreting input meaning
  2. Natural Language Generation (NLG) - producing human-like responses
• Processing workflow:
  • Input analysis → Intent identification → Entity extraction → Context leveraging → Response generation
• Business applications: customer service, sentiment analysis, language translation

Bank of America Case Study

• Implemented Erica AI assistant using NLP and machine learning
• Addressed challenges:
  • High call volumes and wait times
  • Inconsistent responses
  • Need for 24/7 support
• Results: automated banking services, smart search capabilities, improved customer engagement

Generative AI Model Types

• Transformer-based models: GPT, BERT, T5 (encoder-decoder combination)
• GANs (Generative Adversarial Networks): two competing models (generator vs discriminator)
• Variational Auto-Encoders (VAEs): compression with statistical variation for multiple outputs
• Diffusion models: for image generation (covered in Unit 6)

Fine-Tuning vs Other Approaches

• Prompt engineering: refining instructions without changing model weights
• RAG (Retrieval Augmented Generation): grounding responses in external data
• Fine-tuning: retraining specific model layers on domain-specific data
  • Example: GPT model specialized for healthcare by training on medical datasets

Prompt Engineering Best Practices

• Be specific in instructions
• Iterate and refine prompts based on outputs
• Use templates for consistency
• Consider courtesy in prompting (may improve responses with some models)
• Zero-shot vs few-shot approaches depending on task complexity