Notes from the Deep Reinforcement Learning Course by HuggingFace UNIT 1. INTRODUCTION TO DEEP REINFORCEMENT LEARNING Definition Reinforcement learning is a framework where an agent learns by doing actions within an environment and receiving rewards Return function Let $r_t$ be the reward received at the time step $t$.The cumulative reward(=return) $R(\tau)$ received from time step $t$, following the trajectory $\tau$ (sequence of states/actions) with a discount rate $\gamma$ is defined by:

DeepLearning.AI MLOps Specialization notes Course 1: Introduction to Machine Learning in Production Week 1: Overview of the ML Lifecycle and Deployment Concept drift / Data drift When data in production diverges from training data. Steps of a ML project Scoping Data modeling Deployment This is not a linear but an iterative process. Deployment patterns gradual (Canary) deployment: Send an increasing proportion of traffic to the new version. blue-green deployment: Add a router between the old and new algorithm then switch, when we have to switch all users at once but are able to rollback instantly.

Kubernetes concepts Node: server (physical or VM) Pod: low level abstraction over a container (abstract docker or another container technology). Usually one container by pod each pod has its own virtual IP address (internal to K8). if The pod die, the new one has a new Ip address. Service component: a permanent IP address that can be attached to each pod so pods can communicates with each others. It’s also a Load balancer to distribute requests to pods less busy (which are replacted)