diff --git a/2026/day-01/introduction-to-devops.md b/2026/day-01/introduction-to-devops.md new file mode 100644 index 0000000000..a7c70b707e --- /dev/null +++ b/2026/day-01/introduction-to-devops.md @@ -0,0 +1,14 @@ +# πŸš€ Day 01 Task - Introduction to DevOps & Cloud Computing + +**Author:** Rashid Khan +**Batch:** #90DaysOfDevOps (2026) + +--- + +## πŸ“Œ Solution Overview +DevOps is a set of practices, tools, and a cultural philosophy that automates and integrates the processes between software development and IT teams. + +### Key Learnings & Practical Applications: +- **Development + Operations Integration:** Reducing delivery cycles and manual handoffs. +- **CI/CD Pipeline Fundamentals:** Automating code testing and production deployments. +- **Cloud Infrastructure Benefits:** On-demand resource provisioning, scalability, and high availability. diff --git a/2026/day-02/linux-basic-commands.md b/2026/day-02/linux-basic-commands.md new file mode 100644 index 0000000000..c0f0e848be --- /dev/null +++ b/2026/day-02/linux-basic-commands.md @@ -0,0 +1,12 @@ +# 🐧 Day 02 Task - Linux Architecture & Essential Commands + +**Author:** Rashid Khan +**Batch:** #90DaysOfDevOps (2026) + +--- + +## πŸ“Œ Essential Commands & Terminal Outputs +- `pwd` β€” Print working directory (`/home/ubuntu/90DaysOfDevOpsChallengeRashidKhan`) +- `ls -la` β€” List files with permissions, hidden items, and detailed metadata +- `cd` & `mkdir` β€” Navigation and directory tree management +- `uname -r` β€” View Linux Kernel Version diff --git a/2026/day-03/linux-permissions-user-management.md b/2026/day-03/linux-permissions-user-management.md new file mode 100644 index 0000000000..04fccb2611 --- /dev/null +++ b/2026/day-03/linux-permissions-user-management.md @@ -0,0 +1,11 @@ +# πŸ” Day 03 Task - Linux Permissions & User Management + +**Author:** Rashid Khan +**Batch:** #90DaysOfDevOps (2026) + +--- + +## πŸ“Œ Key Practices & Commands +- **Ownership Management:** `sudo chown ubuntu:ubuntu filename` +- **Permission Bit Configuration:** `chmod 755 filename` (`rwxr-xr-x`) +- **User & Group Administration:** `sudo useradd` & `sudo usermod -aG sudo username` diff --git a/2026/day-04/package-and-services-guide.md b/2026/day-04/package-and-services-guide.md new file mode 100644 index 0000000000..724360bf61 --- /dev/null +++ b/2026/day-04/package-and-services-guide.md @@ -0,0 +1,11 @@ +# πŸ“¦ Day 04 Task - Package Management & Systemctl Services + +**Author:** Rashid Khan +**Batch:** #90DaysOfDevOps (2026) + +--- + +## πŸ“Œ Key Operations +- **Package Maintenance:** `sudo apt update && sudo apt upgrade -y` +- **Systemd Service Inspection:** `sudo systemctl status nginx` +- **Service Boot Enablement:** `sudo systemctl enable nginx` diff --git a/2026/day-05/linux-troubleshooting-runbook.md b/2026/day-05/linux-troubleshooting-runbook.md new file mode 100644 index 0000000000..e34da0371c --- /dev/null +++ b/2026/day-05/linux-troubleshooting-runbook.md @@ -0,0 +1,68 @@ +# πŸ› οΈ Linux Incident Troubleshooting & Triage Runbook + +**Author:** Rashid Khan +**Batch:** #90DaysOfDevOps (2026) +**Task:** Day 05 - Production System Diagnostics & Incident Mitigation + +--- + +## πŸ“Œ Incident Resolution Workflow + +Whenever a server issues an alert, I follow this 4-step triage methodology: +1. **Resource Check:** Inspect RAM, CPU, and Disk space. +2. **Process Check:** Identify high-resource or hung process IDs (PIDs). +3. **Network Check:** Verify open ports and listening sockets. +4. **Log Inspection:** Check systemd logs and restart failed services. + +--- + +## 1. System Resource Diagnostics + +### A. Check RAM Usage +```bash +free -h -t +``` + +### B. Check Disk Space +```bash +df -h +``` + +### C. Check System Load +```bash +uptime +``` + +## 2. Process Identification & Management + +### A. Find Top CPU Consuming Processes +```bash +ps aux --sort=-%cpu | head -n 10 +``` +### B. Find Top RAM Consuming Processes +```bash +ps aux --sort=-%mem | head -n 10 +``` +### C. Kill Unresponsive Processes +```bash +pgrep -fl nginx +kill 15 +kill -9 +``` +## 3. Network & Port Diagnostics + +### A. Check Open Listening Ports +```bash +sudo ss -tulpn | grep LISTEN +``` + +## 4. Service Lifecycle & Log Inspection + +### A. Check Service Status +```bash +sudo systemctl status nginx +``` +### B. View Live Logs for Troubleshooting +```bash +sudo journalctl -u nginx -n 50 --no-pager +``` diff --git a/2026/day-06/file-io-practice.md b/2026/day-06/file-io-practice.md new file mode 100644 index 0000000000..c85dca7dd1 --- /dev/null +++ b/2026/day-06/file-io-practice.md @@ -0,0 +1,60 @@ +# πŸ“ Day 06 Task - Linux Fundamentals: Read and Write Text Files + +**Author:** Rashid Khan +**Batch:** #90DaysOfDevOps (2026) + +--- + +## πŸ“Œ Overview +File manipulation (creating, writing, appending, and inspecting) is a core foundational skill in DevOps. System configurations, deployment logs, and automation scripts are all plain text files. Mastering CLI file I/O operations enables faster debugging and seamless scripting. + +--- + +## πŸ› οΈ Step-by-Step Hands-on Execution & Output Log + +### 1. File Creation & Line Overwriting (`>`) +Created an empty file `notes.txt` and initialized it with the first line using stdout redirection (`>`): +```bash +touch notes.txt +echo "Line 1: Initializing Linux File I/O Practice" > notes.txt +``` + +### 2. Appending Lines (>>) +Appended new log entries without overwriting existing file content using (`>>`): +```bash +echo "Line 2: Appending logs using standard redirection" >> notes.txt +``` + +### 3. Simultaneous Output & Append (tee -a) +Used (`tee -a`) to append a new line to the file while simultaneously piping stdout to the terminal: +```bash +echo "Line 3: Writing and displaying live via tee" | tee -a notes.txt +``` +#### Terminal Output: +Line 3: Writing and displaying live via tee + +### 4. Reading Full File Content (cat) +Inspected the complete contents of (`notes.txt`): +```bash +cat notes.txt +``` +#### Terminal Output: +Line 1: Initializing Linux File I/O Practice +Line 2: Appending logs using standard redirection +Line 3: Writing and displaying live via tee + +### 5. Inspecting File Parts (head & tail) +Extracted specific top and bottom lines for quick log auditing: +```bash +# Read first 2 lines +head -n 2 notes.txt + +# Read last 2 lines +tail -n 2 notes.txt +``` +#### Terminal Output (head -n 2): +Line 1: Initializing Linux File I/O Practice +Line 2: Appending logs using standard redirection +#### Terminal Output (tail -n 2): +Line 2: Appending logs using standard redirection +Line 3: Writing and displaying live via tee diff --git a/2026/day-06/notes.txt b/2026/day-06/notes.txt new file mode 100644 index 0000000000..a1e30f529c --- /dev/null +++ b/2026/day-06/notes.txt @@ -0,0 +1,3 @@ +Line 1: Initializing Linux File I/O Practice +Line 2: Appending logs using standard redirection +Line 3: Writing and displaying live via tee diff --git a/2026/day-07/backup.sh b/2026/day-07/backup.sh new file mode 100755 index 0000000000..98c09975c4 --- /dev/null +++ b/2026/day-07/backup.sh @@ -0,0 +1,2 @@ +#!/bin/bash +echo "Running Backup..." diff --git a/2026/day-07/day-07-linux-fs-and-scenarios.md b/2026/day-07/day-07-linux-fs-and-scenarios.md new file mode 100644 index 0000000000..1ca8b5c041 --- /dev/null +++ b/2026/day-07/day-07-linux-fs-and-scenarios.md @@ -0,0 +1,117 @@ +# Day 07 Task - Linux File System Hierarchy & Scenario-Based Practice + +**Author:** Rashid Khan +**Batch:** #90DaysOfDevOps (2026) + +--- + +## Part 1: Linux File System Hierarchy + +Understanding where system assets reside is critical for rapid incident diagnosis and system configuration. + +| Directory | Purpose & Contents | Command Example | Use Case | +| :--- | :--- | :--- | :--- | +| **`/` (root)** | Top-level starting point of the entire directory tree | `ls -l /` | Navigating overall file system structure. | +| **`/home`** | Personal directories for regular users | `ls -la ~` | Storing user scripts, configs, and SSH keys. | +| **`/root`** | Home directory for the administrative superuser | `sudo ls -l /root` | Administrative maintenance and root-specific operations. | +| **`/etc`** | System-wide configuration files and startup scripts | `cat /etc/hostname` | Modifying network, system, or application settings. | +| **`/var/log`** | Variable data files, specifically application & system logs | `du -sh /var/log/* 2>/dev/null \| sort -h \| tail -5` | Debugging service failures and auditing logs. | +| **`/tmp`** | Temporary files created by system and programs | `ls -l /tmp` | Inspecting transient runtime data (cleared on reboot). | +| **`/bin`** | Essential user command binaries | `ls /bin` | Running single-user mode recovery utilities (`ls`, `cat`). | +| **`/usr/bin`** | Primary directory for user executable commands | `ls /usr/bin` | Standard CLI application execution. | +| **`/opt`** | Subdirectory for add-on/third-party application software packages | `ls /opt` | Installing standalone enterprise software (e.g., custom agents). | + +--- + +## Part 2: Scenario-Based Troubleshooting Practice + +### Scenario 1: Service Not Starting (`myapp`) +**Problem:** A web application service called `myapp` failed to start after a server reboot. + + **Step 1:** Check current service status. + ```bash + systemctl status myapp + ``` + Why: Identifies whether the service is inactive, active, or in a failed state. + +**Step 2:** Inspect failure logs in systemd journal. + ```bash + journalctl -u myapp -n 50 + ``` + Why: Extracts the last 50 log lines to pinpoint stdout/stderr runtime errors. + +**Step 3:** Verify if the service is configured to auto-start on boot. + ```bash + systemctl is-enabled myapp + ``` + Why: Determines if the unit file is missing boot-time enablement. + +**Step 4:** List all active services if myapp is missing. + ```bash + systemctl list-units --type=service + ``` + Why: Confirms exact service naming syntax installed on the host. + +### Scenario 2: High CPU Usage Investigation +**Problem:** Server response latency is high due to CPU saturation. + + **Step 1:** Inspect top CPU-consuming processes interactively. + ```bash + top + ``` + Why: Displays real-time live resource usage breakdown sorted by %CPU. + + **Step 2:** Get a top-10 snapshot sorted by CPU usage. + ```bash + ps aux --sort=-%cpu | head -10 + ``` + Why: Instantly lists process owner, PID, CPU percentage, and exact executing command line. + +### Scenario 3: Finding & Tailing Service Logs (`docker`) +**Problem:** Developer requests real-time log monitoring for the docker service. + + **Step 1:** Verify active Docker service status. + ```bash + systemctl status docker + ``` + Why: Ensures the docker daemon is active before inspecting logs. + + **Step 2:** Inspect recent Docker logs. + ```bash + journalctl -u docker -n 50 + ``` + Why: Reviews recent startup and operational logs. + + **Step 3:** Stream live Docker logs continuously. + ```bash + journalctl -u docker -f + ``` + Why: Follows real-time log streams for active debugging. + +### Scenario 4: Fixing Script Execution Permission (`backup.sh`) +**Problem:** Executing /home/user/backup.sh throws Permission denied. + **Step 1:** Check current file permissions. + ```bash + ls -l /home/user/backup.sh + ``` + Output: -rw-r--r-- (Missing execute x permissions). + + **Step 2:** Add execute permission for the file owner/group. + ```bash + chmod +x /home/user/backup.sh + ``` + Why: Grants binary execution privileges. + + **Step 3:** Verify permission change. + ```bash + ls -l /home/user/backup.sh + ``` + Output: -rwxr-xr-x (Execute permission granted). + + **Step 4:** Execute the backup script. + ```bash + ./backup.sh + ``` + + + diff --git a/2026/day-08/day-08-cloud-deployment.md b/2026/day-08/day-08-cloud-deployment.md new file mode 100644 index 0000000000..010ec27d91 --- /dev/null +++ b/2026/day-08/day-08-cloud-deployment.md @@ -0,0 +1,74 @@ +# Day 08 – Cloud Server Setup: Docker, Nginx & Web Deployment + +## Overview +Today's task focused on launching an AWS EC2 instance, establishing remote SSH communication, installing Nginx and Docker, configuring AWS Security Groups for HTTP Port 80 and Custom TCP Port 8080 access, and extracting access logs for audit. + +--- + +## Step-by-Step Execution & Screenshots + +### 1. Cloud Instance Access & System Setup +Establish secure remote connection using SSH: +```bash +chmod 400 devopsrashid.pem +ssh -i devopsrashid.pem ubuntu@ +``` +![SSH Connection Terminal](ssh-connection.png) + +Update system package indexes: +```bash +sudo apt update && sudo apt upgrade -y +``` + +### 2. Nginx & Docker Setup +Install Nginx web server: +```bash +sudo apt install nginx -y +sudo systemctl status nginx +``` + +Install Docker engine and run an isolated Nginx container on port 8080: +```bash +sudo apt install docker.io -y +sudo systemctl enable --now docker +sudo usermod -aG docker $USER +sudo docker run -d -p 8080:80 --name docker-nginx-test nginx +``` + +Verify active container status: +```bash +sudo docker ps +``` +![Docker Nginx Container Running](docker-nginx.png) + +### 3. Inbound Web Verification +Configured AWS Security Group inbound rules for HTTP (Port 80). Verified web accessibility via browser: +![Nginx Welcome Webpage](nginx-webpage.png) + +### 4. Log Extraction & Local SCP Transfer +Export active access logs and transfer to local machine for audit: +```bash +sudo tail -n 50 /var/log/nginx/access.log > ~/nginx-logs.txt +scp -i ~/Downloads/devopsrashid.pem ubuntu@:~/nginx-logs.txt ~/Downloads/ +``` + +## Challenges Faced & Solutions +**Challenge:** Nginx installed and running successfully, but the web page was timing out on the browser using EC2 +Public IP + +**Root Cause:** AWS Security Group inbound firewall rules only permitted SSH traffic on Port 22 by default. + +**Solution:** Navigated to AWS EC2 Security Groups -> Inbound Rules, added HTTP (Port 80) and Custom TCP (Port 8080) +from 0.0.0.0/0. + +## What I Learned +**Cloud Provisioning:** Creating, managing, and connecting to AWS EC2 instances securely via SSH. + +**Network Security:** Configuring Security Groups to allow specific inbound web traffic. + +**Application Deployment:** Running web servers via native Linux services as well as isolated Docker containers. + +**Log Auditing:** Isolating log entries from /var/log/nginx/access.log and pulling them locally using scp. + + + diff --git a/2026/day-08/docker-nginx.png b/2026/day-08/docker-nginx.png new file mode 100644 index 0000000000..a5712a8c9f Binary files /dev/null and b/2026/day-08/docker-nginx.png differ diff --git a/2026/day-08/nginx-logs.txt b/2026/day-08/nginx-logs.txt new file mode 100644 index 0000000000..4d296a239f --- /dev/null +++ b/2026/day-08/nginx-logs.txt @@ -0,0 +1,5 @@ +43.248.243.0 - - [12/Aug/2026:22:28:54 +0000] "GET / HTTP/1.1" 200 409 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/27.0 Safari/605.1.15" +43.248.243.0 - - [12/Aug/2026:22:28:55 +0000] "GET /favicon.ico HTTP/1.1" 404 134 "http://100.53.20.2/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/27.0 Safari/605.1.15" +146.190.169.94 - - [12/Aug/2026:22:35:24 +0000] "GET / HTTP/1.1" 200 409 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" +205.210.31.242 - - [12/Aug/2026:22:46:58 +0000] "GET / HTTP/1.0" 200 615 "-" "Hello from Palo Alto Networks, find out more about our scans in https://docs-cortex.paloaltonetworks.com/r/1/Cortex-Xpanse/Scanning-activity" +::1 - - [12/Aug/2026:23:32:42 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.18.0" diff --git a/2026/day-08/nginx-webpage.png b/2026/day-08/nginx-webpage.png new file mode 100644 index 0000000000..f87f3641f9 Binary files /dev/null and b/2026/day-08/nginx-webpage.png differ diff --git a/2026/day-08/ssh-connection.png b/2026/day-08/ssh-connection.png new file mode 100644 index 0000000000..4129645dc2 Binary files /dev/null and b/2026/day-08/ssh-connection.png differ diff --git a/2026/day-09/day-09-user-management.md b/2026/day-09/day-09-user-management.md new file mode 100644 index 0000000000..59192907be --- /dev/null +++ b/2026/day-09/day-09-user-management.md @@ -0,0 +1,84 @@ +# Day 09 Challenge + +## Users & Groups Created +- Users: tokyo, berlin, professor, nairobi +- Groups: developers, admins, project-team + +## Group Assignments +- tokyo -> developers, project-team +- berlin -> developers, admins +- professor -> admins +- nairobi -> project-team + +## Directories Created +- `/opt/dev-project` (Owner: root, Group: developers, Permissions: 775) +- `/opt/team-workspace` (Owner: root, Group: project-team, Permissions: 775) + +## Commands Used & Execution Proof + +### Task 1: Create Users +```bash +sudo useradd -m tokyo +sudo useradd -m berlin +sudo useradd -m professor +sudo passwd tokyo +sudo passwd berlin +sudo passwd professor +tail -n 3 /etc/passwd +ls -l /home/ +``` +![Task 1 - Create Users](./task-1-create-users.png) + +### Task 2 & 3: Create Groups and Assign Users +```bash +sudo groupadd developers +sudo groupadd admins +sudo usermod -aG developers tokyo +sudo usermod -aG developers,admins berlin +sudo usermod -aG admins professor +tail -n 2 /etc/group +groups tokyo berlin professor +``` +![Task 2 - Groups & Users](./task-2-groups.png) + +### Task 4: Shared Directory +```bash +sudo mkdir -p /opt/dev-project +sudo chgrp developers /opt/dev-project +sudo chmod 775 /opt/dev-project +sudo -u tokyo touch /opt/dev-project/tokyo_file.txt +sudo -u berlin touch /opt/dev-project/berlin_file.txt +ls -ld /opt/dev-project +ls -l /opt/dev-project/ +``` +![Task 4 - Shared Directory](./task-4-shared-dir.png) + +### Task 5: Team Workspace +```bash +sudo useradd -m nairobi +sudo groupadd project-team +sudo usermod -aG project-team nairobi +sudo usermod -aG project-team tokyo +sudo mkdir -p /opt/team-workspace +sudo chgrp project-team /opt/team-workspace +sudo chmod 775 /opt/team-workspace +sudo -u nairobi touch /opt/team-workspace/nairobi_test.txt +ls -ld /opt/team-workspace +ls -l /opt/team-workspace/ +``` +![Task 5 - Team Workspace](./task-5-team-workspace.png) + + +## What I Learned + +**The `-aG` Flag:** Always use `-aG` with `usermod` to append a user to a group. Using just `-G` removes the user from + thier existing secondary groups. + +**Shared Directory Permissions:** Using `chmod 775` along with `chgrp` is the standard way to allow a specific group + to collaboratively read, write, and execute files within a shared directory. + +**Cross-User Privilege Testing:** Using `sudo -u ` is an incredibly efficient way to simulate and + verify another user's permissions without needing their password. + + + diff --git a/2026/day-09/task-1-create-users.png b/2026/day-09/task-1-create-users.png new file mode 100644 index 0000000000..ab17fadd39 Binary files /dev/null and b/2026/day-09/task-1-create-users.png differ diff --git a/2026/day-09/task-2-groups.png b/2026/day-09/task-2-groups.png new file mode 100644 index 0000000000..d693190e91 Binary files /dev/null and b/2026/day-09/task-2-groups.png differ diff --git a/2026/day-09/task-4-shared-dir.png b/2026/day-09/task-4-shared-dir.png new file mode 100644 index 0000000000..c3b3aba1bf Binary files /dev/null and b/2026/day-09/task-4-shared-dir.png differ diff --git a/2026/day-09/task-5-team-workspace.png b/2026/day-09/task-5-team-workspace.png new file mode 100644 index 0000000000..5392d9b9db Binary files /dev/null and b/2026/day-09/task-5-team-workspace.png differ diff --git a/2026/day-10/chmod.png b/2026/day-10/chmod.png new file mode 100644 index 0000000000..2b47c93556 Binary files /dev/null and b/2026/day-10/chmod.png differ diff --git a/2026/day-10/day-10-file-permissions.md b/2026/day-10/day-10-file-permissions.md new file mode 100644 index 0000000000..d9939b4e79 --- /dev/null +++ b/2026/day-10/day-10-file-permissions.md @@ -0,0 +1,53 @@ +# Day 10 Challenge + +## Files Created +- `devops.txt` (Empty file created using `touch`) +- `notes.txt` (File with content created using `echo`) +- `script.sh` (Bash script created using `vim`) +- `project/` (Directory created using `mkdir`) + +![Creating Files](files.png) +![Reading Files](read.png) + +## Permission Changes + +**Initial Permissions Analysis:** +- Default files were created with `rw-rw-r--` (Read/Write for Owner & Group, Read for Others). +![Default Permissions](perms.png) + +**Modifications Made:** +- **`script.sh`:** Before `-rw-rw-r--` | After `-rwxrwxr-x` (Made executable) +- **`devops.txt`:** Before `-rw-rw-r--` | After `-r--r--r--` (Made read-only for all) +- **`notes.txt`:** Before `-rw-rw-r--` | After `-rw-r-----` (Set to 640 numeric) +- **`project/`:** Set directly to `drwxr-xr-x` (Set to 755 numeric) + +![Modified Permissions](chmod.png) + +## Commands Used +```bash +# Creating files +mkdir day-10 +touch devops.txt +echo "Learning File Permissions on Day 10" > notes.txt +vim script.sh + +# Reading files +cat notes.txt +vim -R script.sh +head -n 5 /etc/passwd +tail -n 5 /etc/passwd + +# Modifying permissions +chmod +x script.sh +chmod a-w devops.txt +chmod 640 notes.txt +mkdir project +chmod 755 project +``` + +## What I Learned +1. **The 4-2-1 Numeric Rule:** Permissions are simple math! Read is 4, Write is 2, and Execute is 1. Combining them (like 6 for rw, or 7 for rwx) gives you exact control over access. +2. **Execution is Explicit:** Linux does not allow any file to run as a script automatically. You must explicitly grant execute (`+x`) permissions, which is a massive security advantage. +3. **Intentional Errors are Guides:** Trying to write to a read-only file or executing a non-executable file triggers `Permission denied`. Understanding these errors helps in debugging server incidents faster. + +![Intentional Permission Errors](error.png) diff --git a/2026/day-10/error.png b/2026/day-10/error.png new file mode 100644 index 0000000000..85d7b784d5 Binary files /dev/null and b/2026/day-10/error.png differ diff --git a/2026/day-10/files.png b/2026/day-10/files.png new file mode 100644 index 0000000000..8873b68178 Binary files /dev/null and b/2026/day-10/files.png differ diff --git a/2026/day-10/perms.png b/2026/day-10/perms.png new file mode 100644 index 0000000000..5deb008337 Binary files /dev/null and b/2026/day-10/perms.png differ diff --git a/2026/day-10/read.png b/2026/day-10/read.png new file mode 100644 index 0000000000..c39b0c923c Binary files /dev/null and b/2026/day-10/read.png differ diff --git a/2026/day-11/chgrp.png b/2026/day-11/chgrp.png new file mode 100644 index 0000000000..87ca4fde62 Binary files /dev/null and b/2026/day-11/chgrp.png differ diff --git a/2026/day-11/chown-combined.png b/2026/day-11/chown-combined.png new file mode 100644 index 0000000000..4ce02aa737 Binary files /dev/null and b/2026/day-11/chown-combined.png differ diff --git a/2026/day-11/chown-recursive.png b/2026/day-11/chown-recursive.png new file mode 100644 index 0000000000..ac69333004 Binary files /dev/null and b/2026/day-11/chown-recursive.png differ diff --git a/2026/day-11/chown.png b/2026/day-11/chown.png new file mode 100644 index 0000000000..504e68ec5b Binary files /dev/null and b/2026/day-11/chown.png differ diff --git a/2026/day-11/day-11-file-ownership.md b/2026/day-11/day-11-file-ownership.md new file mode 100644 index 0000000000..04f3472817 --- /dev/null +++ b/2026/day-11/day-11-file-ownership.md @@ -0,0 +1,94 @@ +# Day 11 Challenge: File Ownership (chown & chgrp) + +This file contains the practical documentation for Day 11 of the #90DaysOfDevOps challenge. Today, I mastered Linux File Ownership using a "Bank Heist" theme to understand Role-Based Access Control (RBAC). + +--- + +## Users & Groups Created +- **Users:** `tokyo`, `berlin`, `nairobi`, `professor` +- **Groups:** `heist-team`, `planners`, `vault-team`, `tech-team` + +## Files & Directories Created +- `devops-file.txt` +- `team-notes.txt` +- `project-config.yaml` +- `app-logs/` (directory) +- `heist-project/` (directory with subfolders `vault/` and `plans/`, files `gold.txt` and `strategy.conf`) +- `bank-heist/` (directory with files `access-codes.txt`, `blueprints.pdf`, `escape-plan.txt`) + +## Ownership Changes +- `devops-file.txt`: ubuntu:ubuntu β†’ tokyo:ubuntu β†’ berlin:ubuntu +- `team-notes.txt`: ubuntu:ubuntu β†’ ubuntu:heist-team +- `project-config.yaml`: ubuntu:ubuntu β†’ professor:heist-team +- `app-logs/`: ubuntu:ubuntu β†’ berlin:heist-team +- `heist-project/` (and its contents): ubuntu:ubuntu β†’ professor:planners +- `bank-heist/access-codes.txt`: root:root β†’ tokyo:vault-team +- `bank-heist/blueprints.pdf`: root:root β†’ berlin:tech-team +- `bank-heist/escape-plan.txt`: root:root β†’ nairobi:vault-team + +Example: +- devops-file.txt: user:user β†’ tokyo:heist-team + +## Commands Used +- `ls -l` : Check detailed permissions and ownership of files. +- `ls -lR` : Check ownership recursively for a directory. +- `sudo chown ` : Change the owner of a file. +- `sudo chgrp ` : Change the group of a file. +- `sudo chown : ` : Change both owner and group in one command. +- `sudo chown -R : ` : Change ownership recursively for a folder and everything inside it. + +--- + +## Steps Executed + +## Step 1: Understanding Ownership +Identified the difference between an owner and a group in Linux using the `ls -l` command. +* **Owner:** The specific user who has primary control over the file. +* **Group:** A collection of users who share specific access permissions to the file. + +![Understanding Ownership Concept](owners.png) + +--- +️ +## Step 2: Basic `chown` Operations +Created users `tokyo` and `berlin`. Changed the ownership of `devops-file.txt` from the default user to `tokyo`, and then transferred it to `berlin`. + +![Basic chown Operations](chown.png) + +--- + +## Step 3: Basic `chgrp` Operations +Created a new group called `heist-team`. Assigned the `team-notes.txt` file to this specific group so multiple team members can access it safely. + +![Basic chgrp Operations](chgrp.png) + +--- + +## Step 4: Combined Owner & Group Change +Learned the time-saving colon (`:`) shortcut. Changed both the owner and the group of `project-config.yaml` and the `app-logs` directory in a single command (`chown owner:group`). + +![Combined chown and chgrp](chown-combined.png) + +--- + +## Step 5: Recursive Ownership (`-R`) +Created a complex project directory (`heist-project/`). Instead of changing file ownership one by one, used the `-R` (Recursive) flag to change the owner to `professor` and group to `planners` for all internal folders and files simultaneously. + +![Recursive Ownership Change](chown-recursive.png) + +--- + +## Step 6: The Final Bank Heist Practice Challenge +Set up a real-world RBAC (Role-Based Access Control) microservices scenario. Created a `bank-heist` directory and restricted access to different files based on specific roles: +* `access-codes.txt` β†’ Owner: tokyo, Group: vault-team +* `blueprints.pdf` β†’ Owner: berlin, Group: tech-team +* `escape-plan.txt` β†’ Owner: nairobi, Group: vault-team + +![Final Bank Heist Structure](heist-final.png) + +--- + +## What I Learned (Key Takeaways) +1. **The Sudo Barrier:** You cannot give away file ownership to another user without `sudo` privileges. This is a brilliant built-in security constraint in Linux. +2. **The `:` Shortcut:** `sudo chown user:group file.txt` is the most efficient way to configure ownership. +3. **Real-world Application:** Proper ownership (`chown -R`) is the backbone of deploying web servers (like Nginx), managing CI/CD pipeline artifacts, and configuring shared team workspaces. diff --git a/2026/day-11/heist-final.png b/2026/day-11/heist-final.png new file mode 100644 index 0000000000..d97751b469 Binary files /dev/null and b/2026/day-11/heist-final.png differ diff --git a/2026/day-11/owners.png b/2026/day-11/owners.png new file mode 100644 index 0000000000..cbb1296ea7 Binary files /dev/null and b/2026/day-11/owners.png differ diff --git a/2026/day-12/day-12-revision.md b/2026/day-12/day-12-revision.md new file mode 100644 index 0000000000..9c91270545 --- /dev/null +++ b/2026/day-12/day-12-revision.md @@ -0,0 +1,63 @@ +# Day 12: Breather & Revision (Days 01–11) + +## Section 1: Consolidation & Review Notes + +### Mindset & Plan Review +* **Goal Check:** The Day 01 goals are absolutely on track. Moving from basic commands to production-level scenarios (like RBAC & Cloud Deployment) has built strong confidence. +* **Tweaks:** I need to focus slightly more on "Muscle Memory". Going forward, I will try to type commands faster without looking at my cheat sheets. + +###️ Processes & Services Observation +* I re-ran process and log diagnostics for `nginx` to check its health and recent logs. +* **Commands Executed:** + ```bash + sudo systemctl status nginx + sudo journalctl -u nginx -n 50 --no-pager + ``` +* **Observation:** `systemctl status` gives an instant health snapshot (active/running), but `journalctl -u --no-pager` is the real lifesaver for seeing *why* a service failed in the backend logs without getting trapped in the terminal pager. + +![Service Health Check](service-check.png) + +### File Skills, User & Group Sanity (RBAC) +* Practiced a continuous flow: Created a workspace, appended data, took a safe backup, created a test auditor, and locked down access securely. +* **Commands Executed:** + ```bash + mkdir secure-audit + echo "Server health is 100% OK. No anomalies found." >> secure-audit/health-report.txt + cp secure-audit/health-report.txt secure-audit/backup-report.txt + sudo useradd -m inspector + id inspector + sudo chown -R inspector:inspector secure-audit/ + sudo chmod 640 secure-audit/* + ls -l secure-audit/ + ``` +* **Verification:** Verified via `id` and `ls -l`. Zero-Trust architecture successfully simulated at the file level. The copy command (`cp`) ensured we had a `.txt` backup before changing permissions. + +![File Skills & RBAC Test](file-skills-rbac.png) + +### Cheat Sheet Refresh: Top 5 Incident-Response Commands +If a server crashes at 2 AM, these are the first 5 commands I am reaching for: +1. `systemctl status ` (Is it dead or alive?) +2. `journalctl -xeu ` (Why did it die?) +3. `ps -aux | grep ` (Is a rogue process eating resources?) +4. `ls -lah` (Are permissions or file sizes messed up?) +5. `history | grep ` (What did the last engineer run before it broke?) + +--- + +## Section 2: Mini Self-Check + +**Q1. Which 3 commands save you the most time right now, and why?** +1. `grep`: Saves hours of reading by filtering exact errors from thousands of lines of logs. +2. `history`: Prevents me from retyping complex, long commands I figured out yesterday. +3. `ls -lah`: Gives me hidden files, human-readable sizes, and exact permission blocks in one go. + +**Q2. How do you check if a service is healthy? List the exact 2–3 commands you’d run first.** +1. `sudo systemctl status ` (To check active/failed state). +2. `sudo journalctl -u -n 50 --no-pager` (To read the last 50 error logs of that specific service). + +**Q3. How do you safely change ownership and permissions without breaking access? Give one example command.** +* **Safety Rule:** Never run `chmod 777`. Always target a specific directory and use recursive flags carefully. +* **Example:** `sudo chown -R inspector:inspector /secure-audit/` (Changes owner and group safely for a folder and its contents). Followed by `sudo chmod 640 /secure-audit/*` (Owner reads/writes, group reads, others get nothing). + +**Q4. What will you focus on improving in the next 3 days?** +* Automating these basic file and user operations using basic Bash scripting, and diving deeper into networking concepts to connect the dots between server files and web traffic. diff --git a/2026/day-12/file-skills-rbac.png b/2026/day-12/file-skills-rbac.png new file mode 100644 index 0000000000..db0acd8af8 Binary files /dev/null and b/2026/day-12/file-skills-rbac.png differ diff --git a/2026/day-12/service-check.png b/2026/day-12/service-check.png new file mode 100644 index 0000000000..9607f51e5b Binary files /dev/null and b/2026/day-12/service-check.png differ diff --git a/2026/day-13/day-13-lvm.md b/2026/day-13/day-13-lvm.md new file mode 100644 index 0000000000..6e7dc15b35 --- /dev/null +++ b/2026/day-13/day-13-lvm.md @@ -0,0 +1,131 @@ +# Day 13: Linux Volume Management (LVM) + +## Objective +Learn LVM to manage storage flexibly – create, extend, and mount volumes with zero downtime. + +--- + +️## Prerequisites & Setup (Virtual Disk) + +**Command:** +`sudo -i` +* **What I observed:** The terminal prompt changed to `root`, granting me superuser privileges which are strictly required to perform disk, partition, and volume manipulation safely. + +**Command:** +`dd if=/dev/zero of=/tmp/disk1.img bs=1M count=1024` +* **What I observed:** The system copied zeroes to create a 1GB raw image file in the `/tmp` directory. The output showed the records read/written and the total bytes copied (1073741824 bytes). + +**Command:** +`losetup -fP /tmp/disk1.img` +* **What I observed:** The command executed silently. It found the first available loop device and attached the 1GB image file to it, scanning for any existing partitions. + +**Command:** +`losetup -a` +* **What I observed:** The output displayed the mapping, confirming that the loop device (e.g., `/dev/loop0`) is now actively associated with `/tmp/disk1.img`. + +*(Note: For the challenge tasks below, `/dev/loop0` represents the newly attached virtual disk).* + +--- + +## Challenge Tasks + +### Task 1: Check Current Storage + +**Command:** +`lsblk` +* **What I observed:** Displayed a tree-like list of all block devices. I clearly saw my primary disks and the newly attached `/dev/loop0` of 1GB size with no partitions or mount points attached to it yet. + +**Command:** +`pvs` +* **What I observed:** Displayed the current Physical Volumes. Initially, it either showed default system PVs or nothing. `/dev/loop0` was not listed as it hadn't been initialized for LVM yet. + +**Command:** +`vgs` +* **What I observed:** Displayed Volume Groups. My custom VG was not there, proving the LVM pool was empty. + +**Command:** +`lvs` +* **What I observed:** Displayed Logical Volumes currently active. No custom LVs were present. + +**Command:** +`df -h` +* **What I observed:** Showed all currently mounted filesystems and their human-readable sizes. The new virtual disk `/dev/loop0` was absent because it was unformatted and unmounted. + +--- + +### Task 2: Create Physical Volume (PV) + +**Command:** +`pvcreate /dev/loop0` +* **What I observed:** The terminal outputted: `"Physical volume '/dev/loop0' successfully created."` This confirmed the raw disk was successfully initialized and stamped with an LVM header. + +**Command:** +`pvs` +* **What I observed:** Now, `/dev/loop0` appeared in the list under the 'PV' column with a size of exactly 1024M (1GB) and 0 free space used, proving the physical volume is ready. + +--- + +### Task 3: Create Volume Group (VG) + +**Command:** +`vgcreate devops-vg /dev/loop0` +* **What I observed:** The terminal outputted: `"Volume group 'devops-vg' successfully created."` This successfully grouped my physical volume into a unified storage pool named `devops-vg`. + +**Command:** +`vgs` +* **What I observed:** The new volume group `devops-vg` showed up in the list with 1 PV assigned to it, 0 LVs (Logical Volumes), and a total free size of ~1020M. + +--- + +### Task 4: Create Logical Volume (LV) + +**Command:** +`lvcreate -L 500M -n app-data devops-vg` +* **What I observed:** The terminal outputted: `"Logical volume 'app-data' created."` I instructed the system to carve out exactly 500MB from the `devops-vg` storage pool. + +**Command:** +`lvs` +* **What I observed:** The Logical Volume `app-data` is now clearly listed under the `devops-vg` Volume Group with an allocated size of 500.00m. + +--- + +### Task 5: Format and Mount + +**Command:** +`mkfs.ext4 /dev/devops-vg/app-data` +* **What I observed:** The terminal displayed the process of writing inode tables, block groups, and creating the journal. It successfully formatted the raw logical volume with the `ext4` filesystem. + +**Command:** +`mkdir -p /mnt/app-data` + +**Command:** +`mount /dev/devops-vg/app-data /mnt/app-data` +* **What I observed:** Both commands executed silently. The `mkdir` command created the mount point, and `mount` successfully attached the logical volume to the Linux directory tree at `/mnt/app-data`. + +**Command:** +`df -h /mnt/app-data` +* **What I observed:** It confirmed the mount was successful. It showed `/dev/mapper/devops--vg-app--data` mounted on `/mnt/app-data` with a total usable size of ~474M (the difference is due to ext4 filesystem journaling overhead). + +--- + +### Task 6: Extend the Volume (Live Scaling) + +**Command:** +`lvextend -L +200M /dev/devops-vg/app-data` +* **What I observed:** The terminal outputted: `"Size of logical volume devops-vg/app-data changed from 500.00 MiB to 700.00 MiB. Logical volume successfully resized."` This confirmed the underlying block device was expanded dynamically. + +**Command:** +`resize2fs /dev/devops-vg/app-data` +* **What I observed:** The terminal outputted: `"The filesystem on /dev/devops-vg/app-data is now block-size long."` This crucial step synced the `ext4` filesystem to recognize and utilize the newly added 200MB from the LVM layer. + +**Command:** +`df -h /mnt/app-data` +* **What I observed:** Verified the live extension successfully! The available size of the mount point instantly jumped from ~500M to ~700M, and all this happened without unmounting the drive or stopping any server processes. + +--- + +## What I Learned (Key Takeaways) + +1. **The LVM Architecture (PV -> VG -> LV):** I learned that LVM abstracts raw hardware. You initialize disks as Physical Volumes (PV), pool them into a Volume Group (VG), and then carve out Logical Volumes (LV) like slicing a pie. +2. **Live Resizing is a Superpower:** Unlike standard rigid partitions which require server reboots or unmounting (risking data loss and downtime), `lvextend` combined with `resize2fs` allows a DevOps engineer to instantly increase disk space on a live production server. +3. **Virtual Disks via Loop Devices:** Using `dd` and `losetup` is an incredible technique to simulate and practice hardware-level operations (like attaching physical hard drives) purely through software, making sandbox testing extremely safe. diff --git a/2026/day-14/day-14-networking.md b/2026/day-14/day-14-networking.md new file mode 100644 index 0000000000..2c9be9b236 --- /dev/null +++ b/2026/day-14/day-14-networking.md @@ -0,0 +1,62 @@ +# Day 14: Networking Fundamentals & Hands-on Checks + +## Objective +Master core networking concepts (OSI vs TCP/IP) and execute essential CLI commands to troubleshoot connectivity, DNS, and ports in a real-world Linux environment. + +--- + +## Quick Concepts: OSI vs TCP/IP + +* **OSI Layers (L1-L7):** A theoretical 7-layer framework to understand how data travels from a physical cable up to an application (Physical, Data Link, Network, Transport, Session, Presentation, Application). +* **TCP/IP Stack (4 Layers):** The practical, real-world model actually used on the internet (Network Access, Internet, Transport, Application). +* **Where protocols sit in the stack:** + * **IP (Internet Protocol):** L3 (Network Layer) - Handles logical addressing and routing. + * **TCP/UDP:** L4 (Transport Layer) - Handles port-to-port delivery. + * **HTTP/HTTPS & DNS:** L7 (Application Layer) - User-facing protocols for web data and name resolution. +* **Real Example:** Running `curl https://example.com` = Application layer (HTTPS) negotiating over Transport layer (TCP port 443) routed via Network layer (IP) over a physical medium. + +--- + +## Hands-on Checklist & Observations +*(Target Host Used: `google.com` | Local Env: AWS EC2 Ubuntu)* + +### 1. Identity (`hostname -I` / `ip addr show`) +* **Observation:** Instantly printed my EC2 private IP (`172.31.29.81`). This confirms my local identity on the AWS subnet. + +### 2. Reachability (`ping -c 4 google.com`) +* **Observation:** ICMP echo requests returned with 0% packet loss and low latency. Confirms basic L3 (Network layer) outbound connectivity is healthy. + +### 3. Path (`traceroute google.com`) +* **Observation:** Mapped the hops. First few were internal AWS gateways (very fast, ~0.2ms), followed by ISPs. Reached the target at hop 21. `* * *` in the middle showed that some routers block ICMP for security but still forward the traffic. + +### 4. Ports (`sudo ss -tulpn`) +* **Observation:** Displayed all active listening sockets. Noticed `sshd` bound to `0.0.0.0:22` and `nginx` bound to `0.0.0.0:80`. Confirms L4 is accepting traffic. + +### 5. Name Resolution (`dig google.com +short` / `nslookup`) +* **Observation:** Resolved human-readable domain to IPv4/IPv6 addresses instantly. Confirms L7 DNS resolver is perfectly translating names to IPs. + +### 6. HTTP Check (`curl -I https://google.com`) +* **Observation:** Fetched HTTP headers. Returned `HTTP/2 301 Moved Permanently` pointing to `www.google.com`. Confirms L7 web server is operational and handling URL redirection correctly. + +### 7. Connections Snapshot (`netstat -ant | head -n 15`) +* **Observation:** Saw a mix of states. Multiple sockets were in `LISTEN` state, and one specific connection on port 22 was `ESTABLISHED` with my local laptop's IP, proving active data transfer. + +--- + +## Mini Task: Port Probe & Interpret + +* **Identified Port:** SSH on port `22` +* **Probe Command:** `nc -zv localhost 22` +* **Observation:** Returned `Connection to localhost 22 port [tcp/ssh] succeeded!`. +* **Interpretation:** Is it reachable? Yes. If it had failed (e.g., Connection refused), my next check would be verifying the service status (`systemctl status ssh`) or inspecting the local firewall rules. + +--- + +## Reflection & Incident Response Methodology + +1. **Fastest Signal:** `curl -I ` gives the fastest overall health signal. If it returns a status code, layers 1 through 7 are functional. If it fails, I drop to `ping` (L3) to see if the server itself is down. +2. **DNS Failure:** If DNS fails, I would inspect the L7 configuration (`/etc/resolv.conf`) and test reachability to the DNS server directly (e.g., `ping 8.8.8.8`). +3. **HTTP 500 Error:** An HTTP 500 means L1-L4 are fine, but the application code (L7) crashed. Next step is checking application/web server logs (`journalctl` or `/var/log/nginx/error.log`). +4. **Two follow-up checks in a real incident:** + * Check system resources (`free -m` and `df -h`) to ensure the server isn't out of memory or disk space. + * Verify Cloud Security Groups / Firewalls to ensure ports aren't accidentally blocked. diff --git a/2026/day-15/day-15-networking-concepts.md b/2026/day-15/day-15-networking-concepts.md new file mode 100644 index 0000000000..ec2ac7dec7 --- /dev/null +++ b/2026/day-15/day-15-networking-concepts.md @@ -0,0 +1,106 @@ +# Day 15 - Networking Concepts: DNS, IP, Subnets & Ports + +## Objective +To build on yesterday's networking fundamentals by researching, understanding, and documenting the core building blocks of networking (DNS, IPs, CIDR, and Ports) from a DevOps perspective. Below are my findings and observations from today's tasks. + +--- + +## Task 1: DNS - How Names Become IPs + +**What happens when you type google.com in a browser?** + +Based on my research, computers only understand IP addresses. When I type google.com, my computer queries a DNS resolver. This resolver acts like a phonebook, asking Root servers, TLD servers, and Authoritative Name Servers to find the exact IP address mapped to google.com. Once the IP is returned, my browser connects to that server to load the page. + +**DNS Record Types (One-line definitions):** +* **A:** Maps a domain name directly to an IPv4 address. +* **AAAA:** Maps a domain name directly to an IPv6 address. +* **CNAME:** Acts as an alias, pointing one domain name to another domain name. +* **MX:** Directs incoming emails to the correct mail server for the domain. +* **NS:** Indicates which Name Servers hold the actual DNS records for the domain. + +**My Observation (dig command):** + +When I ran `dig google.com` in my terminal, I checked the "ANSWER SECTION". I observed the A record which displayed the exact IPv4 address my system was going to connect to. I also noticed the TTL (Time To Live) number, which tells my system how many seconds to cache this IP before looking it up again. + +--- + +## Task 2: IP Addressing + +**What is an IPv4 address and how is it structured?** + +My understanding is that an IPv4 address is a 32-bit unique identifier for a device on a network. It is structured into four sections called octets, separated by dots (for example, 192.168.1.10), where each number ranges from 0 to 255. + +**Public vs Private IPs:** +* **Public IP:** These are globally unique and can be accessed directly over the internet. (Example: Google's DNS at 8.8.8.8) +* **Private IP:** These are restricted to local networks (LAN) and cannot be routed on the public internet. They allow internal devices to talk securely. (Example: My home router assigning 192.168.1.5 to my laptop) + +**Private IP Ranges:** +* 10.0.0.0 to 10.255.255.255 (10.x.x.x) +* 172.16.0.0 to 172.31.255.255 +* 192.168.0.0 to 192.168.255.255 + +**My Observation (ip addr show):** + +When I ran `ip addr show` on my AWS EC2 instance, I identified my private IPs. My main network interface (`enX0`) was assigned 172.31.29.81, which falls into the 172.16-31.x private range. I also noticed a `docker0` interface using 172.17.0.1, showing how Docker sets up its own internal private network. + +--- + +## Task 3: CIDR & Subnetting + +**What does /24 mean in 192.168.1.0/24?** + +The /24 is a CIDR notation. It means that the first 24 bits of the 32-bit address are locked to identify the network itself, leaving the remaining 8 bits to be assigned as addresses for individual hosts (devices) in that network. + +**Why do we subnet?** + +From my learning today, we subnet to take one massive network and break it down into smaller, logical chunks. This prevents network congestion (by isolating broadcast traffic), improves security by separating different types of servers, and makes network administration much easier. + +**CIDR Subnetting Exercise Table:** + +| CIDR | Subnet Mask | Total IPs | Usable Hosts | +| :--- | :-------------- | :-------- | :----------- | +| /24 | 255.255.255.0 | 256 | 254 | +| /16 | 255.255.0.0 | 65536 | 65534 | +| /28 | 255.255.255.240 | 16 | 14 | + +*(Note: Usable hosts are always Total IPs minus 2, because the first IP is reserved for the Network ID and the last IP is reserved for the Broadcast ID).* + +--- + +## Task 4: Ports - The Doors to Services + +**What is a port and why do we need them?** + +If an IP address is the main address of a large office building, a port is the specific room number inside that building. We need ports so that a single server can run multiple services (like a web server and a database) simultaneously without the incoming traffic getting mixed up. + +**Common Ports Documented:** +* **Port 22:** SSH (Secure Shell for remote terminal access) +* **Port 80:** HTTP (Unencrypted web traffic) +* **Port 443:** HTTPS (Encrypted web traffic) +* **Port 53:** DNS (Domain Name resolution) +* **Port 3306:** MySQL / MariaDB (Relational Database) +* **Port 6379:** Redis (In-memory caching) +* **Port 27017:** MongoDB (NoSQL Database) + +**My Observation (ss -tulpn):** + +When I ran `sudo ss -tulpn`, I was able to match listening ports to their actual services on my server. I observed that Port 80 was actively listening and mapped to the `nginx` process, while Port 22 was listening and mapped to the `sshd` process. + +--- + +## Task 5: Putting It Together + +**Scenario 1: You run curl http://myapp.com:8080 β€” what networking concepts from today are involved?** + +First, DNS translates myapp.com into an IP address. The network layer routes the traffic to that specific IP, and then the transport layer targets Port 8080 to deliver the HTTP request directly to the application running on that port. + +**Scenario 2: Your app cannot reach a database at 10.0.1.50:3306 β€” what would you check first?** + +Since 10.0.1.50 is a private IP, I would first check network reachability by running `ping 10.0.1.50`. If the ping works, I would then check if the specific port is accessible by running a tool like `nc -zv 10.0.1.50 3306` to ensure a firewall or security group isn't blocking the connection. + +--- + +## What I Learned (3 Key Points) +1. DNS is the backbone of usability on the internet; without it, we would have to memorize complex public IP addresses for every service. +2. Understanding CIDR math is crucial for cloud engineering, especially when planning VPCs, because you must account for the 2 reserved IPs (Network and Broadcast) when sizing subnets. +3. Troubleshooting requires a layered approach: verifying the IP (reachability) before verifying the port (service availability). diff --git a/2026/day-16/day-16-shell-scripting.md b/2026/day-16/day-16-shell-scripting.md new file mode 100644 index 0000000000..24a661b919 --- /dev/null +++ b/2026/day-16/day-16-shell-scripting.md @@ -0,0 +1,139 @@ +# Day 16 - Shell Scripting Basics + +## Objective +Today, I started my journey into Shell Scripting. The goal was to understand the fundamentals like shebangs, variables, user inputs, and conditional logic to automate daily Linux tasks. Below is the documentation of the scripts I wrote and my observations. + +--- + +## Task 1: Your First Script + +**Code (hello.sh):** +```bash +#!/bin/bash +echo "Hello, DevOps!" +``` + +**Observation:** + +I created the script and used chmod +x hello.sh to give it execution rights, then ran it using ./hello.sh. + + +**What happens if you remove the shebang line (#!/bin/bash)?** + +If I remove the shebang, the script will execute using my current default shell. While it might still run, this is a bad practice. The shebang explicitly tells the operating system which interpreter to use (Bash), ensuring the script behaves consistently regardless of who runs it or what their default shell is. + +--- + +## Task 2: Variables + +**Code (variables.sh):** +```bash +#!/bin/bash +NAME="Rashid" +ROLE="DevOps Engineer" + +echo "--- Double Quotes Test ---" +echo "Hello, I am $NAME and I am a $ROLE" + +echo "--- Single Quotes Test ---" +echo 'Hello, I am $NAME and I am a $ROLE' +``` + +**Observation:** + +I learned that Bash is very strict about spaces during variable assignment; NAME="Rashid" works, but NAME = "Rashid" throws an error. I also observed the difference between quotes: double quotes evaluate the variables and print their actual values, whereas single quotes treat the variable names as literal strings. + +![Task 2 Output](task-2-variables.png) + +--- + +## Task 3: User Input with read + +**Code (greet.sh):** +```bash +#!/bin/bash +read -p "Enter your name: " name +read -p "Enter your favourite DevOps tool: " tool + +echo "Hello $name, your favourite tool is $tool!" +``` + +**Observation:** + +Using the read -p command allowed me to make the script interactive. Instead of hardcoding values, the script paused, prompted me for input, stored my response into a variable, and printed it dynamically. + +![Task 3 Output](task-3-greet.png) + +--- + +## Task 4: If-Else Conditions + +### Script 1: check_number.sh +**Code:** +```bash +#!/bin/bash +read -p "Enter a number: " num + +if [ $num -gt 0 ]; then + echo "The number is positive." +elif [ $num -lt 0 ]; then + echo "The number is negative." +else + echo "The number is zero." +fi +``` + +![Check Number Output](task-4-check-number.png) + +### Script 2: file_check.sh +**Code:** +```bash +#!/bin/bash +read -p "Enter a filename to check: " filename + +if [ -f $filename ]; then + echo "Yes, the file '$filename' exists!" +else + echo "No, the file '$filename' does not exist." +fi +``` + +![File Check Output](task-4-file-check.png) + +**Observation:** + +I learned how to use square brackets [ ] for conditional checks and understood that spaces inside the brackets are mandatory in Bash. I used -gt (greater than) and -lt (less than) for integer comparisons, and the -f flag to verify if a specific file exists on the filesystem. + +--- + +## Task 5: Combine It All (Real-world Scenario) + +**Code (server_check.sh):** +```bash +#!/bin/bash +SERVICE="sshd" + +read -p "Do you want to check the status of $SERVICE? (y/n): " choice + +if [ "$choice" == "y" ]; then + echo "Checking status for $SERVICE..." + systemctl status $SERVICE --no-pager +elif [ "$choice" == "n" ]; then + echo "Skipped." +else + echo "Invalid input. Please enter 'y' or 'n'." +fi +``` + +**Observation:** + +I combined variables, user input, and if-else logic to create a functional server status checking script. By adding --no-pager to the systemctl status command, I ensured the output prints directly to the terminal without freezing the screen in an interactive pager mode. + +![Server Check Output](task-5-server-check.png) + +--- + +## What I Learned (3 Key Points) +1. **Automation starts with basic logic:** Writing repetitive terminal commands into a single .sh file saves time and reduces manual typing errors. +2. **Quotes matter in Bash:** Double quotes resolve variable values, while single quotes read everything as raw text. Knowing when to use which is crucial for debugging. +3. **Execution requires explicit permission:** By default, Linux prevents new files from running to ensure security. Using chmod +x is mandatory before executing any custom script. diff --git a/2026/day-16/task-2-variables.png b/2026/day-16/task-2-variables.png new file mode 100644 index 0000000000..2b53f98d34 Binary files /dev/null and b/2026/day-16/task-2-variables.png differ diff --git a/2026/day-16/task-3-greet.png b/2026/day-16/task-3-greet.png new file mode 100644 index 0000000000..c3e9207feb Binary files /dev/null and b/2026/day-16/task-3-greet.png differ diff --git a/2026/day-16/task-4-check-number.png b/2026/day-16/task-4-check-number.png new file mode 100644 index 0000000000..62cdd2ec5e Binary files /dev/null and b/2026/day-16/task-4-check-number.png differ diff --git a/2026/day-16/task-4-file-check.png b/2026/day-16/task-4-file-check.png new file mode 100644 index 0000000000..be34a181d4 Binary files /dev/null and b/2026/day-16/task-4-file-check.png differ diff --git a/2026/day-16/task-5-server-check.png b/2026/day-16/task-5-server-check.png new file mode 100644 index 0000000000..6a6ad7bbce Binary files /dev/null and b/2026/day-16/task-5-server-check.png differ diff --git a/2026/day-17/day-17-scripting.md b/2026/day-17/day-17-scripting.md new file mode 100644 index 0000000000..e51801331e --- /dev/null +++ b/2026/day-17/day-17-scripting.md @@ -0,0 +1,184 @@ +# Day 17 - Shell Scripting: Loops, Arguments & Error Handling + +This repository contains scripts covering dynamic iteration, command-line argument passing, package management, and safe execution practices in Bash. + +--- + +## Task 1: For Loop + +### 1. Iterating through a list (for_loop.sh) +```bash +#!/bin/bash +# Looping through a list of 5 fruits + +echo "--- Fruit List ---" +for fruit in Apple Mango Orange Banana Cherry; do + echo "I like $fruit" +done +``` + +**Output Proof:** +![For Loop Execution](task-1-for-loop.png) + +### 2. Iterating through numbers using C-style loop (count.sh) +```bash +#!/bin/bash +# Counting numbers from 1 to 10 using C-style for loop + +echo "--- Counting Numbers ---" +for (( i=1; i<=10; i++ )); do + echo "Number: $i" +done +``` + +**Output Proof:** +![Count Execution](task-1-count.png) + +--- + +## Task 2: While Loop + +### Reverse Countdown Timer (countdown.sh) +```bash +#!/bin/bash +# Countdown timer using while loop + +read -p "Enter a number to start countdown: " num + +while [ $num -ge 0 ]; do + echo "Countdown: $num" + num=$((num - 1)) + sleep 1 +done + +echo "Done!" +``` + +**Output Proof:** +![Countdown Execution](task-2-countdown.png) + +--- + +## Task 3: Command-Line Arguments + +### 1. Handling Positional Parameters (greet.sh) +```bash +#!/bin/bash +# Script to demonstrate positional parameter ($1) + +if [ -z "$1" ]; then + echo "Usage: ./greet.sh " +else + echo "Hello, $1!" +fi +``` + +**Output Proof:** +![Greet Execution](task-3-greet.png) + +### 2. Handling Special Variables (args_demo.sh) +```bash +#!/bin/bash +# Script to demonstrate special variables ($0, $#, $@) + +echo "Script Name (\$0): $0" +echo "Total Arguments passed (\$#): $#" +echo "All Arguments list (\$@): $@" +``` + +**Output Proof:** +![Args Demo Execution](task-3-args-demo.png) + +--- + +## Task 4: Install Packages via Script (With Custom Uninstall Logic) + +### The Master Installer (install_packages.sh) +This script checks for required packages, installs them automatically if missing, and includes a custom logic block to optionally uninstall them based on user input. It also includes a root privilege check. + +```bash +#!/bin/bash +# Master Script to Install/Uninstall Packages + +# 1. ROOT CHECK +if [ "$EUID" -ne 0 ]; then + echo "Error: Please run this script as root (use: sudo ./install_packages.sh)" + exit 1 +fi + +PACKAGES="nginx curl wget docker.io python3" + +# 2. LOOP THROUGH EACH PACKAGE +for pkg in $PACKAGES; do + echo "----------------------------------------" + echo "Checking package: $pkg" + + PACKAGE_READY=false + + # 3. CHECK IF INSTALLED + if dpkg -s "$pkg" &> /dev/null; then + echo "[+] $pkg is already installed." + PACKAGE_READY=true + else + echo "[-] $pkg is missing. Installing now..." + if apt-get install -y "$pkg" &> /dev/null; then + echo "[+] $pkg installed successfully." + PACKAGE_READY=true + else + echo "[!] Failed to install $pkg." + fi + fi + + # 4. CUSTOM UNINSTALL LOGIC + if [ "$PACKAGE_READY" = true ]; then + read -p "Do you want to UNINSTALL $pkg? (yes/no): " choice + if [ "$choice" == "yes" ]; then + echo "Uninstalling $pkg..." + apt-get remove -y "$pkg" &> /dev/null + echo "[-] $pkg has been completely uninstalled." + else + echo "[*] Keeping $pkg as is." + fi + fi +done + +echo "----------------------------------------" +echo "Script execution completed!" +``` + +**Output Proof:** +![Master Installer Execution](task-4-installer.png) + +--- + +## Task 5: Error Handling + +### Safe Execution with set -e (safe_script.sh) +```bash +#!/bin/bash +# Script to demonstrate error handling with set -e and || + +set -e + +echo "Step 1: Creating directory /tmp/devops-test..." +mkdir /tmp/devops-test || echo "[-] Directory already exists, moving forward..." + +echo "Step 2: Navigating into the directory..." +cd /tmp/devops-test + +echo "Step 3: Creating a test file inside it..." +touch testfile.txt + +echo "-------------------------------------------" +echo "Success: script executed completely and safely!" +``` + +**Output Proof:** +![Safe Script Execution](task-5-safe-script.png) + +--- + +## Key Learnings +1. **Dynamic Execution with Arguments:** Using `$1`, `$@`, and `$#` eliminates the need to hardcode values or rely entirely on read prompts, making scripts highly reusable for CI/CD pipelines. +2. **Robust Error Handling:** Implementing `set -e` prevents a script from cascading failures. Using the Logical OR `||` acts as a crucial safety net to handle expected non-fatal errors (like creating a directory that already exists). +3. **Advanced Flow Control:** Combining for loops with package managers (like apt-get or dpkg) allows for rapid scaling of infrastructure setup. Adding automated root checks ($EUID) prevents permission-based runtime crashes. diff --git a/2026/day-17/task-1-count.png b/2026/day-17/task-1-count.png new file mode 100644 index 0000000000..209be283ec Binary files /dev/null and b/2026/day-17/task-1-count.png differ diff --git a/2026/day-17/task-1-for-loop.png b/2026/day-17/task-1-for-loop.png new file mode 100644 index 0000000000..ce50f98a06 Binary files /dev/null and b/2026/day-17/task-1-for-loop.png differ diff --git a/2026/day-17/task-2-countdown.png b/2026/day-17/task-2-countdown.png new file mode 100644 index 0000000000..6427c8b3b7 Binary files /dev/null and b/2026/day-17/task-2-countdown.png differ diff --git a/2026/day-17/task-3-args-demo.png b/2026/day-17/task-3-args-demo.png new file mode 100644 index 0000000000..2f203f230a Binary files /dev/null and b/2026/day-17/task-3-args-demo.png differ diff --git a/2026/day-17/task-3-greet.png b/2026/day-17/task-3-greet.png new file mode 100644 index 0000000000..01db0aaf62 Binary files /dev/null and b/2026/day-17/task-3-greet.png differ diff --git a/2026/day-17/task-4-installer.png b/2026/day-17/task-4-installer.png new file mode 100644 index 0000000000..02482feb68 Binary files /dev/null and b/2026/day-17/task-4-installer.png differ diff --git a/2026/day-17/task-5-safe-script.png b/2026/day-17/task-5-safe-script.png new file mode 100644 index 0000000000..f4e624ee78 Binary files /dev/null and b/2026/day-17/task-5-safe-script.png differ diff --git a/2026/day-18/day-18-scripting.md b/2026/day-18/day-18-scripting.md new file mode 100644 index 0000000000..e3949dfdb0 --- /dev/null +++ b/2026/day-18/day-18-scripting.md @@ -0,0 +1,275 @@ +# Day 18: Shell Scripting - Functions & Intermediate Concepts + +## Objective +To write cleaner, reusable, and production-ready scripts by mastering bash functions, understanding variable scopes (local vs global), and enforcing strict execution modes to prevent catastrophic failures. + +--- + +## Task 1: Basic Functions +**Goal:** Create a script with functions that accept arguments. + +**Code:** +```bash +#!/bin/bash + +greet() { + echo "Hello, $1!" +} + +add() { + local sum=$(( $1 + $2 )) + echo "The sum of $1 and $2 is: $sum" +} + +echo "--- Testing Greet Function ---" +greet "Shubham" +greet "Rashid" + +echo -e "\n--- Testing Add Function ---" +add 10 20 +add 50 75 +``` + +**Output Proof:** +![Task 1 Output](./task-1-functions.png) + +--- + +## Task 2: Functions with Return Values +**Goal:** Use command substitution to capture the output of functions into variables. + +**Code:** +```bash +#!/bin/bash + +check_disk() { + df -h / +} + +check_memory() { + free -h +} + +echo "Fetching system information..." + +DISK_RESULT=$(check_disk) +MEMORY_RESULT=$(check_memory) + +echo -e "\n--- Disk Usage Result ---" +echo "$DISK_RESULT" + +echo -e "\n--- Memory Usage Result ---" +echo "$MEMORY_RESULT" +``` + +**Output Proof:** +![Task 2 Output](./task-2-disk-check.png) + +--- + +## Task 3: Strict Mode (set -euo pipefail) +**Goal:** Demonstrate how the fail-fast mechanism protects servers from executing dangerous commands after an error occurs. + +**Documentation: What does each flag do?** +- **set -e**: Exit immediately if a command exits with a non-zero status (Fail fast). +- **set -u**: Treat unset variables as an error and exit immediately. Prevents catastrophic empty variable expansions. +- **set -o pipefail**: Ensures a piped command fails if any part of it fails, rather than just looking at the last command. + +**Code:** +```bash +#!/bin/bash +set -euo pipefail + +echo "==================================" +echo " Strict Mode Demonstration " +echo "==================================" + +echo "Step 1: Doing some normal work..." +echo "Working fine..." + +echo -e "\nStep 2: Trying to access a fake directory (This will FAIL)" +ls /this_fake_folder_does_not_exist + +# Script dies above. Below line will never execute. +echo -e "\nStep 3: This line will NEVER print because script died at Step 2!" +``` + +**Output Proof:** +![Task 3 Output](./task-3-strict-mode.png) + +--- + +## Task 4: Local Variables +**Goal:** Understand variable scoping and prevent function variables from leaking into the global environment. + +**Code:** +```bash +#!/bin/bash + +MY_VAR="I am the ORIGINAL GLOBAL variable" + +good_function() { + local MY_VAR="I am a STRICTLY LOCAL variable" + echo "Inside good_function: $MY_VAR" +} + +bad_function() { + MY_VAR="Oh no! I OVERWROTE the global variable!" + echo "Inside bad_function: $MY_VAR" +} + +echo "--- Initial State ---" +echo "Current Value: $MY_VAR" + +echo -e "\n--- Testing good_function (Safe) ---" +good_function +echo "Value outside: $MY_VAR" + +echo -e "\n--- Testing bad_function (Danger) ---" +bad_function +echo "Value outside: $MY_VAR" +``` + +**Output Proof:** +![Task 4 Output](./task-4-local-vars.png) + +--- + +## Task 5: System Info Reporter +**Goal:** Build a comprehensive health check script utilizing functions, piped commands, and strict mode. + +**Code:** +```bash +#!/bin/bash +set -euo pipefail + +print_os_info() { + echo "Hostname: $(hostname)" + local os_name=$(cat /etc/os-release | grep "^PRETTY_NAME=" | cut -d '=' -f 2 | tr -d '"' || true) + echo "OS Release: $os_name" +} + +print_uptime() { + echo "Uptime: $(uptime -p)" +} + +print_disk_usage() { + echo "Filesystem Size Used Avail Use% Mounted on" + df -h | sort -hr | head -n 6 +} + +print_memory_usage() { + free -m | awk 'NR==1{printf "%-15s %-15s %-15s\n", "Total(MB)", "Used(MB)", "Free(MB)"} NR==2{printf "%-15s %-15s %-15s\n", $2, $3, $4}' +} + +print_top_cpu_processes() { + ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head -n 6 +} + +main() { + echo "==============================================" + echo " SERVER HEALTH REPORT - $(date '+%F') " + echo "==============================================" + + echo -e "\n[1] SYSTEM INFORMATION" + echo "----------------------------------------------" + print_os_info + print_uptime + + echo -e "\n[2] MEMORY USAGE" + echo "----------------------------------------------" + print_memory_usage + + echo -e "\n[3] DISK USAGE (TOP 5)" + echo "----------------------------------------------" + print_disk_usage + + echo -e "\n[4] TOP 5 CPU CONSUMING PROCESSES" + echo "----------------------------------------------" + print_top_cpu_processes + + echo -e "\n==============================================" +} + +main +``` + +**Output Proof:** +![Task 5 Output](./task-5-system-info.png) + +--- + +## Task 6: Custom Interactive Calculator (Feature Addition) +**Goal:** Build a custom interactive calculator using mathematical functions, case statements for menu selection, input validation (divide by zero), and an infinite loop for continuous operation. + +**Code:** +```bash +#!/bin/bash + +add() { + echo $(( $1 + $2 )) +} + +subtract() { + echo $(( $1 - $2 )) +} + +multiply() { + echo $(( $1 * $2 )) +} + +divide() { + if [ $2 -eq 0 ]; then + echo "Error: Cannot divide by zero!" + else + echo $(( $1 / $2 )) + fi +} + +main_calculator() { + echo -e "\n=============================" + echo " DevOps Master Calculator " + echo "=============================" + echo "1. Add (+)" + echo "2. Subtract (-)" + echo "3. Multiply (*)" + echo "4. Divide (/)" + + read -p "Choose an operation (1/2/3/4): " choice + read -p "Enter first number: " num1 + read -p "Enter second number: " num2 + + echo "-----------------------------" + echo -n "Result: " + + case $choice in + 1) add $num1 $num2 ;; + 2) subtract $num1 $num2 ;; + 3) multiply $num1 $num2 ;; + 4) divide $num1 $num2 ;; + *) echo "Invalid choice! Please select 1, 2, 3, or 4." ;; + esac +} + +while true; do + main_calculator + + echo "-----------------------------" + read -p "Calculate again? (yes/no): " play_again + + if [ "$play_again" != "yes" ]; then + echo "Bye! Thanks for using the calculator." + break + fi +done +``` + +**Output Proof:** +![Task 6 Output](./task-6-calculator.png) + +--- + +## What I Learned (3 Key Points) +1. **Scope Management is Critical:** The local keyword is non-negotiable in large scripts to prevent function variables from accidentally modifying global states. +2. **Bash Returns Outputs, Not Values:** Unlike other languages, Bash functions return exit codes. To retrieve data from a function, command substitution `$()` must be used to capture its printed output. +3. **Strict Mode Prevents Disasters:** Implementing `set -euo pipefail` enforces a fail-fast architecture, ensuring scripts immediately halt upon encountering undefined variables, command failures, or hidden pipe errors. diff --git a/2026/day-18/task-1-functions.png b/2026/day-18/task-1-functions.png new file mode 100644 index 0000000000..9bc877f14f Binary files /dev/null and b/2026/day-18/task-1-functions.png differ diff --git a/2026/day-18/task-2-disk-check.png b/2026/day-18/task-2-disk-check.png new file mode 100644 index 0000000000..468177bd66 Binary files /dev/null and b/2026/day-18/task-2-disk-check.png differ diff --git a/2026/day-18/task-3-strict-mode.png b/2026/day-18/task-3-strict-mode.png new file mode 100644 index 0000000000..a9baf21619 Binary files /dev/null and b/2026/day-18/task-3-strict-mode.png differ diff --git a/2026/day-18/task-4-local-vars.png b/2026/day-18/task-4-local-vars.png new file mode 100644 index 0000000000..fd95219574 Binary files /dev/null and b/2026/day-18/task-4-local-vars.png differ diff --git a/2026/day-18/task-5-system-info.png b/2026/day-18/task-5-system-info.png new file mode 100644 index 0000000000..c4cd1ae697 Binary files /dev/null and b/2026/day-18/task-5-system-info.png differ diff --git a/2026/day-18/task-6-calculator.png b/2026/day-18/task-6-calculator.png new file mode 100644 index 0000000000..2adba10afd Binary files /dev/null and b/2026/day-18/task-6-calculator.png differ diff --git a/2026/day-19/day-19-project.md b/2026/day-19/day-19-project.md new file mode 100644 index 0000000000..ea2aed450e --- /dev/null +++ b/2026/day-19/day-19-project.md @@ -0,0 +1,186 @@ +# Day 19: Shell Scripting Project - Log Rotation, Backup & Crontab + +## Objective +Apply concepts from previous days (loops, arguments, strict mode, conditionals) to build real-world, production-ready DevOps mini-projects. + +--- + +## Task 1: Log Rotation Script +**Goal:** Create a script to compress old logs and delete very old archives to save disk space. It must validate inputs and count affected files. + +**Code (log_rotate.sh):** +```bash +#!/bin/bash +set -euo pipefail + +# Requirement: Exits with an error if the directory doesn't exist / Not provided +if [ $# -eq 0 ]; then + echo "Error: Please provide a log directory as an argument." + exit 1 +fi + +LOG_DIR=$1 + +if [ ! -d "$LOG_DIR" ]; then + echo "Error: Directory '$LOG_DIR' does not exist!" + exit 1 +fi + +# Requirement: Count files to be compressed +COMPRESS_COUNT=$(find "$LOG_DIR" -name "*.log" -type f -mtime +7 | wc -l) + +# Requirement: Compresses .log files older than 7 days using gzip +if [ "$COMPRESS_COUNT" -gt 0 ]; then + find "$LOG_DIR" -name "*.log" -type f -mtime +7 -exec gzip {} \; +fi + +# Requirement: Count files to be deleted +DELETE_COUNT=$(find "$LOG_DIR" -name "*.gz" -type f -mtime +30 | wc -l) + +# Requirement: Deletes .gz files older than 30 days +if [ "$DELETE_COUNT" -gt 0 ]; then + find "$LOG_DIR" -name "*.gz" -type f -mtime +30 -exec rm {} \; +fi + +# Requirement: Prints how many files were compressed and deleted +echo "-----------------------------------" +echo "Files Compressed (Older than 7 days): $COMPRESS_COUNT" +echo "Files Deleted (Older than 30 days): $DELETE_COUNT" +echo "-----------------------------------" +``` + +**Output Proof:** +![Task 1 Output](./task-1-log-rotate.png) + +--- + +## Task 2: Server Backup Script +**Goal:** Create a robust backup script that handles input arguments safely, creates compressed tarballs, verifies creation, and cleans up old backups. + +**Code (backup.sh):** +```bash +#!/bin/bash +set -euo pipefail + +# Validation: Takes a source directory and backup destination as arguments +if [ $# -ne 2 ]; then + echo "Error: Missing arguments." + echo "Usage: ./backup.sh " + exit 1 +fi + +SOURCE_DIR=$1 +BACKUP_DEST=$2 + +# Handles errors β€” exit if source doesn't exist +if [ ! -d "$SOURCE_DIR" ]; then + echo "Error: Source directory '$SOURCE_DIR' does not exist! Backup aborted." + exit 1 +fi + +# Creates a timestamped .tar.gz archive +TIMESTAMP=$(date +%Y-%m-%d) +ARCHIVE_NAME="backup-${TIMESTAMP}.tar.gz" +ARCHIVE_PATH="${BACKUP_DEST}/${ARCHIVE_NAME}" + +# Create the tar archive (Suppressing absolute path warnings using 2>/dev/null) +tar -czf "$ARCHIVE_PATH" "$SOURCE_DIR" 2>/dev/null + +# Verifies the archive was created successfully & Prints name and size +if [ -f "$ARCHIVE_PATH" ]; then + ARCHIVE_SIZE=$(du -sh "$ARCHIVE_PATH" | awk '{print $1}') + echo "Archive Name: $ARCHIVE_NAME" + echo "Archive Size: $ARCHIVE_SIZE" +else + exit 1 +fi + +# Deletes backups older than 14 days from the destination +find "$BACKUP_DEST" -name "backup-*.tar.gz" -type f -mtime +14 -exec rm {} \; +``` + +**Output Proof:** +![Task 2 Output](./task-2-backup.png) + +--- + +## Task 3: Crontab +**Goal:** Understand cron syntax and schedule the scripts built in earlier tasks. + +**Cron Syntax:** +```text +* * * * * command +β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ └── Day of week (0-7) +β”‚ β”‚ β”‚ └──── Month (1-12) +β”‚ β”‚ └────── Day of month (1-31) +β”‚ └──────── Hour (0-23) +└────────── Minute (0-59) +``` + +**Written Cron Entries:** + +Run log_rotate.sh every day at 2 AM: +```bash +0 2 * * * /home/ubuntu/90DaysOfDevOpsChallengeRashidKhan/2026/day-19/log_rotate.sh /home/ubuntu/test_logs +``` + +Run backup.sh every Sunday at 3 AM: +```bash +0 3 * * 0 /home/ubuntu/90DaysOfDevOpsChallengeRashidKhan/2026/day-19/backup.sh /home/ubuntu/test_logs /home/ubuntu/my_backups +``` + +Run a health check script every 5 minutes: +```bash +*/5 * * * * /home/ubuntu/90DaysOfDevOpsChallengeRashidKhan/2026/day-19/health_check.sh +``` + +--- + +## Task 4: Scheduled Maintenance Script +**Goal:** Combine individual scripts into a master orchestrator script that runs daily, executes tasks sequentially, and logs all outputs properly. + +**Code (maintenance.sh):** +```bash +#!/bin/bash +set -euo pipefail + +BASE_DIR="/home/ubuntu/90DaysOfDevOpsChallengeRashidKhan/2026/day-19" +LOG_DIR="/home/ubuntu/test_logs" +BACKUP_DEST="/home/ubuntu/my_backups" + +MAINTENANCE_LOG="$BASE_DIR/maintenance.log" + +log_message() { + echo "$(date '+%Y-%m-%d %H:%M:%S'): $1" >> "$MAINTENANCE_LOG" +} + +log_message "=== MAINTENANCE STARTED ===" + +# Triggering Log Rotation +log_message "Triggering Log Rotation..." +$BASE_DIR/log_rotate.sh "$LOG_DIR" >> "$MAINTENANCE_LOG" 2>&1 || log_message "ERROR: Log Rotation Failed!" + +# Triggering Server Backup +log_message "Triggering Server Backup..." +$BASE_DIR/backup.sh "$LOG_DIR" "$BACKUP_DEST" >> "$MAINTENANCE_LOG" 2>&1 || log_message "ERROR: Server Backup Failed!" + +log_message "=== MAINTENANCE COMPLETED ===" +``` + +**Cron Entry to run Maintenance Script daily at 1 AM:** +```bash +0 1 * * * /home/ubuntu/90DaysOfDevOpsChallengeRashidKhan/2026/day-19/maintenance.sh +``` + +**Output Proof:** +![Task 4 Output](./task-4-maintenance.png) + +--- + +## What I Learned +1. **Scope and Strict Requirements:** Writing DevOps scripts means sticking strictly to the requested behavior. Handled conditional checking properly to ensure the scripts fail safely if source directories do not exist. +2. **Absolute Paths are King in Cron:** Cron runs in a headless environment without standard user profiles. Using absolute paths for scripts, source folders, and log files is mandatory for successful automation. +3. **Redirection for Observability:** Mastered the redirection operators to ensure background orchestrator scripts operate silently while capturing a full timeline in a dedicated log file. +4. **Mastering the 'find' Command:** Gained deep hands-on experience using `find` with time-based flags (`-mtime +7`, `-mtime +30`) and dynamically executing commands on the results (`-exec gzip {} \;`). This is the backbone of automated file management. +5. **Handling Silent Failures:** Realized the true power of `set -euo pipefail`. Encountered real-world permission issues when testing on system folders and saw firsthand how Strict Mode immediately stops the script instead of causing silent cascading failures. diff --git a/2026/day-19/task-1-log-rotate.png b/2026/day-19/task-1-log-rotate.png new file mode 100644 index 0000000000..735fdf4a35 Binary files /dev/null and b/2026/day-19/task-1-log-rotate.png differ diff --git a/2026/day-19/task-2-backup.png b/2026/day-19/task-2-backup.png new file mode 100644 index 0000000000..3cd2c77056 Binary files /dev/null and b/2026/day-19/task-2-backup.png differ diff --git a/2026/day-19/task-4-maintenance.png b/2026/day-19/task-4-maintenance.png new file mode 100644 index 0000000000..efdbbf4750 Binary files /dev/null and b/2026/day-19/task-4-maintenance.png differ diff --git a/2026/day-20/day-20-solution.md b/2026/day-20/day-20-solution.md new file mode 100644 index 0000000000..18d593c652 --- /dev/null +++ b/2026/day-20/day-20-solution.md @@ -0,0 +1,234 @@ +# Day 20: Bash Scripting Challenge - Log Analyzer and Report Generator + +## Objective +Build a Bash script to automate the process of analyzing server log files, extracting critical events, counting errors, identifying the top 5 error messages, generating a formatted daily summary report, and archiving the processed log files. + +--- + +## Pre-requisite: Generating the Sample Log +Before analyzing, we need raw log data. We used the provided generator script to create a dummy log file with 1000 lines of randomized logs. + +**Script (sample_logs_generator.sh):** +```bash +#!/bin/bash + +if [ "$#" -ne 2 ]; then + echo "Usage: $0 " + exit 1 +fi + +log_file_path="$1" +num_lines="$2" + +if [ -e "$log_file_path" ]; then + echo "Error: File already exists at $log_file_path." + exit 1 +fi + +log_levels=("INFO" "DEBUG" "ERROR" "WARNING" "CRITICAL") +error_messages=("Failed to connect" "Disk full" "Segmentation fault" "Invalid input" "Out of memory") + +generate_log_line() { + local log_level="${log_levels[$((RANDOM % ${#log_levels[@]}))]}" + local error_msg="" + if [ "$log_level" == "ERROR" ]; then + error_msg="${error_messages[$((RANDOM % ${#error_messages[@]}))]}" + fi + echo "$(date '+%Y-%m-%d %H:%M:%S') [$log_level] $error_msg - $RANDOM" +} + +touch "$log_file_path" +for ((i=0; i> "$log_file_path" +done + +echo "Log file created at: $log_file_path with $num_lines lines." +``` + +--- + +## Step-by-Step Execution & Output Proofs + +### Task 1: Input and Validation +**Goal:** Accept the path to a log file as a command-line argument. Exit with a clear error message if no argument is provided or if the file does not exist. + +**Code Snippet:** +```bash +if [ $# -eq 0 ]; then + echo "Error: No log file provided. Usage: ./log_analyzer.sh " + exit 1 +fi + +LOG_FILE=$1 + +if [ ! -f "$LOG_FILE" ]; then + echo "Error: Log file '$LOG_FILE' does not exist." + exit 1 +fi +``` +**Output Proof:** +![Task 1 Validation](./task-1-validation.png) + + +### Task 2: Error Count +**Goal:** Count the total number of lines containing the keyword ERROR or Failed and print the count. + +**Code Snippet:** +```bash +ERROR_COUNT=$(grep -E -c "ERROR|Failed" "$LOG_FILE") +echo "Total error count: $ERROR_COUNT" +``` +**Output Proof:** +![Task 2 Error Count](./task-2-error-count.png) + + +### Task 3: Critical Events +**Goal:** Search for lines containing the keyword CRITICAL and print those lines along with their exact line number. + +**Code Snippet:** +```bash +echo "--- Critical Events ---" +grep -n "CRITICAL" "$LOG_FILE" | sed 's/^\([0-9]*\):/Line \1: /' +``` +**Output Proof:** +![Task 3 Critical Events](./task-3-critical-events.png) + + +### Task 4: Top 5 Error Messages +**Goal:** Extract all lines containing ERROR, clean the text, identify the top 5 most common error messages, and display them with their occurrence count in descending order. + +**Code Snippet:** +```bash +echo "--- Top 5 Error Messages ---" +grep "ERROR" "$LOG_FILE" | awk '{$1=$2=$3=""; print}' | sed 's/ - [0-9]*$//' | sort | uniq -c | sort -rn | head -5 +``` +**Output Proof:** +![Task 4 Top Errors](./task-4-top-errors.png) + + +### Task 5: Summary Report +**Goal:** Generate a summary report to a text file containing all the analyzed data points using simultaneous console and file output. + +**Code Snippet:** +```bash +DATE=$(date +%Y-%m-%d) +REPORT_FILE="log_report_${DATE}.txt" +TOTAL_LINES=$(wc -l < "$LOG_FILE") + +# Wrapping the output commands in {} and piping to tee +{ + echo "Date of analysis: $DATE" + echo "Total lines processed: $TOTAL_LINES" + # ... (other echo statements) ... +} | tee "$REPORT_FILE" +``` +**Output Proof:** +![Task 5 Summary Report](./task-5-summary-report.png) + + +### Task 6 (Optional): Archive Processed Logs +**Goal:** Automatically create an archive folder and move the processed log file into it after successful analysis to keep the workspace clean. + +**Code Snippet:** +```bash +ARCHIVE_DIR="archive" +mkdir -p "$ARCHIVE_DIR" +mv "$LOG_FILE" "$ARCHIVE_DIR/" +echo "Success: Processed log file '$LOG_FILE' moved to '$ARCHIVE_DIR/' directory." +``` +**Output Proof:** +![Task 6 Archive](./task-6-archive.png) + +--- + +## The Complete Master Script (log_analyzer.sh) + +Below is the complete, production-ready script combining all the logic above. + +```bash +#!/bin/bash + +# Task 1: Input and Validation +if [ $# -eq 0 ]; then + echo "Error: No log file provided. Usage: ./log_analyzer.sh " + exit 1 +fi + +LOG_FILE=$1 + +if [ ! -f "$LOG_FILE" ]; then + echo "Error: Log file '$LOG_FILE' does not exist." + exit 1 +fi + +# Variables for Task 5 (Summary Report) +DATE=$(date +%Y-%m-%d) +REPORT_FILE="log_report_${DATE}.txt" +TOTAL_LINES=$(wc -l < "$LOG_FILE") + +# Task 2: Error Count +ERROR_COUNT=$(grep -E -c "ERROR|Failed" "$LOG_FILE") + +# Generate Report and print to console simultaneously using 'tee' +{ + echo "===================================" + echo " Log Analysis Report " + echo "===================================" + echo "Date of analysis: $DATE" + echo "Log file name: $LOG_FILE" + echo "Total lines processed: $TOTAL_LINES" + echo "Total error count: $ERROR_COUNT" + echo "" + + # Task 4: Top 5 Error Messages + echo "--- Top 5 Error Messages ---" + grep "ERROR" "$LOG_FILE" | awk '{$1=$2=$3=""; print}' | sed 's/ - [0-9]*$//' | sort | uniq -c | sort -rn | head -5 + echo "" + + # Task 3: Critical Events + echo "--- Critical Events ---" + grep -n "CRITICAL" "$LOG_FILE" | sed 's/^\([0-9]*\):/Line \1: /' +} | tee "$REPORT_FILE" + +echo "" +echo "Success: Report generated and saved to $REPORT_FILE" + +# Task 6: Archive Processed Logs +ARCHIVE_DIR="archive" +mkdir -p "$ARCHIVE_DIR" +mv "$LOG_FILE" "$ARCHIVE_DIR/" + +echo "Success: Processed log file '$LOG_FILE' moved to '$ARCHIVE_DIR/' directory." +``` + +--- + +## Tools & Commands Used + +**1. grep:** Used with -E for extended regular expressions to search for multiple keywords (ERROR or Failed). Used with -c to count matches and -n to output line numbers. + +**2. awk:** Used to manipulate columns. Specifically, awk '{$1=$2=$3=""; print}' nullified the Date, Time, and Level columns so only the core error message remained. + +**3. sed (Stream Editor):** Used for advanced text replacement. + - To format critical logs: sed 's/^\([0-9]*\):/Line \1: /' + - To clean random trailing numbers from error messages: sed 's/ - [0-9]*$//' + +**4. sort & uniq:** Used in pipeline (sort | uniq -c | sort -rn) to group identical error messages, count their occurrences, and sort them in descending numerical order. + +**5. head:** Used as head -5 to limit the output to only the top 5 most frequent error messages. + +**6. tee:** A highly effective command used to write the report block to standard output (console) and append it to a file simultaneously. + +--- + +## What I Learned (5 Key Points) + +**1. Advanced Data Extraction pipeline:** I learned how to chain commands (grep -> awk -> sed -> sort -> uniq) to take raw, messy log data and distill it into a clean, grouped, and mathematically sorted format. + +**2. Simultaneous Output with Tee:** Instead of writing multiple echo statements with standard redirection (>>), wrapping a block of code in {} and piping it to tee makes generating physical report files incredibly efficient and clean. + +**3. Data Cleaning with Sed:** Real-world logs have dynamic trailing data (like random transaction IDs). I learned that without cleaning that noise using sed regex replacements, grouping and counting with uniq will fail because every line appears strictly unique. + +**4. Handling Dynamic Variables in Bash:** Mastered the use of inline command execution like DATE=$(date +%Y-%m-%d) and TOTAL_LINES=$(wc -l < "$LOG_FILE") to create dynamic file names and extract clean numbers without filename artifacts. + +**5. Building Fail-Safe Architectures:** By implementing strict input validation (checking for arguments and file existence) and creating backup/archive directories dynamically with mkdir -p, I learned how to prevent scripts from failing in production environments. diff --git a/2026/day-20/task-1-validation.png b/2026/day-20/task-1-validation.png new file mode 100644 index 0000000000..1c34ec0fff Binary files /dev/null and b/2026/day-20/task-1-validation.png differ diff --git a/2026/day-20/task-2-error-count.png b/2026/day-20/task-2-error-count.png new file mode 100644 index 0000000000..e7da12dd75 Binary files /dev/null and b/2026/day-20/task-2-error-count.png differ diff --git a/2026/day-20/task-3-critical-events.png b/2026/day-20/task-3-critical-events.png new file mode 100644 index 0000000000..decf7952b6 Binary files /dev/null and b/2026/day-20/task-3-critical-events.png differ diff --git a/2026/day-20/task-4-top-errors.png b/2026/day-20/task-4-top-errors.png new file mode 100644 index 0000000000..2be068b070 Binary files /dev/null and b/2026/day-20/task-4-top-errors.png differ diff --git a/2026/day-20/task-5-summary-report.png b/2026/day-20/task-5-summary-report.png new file mode 100644 index 0000000000..1af4b96f19 Binary files /dev/null and b/2026/day-20/task-5-summary-report.png differ diff --git a/2026/day-20/task-6-archive.png b/2026/day-20/task-6-archive.png new file mode 100644 index 0000000000..774322db85 Binary files /dev/null and b/2026/day-20/task-6-archive.png differ diff --git a/2026/day-21/shell_scripting_cheatsheet.md b/2026/day-21/shell_scripting_cheatsheet.md new file mode 100644 index 0000000000..15e8691e32 --- /dev/null +++ b/2026/day-21/shell_scripting_cheatsheet.md @@ -0,0 +1,384 @@ +# Shell Scripting Cheat Sheet + +## Quick Reference Table (Bonus) +| Topic | Key Syntax | Example | +|---|---|---| +| Variable | VAR="value" | NAME="DevOps" | +| Argument | $1, $2 | ./script.sh arg1 | +| If | if [ condition ]; then | if [ -f file ]; then | +| For loop | for i in list; do | for i in 1 2 3; do | +| Function | name() { ... } | greet() { echo "Hi"; } | +| Grep | grep pattern file | grep -i "error" log.txt | +| Awk | awk '{print $1}' file | awk -F: '{print $1}' /etc/passwd | +| Sed | sed 's/old/new/g' file | sed -i 's/foo/bar/g' config.txt | + +--- + +## 1. Basics + +**Shebang (#!/bin/bash)** +Tells the operating system which interpreter to use to execute the script. Must be the first line. +```bash +#!/bin/bash +``` + +**Running a Script** +Make the script executable first, then run it using a relative path or pass it directly to bash. +```bash +chmod +x script.sh +./script.sh +bash script.sh +``` + +**Comments** +Use the hash symbol for comments. The interpreter ignores anything after it on that line. +```bash +# This is a full-line comment +echo "Hello" # This is an inline comment +``` + +**Variables** +Declare without spaces around the equal sign. Double quotes allow variable expansion; single quotes treat everything literally. +```bash +VAR="DevOps" +echo $VAR # Outputs: DevOps +echo "$VAR" # Outputs: DevOps (Recommended to prevent word splitting) +echo '$VAR' # Outputs: $VAR (Literal string) +``` + +**Reading User Input** +Pauses the script to take input from the user and stores it in a variable. +```bash +echo "Enter your name:" +read USER_NAME +``` + +**Command-line Arguments** +Special variables automatically populated by bash based on how the script was called. +```bash +$0 # Name of the script itself +$1 # First argument passed to the script +$# # Total number of arguments passed +$@ # All arguments as a list +$? # Exit status of the last executed command (0 is success) +``` + +--- + +## 2. Operators and Conditionals + +**String Comparisons** +Used to check if strings are identical, different, empty, or not empty. +```bash +[ "$A" = "$B" ] # True if strings are equal +[ "$A" != "$B" ] # True if strings are not equal +[ -z "$A" ] # True if string is empty (zero length) +[ -n "$A" ] # True if string is NOT empty +``` + +**Integer Comparisons** +Strictly for numerical values. +```bash +[ $A -eq $B ] # Equal to +[ $A -ne $B ] # Not equal to +[ $A -lt $B ] # Less than +[ $A -gt $B ] # Greater than +[ $A -le $B ] # Less than or equal to +[ $A -ge $B ] # Greater than or equal to +``` + +**File Test Operators** +Used to check file attributes before performing operations on them. +```bash +[ -f file.txt ] # True if it is a regular file +[ -d my_dir ] # True if it is a directory +[ -e item ] # True if the file or directory exists +[ -r file.txt ] # True if file is readable +[ -w file.txt ] # True if file is writable +[ -x script.sh ]# True if file is executable +[ -s file.txt ] # True if file size is greater than zero +``` + +**If, Elif, Else Syntax** +Basic conditional branching. Always end the block with fi. +```bash +if [ $AGE -ge 18 ]; then + echo "Adult" +elif [ $AGE -ge 13 ]; then + echo "Teenager" +else + echo "Child" +fi +``` + +**Logical Operators** +Combine multiple conditions. +```bash +[ $A -eq 1 ] && [ $B -eq 2 ] # AND: True if both are true +[ $A -eq 1 ] || [ $B -eq 2 ] # OR: True if either is true +! [ $A -eq 1 ] # NOT: Reverses the condition +``` + +**Case Statements** +Cleaner alternative to multiple if-elif statements when checking exact matches. +```bash +case $ACTION in + start) echo "Starting..." ;; + stop) echo "Stopping..." ;; + *) echo "Invalid action" ;; +esac +``` + +--- + +## 3. Loops + +**For Loop (List-based and C-style)** +Iterates over a predefined list of items or uses traditional numerical conditions. +```bash +# List-based +for COLOR in red green blue; do + echo $COLOR +done + +# C-style +for (( i=1; i<=5; i++ )); do + echo $i +done +``` + +**While Loop** +Runs continuously as long as the condition evaluates to true. +```bash +COUNT=1 +while [ $COUNT -le 5 ]; do + echo $COUNT + COUNT=$((COUNT + 1)) +done +``` + +**Until Loop** +Runs continuously as long as the condition evaluates to false. +```bash +COUNT=1 +until [ $COUNT -gt 5 ]; do + echo $COUNT + COUNT=$((COUNT + 1)) +done +``` + +**Loop Control** +Used to prematurely exit a loop or skip the current iteration. +```bash +break # Exits the loop completely +continue # Skips to the next iteration of the loop +``` + +**Looping Over Files** +Handy for bulk renaming or processing files in a directory. +```bash +for file in *.log; do + echo "Processing $file" +done +``` + +**Looping Over Command Output** +Safest way to read files or command output line-by-line. +```bash +cat errors.log | while read line; do + echo "Found: $line" +done +``` + +--- + +## 4. Functions + +**Defining and Calling a Function** +Group reusable code blocks. Define it first, then call it by its name without parentheses. +```bash +my_function() { + echo "Executing function tasks" +} +my_function # Calling the function +``` + +**Passing Arguments** +Functions use their own $1, $2, etc., separate from the script's global arguments. +```bash +greet() { + echo "Hello, $1" +} +greet "DevOps Engineer" +``` + +**Return Values vs Echo** +Bash functions do not return strings; they return exit statuses (0-255). Use echo to return data. +```bash +check_status() { + return 0 # Used for success/fail checks +} +get_name() { + echo "Rashid" # Used to pass string data back +} +``` + +**Local Variables** +Using local prevents variables inside functions from overwriting global variables. +```bash +calculate() { + local RESULT=100 + echo $RESULT +} +``` + +--- + +## 5. Text Processing Commands + +**grep** +Searches for patterns inside text. +```bash +grep "ERROR" file.log # Basic search +grep -i "error" file.log # Case-insensitive +grep -r "auth" /var/log/ # Search recursively in a directory +grep -c "Failed" auth.log# Count number of matches +grep -n "Deny" auth.log # Print matches with line numbers +grep -v "INFO" file.log # Print lines that DO NOT match +grep -E "ERROR|Failed" # Use extended regex (multiple words) +``` + +**awk** +Powerful column and pattern processing tool. +```bash +awk '{print $1, $3}' file.txt # Print column 1 and 3 (default space delimiter) +awk -F: '{print $1}' /etc/passwd # Change delimiter to colon +awk 'BEGIN {print "Start"} {print $1} END {print "Done"}' # Execute before and after processing +``` + +**sed** +Stream editor for finding and replacing text dynamically. +```bash +sed 's/old/new/' file.txt # Replace first occurrence on each line +sed 's/old/new/g' file.txt # Replace globally across the line +sed '/DEBUG/d' file.log # Delete all lines containing DEBUG +sed -i 's/foo/bar/g' file.txt # Edit the file in-place (saves changes) +``` + +**cut** +Extracts specific sections from each line based on a delimiter. +```bash +cut -d':' -f1 /etc/passwd # Cuts by colon and returns the first field +``` + +**sort** +Sorts lines of text files. +```bash +sort names.txt # Alphabetical sorting +sort -n numbers.txt # Numerical sorting +sort -r list.txt # Reverse sorting +sort -u list.txt # Sort and keep only unique values +``` + +**uniq** +Filters out adjacent duplicate lines. Almost always used after sort. +```bash +uniq file.txt # Remove consecutive duplicates +uniq -c file.txt # Prefix each line with its occurrence count +``` + +**tr** +Translates or deletes characters. Operates on standard input only. +```bash +cat text.txt | tr 'a-z' 'A-Z' # Convert lowercase to uppercase +cat text.txt | tr -d ' ' # Delete all spaces +``` + +**wc** +Counts newlines, words, and bytes. +```bash +wc -l file.txt # Count lines +wc -w file.txt # Count words +wc -c file.txt # Count characters +``` + +**head / tail** +Outputs the first or last parts of files. +```bash +head -n 5 file.txt # Print first 5 lines +tail -n 10 log.txt # Print last 10 lines +tail -f app.log # Follow mode: continuously stream new lines +``` + +--- + +## 6. Useful Patterns and One-Liners + +**Find and delete files older than 7 days** +```bash +find /var/log -type f -name "*.log" -mtime +7 -exec rm {} \; +``` + +**Count lines in all .log files in a directory** +```bash +find . -name "*.log" | xargs wc -l +``` + +**Replace a string across multiple files** +```bash +find . -type f -name "*.conf" -exec sed -i 's/old_host/new_host/g' {} + +``` + +**Check if a service (e.g., Docker) is running** +```bash +systemctl is-active --quiet docker && echo "Running" || echo "Stopped" +``` + +**Monitor disk usage and alert if above 80%** +```bash +df -h / | awk 'NR==2 {print $5}' | sed 's/%//' | while read usep; do [ $usep -ge 80 ] && echo "Disk full!"; done +``` + +**Parse a specific key from a JSON payload (using grep/awk)** +```bash +cat data.json | grep '"status"' | awk -F'"' '{print $4}' +``` + +**Tail a log and filter for specific errors in real-time** +```bash +tail -f /var/log/syslog | grep --line-buffered -E "CRITICAL|Failed" +``` + +--- + +## 7. Error Handling and Debugging + +**Exit Codes** +Every command returns a status. Use exit commands to pass success or failure to the OS. +```bash +$? # Contains the exit code of the last command +exit 0 # Represents a successful execution +exit 1 # Represents an error or failure +``` + +**Strict Execution Modes** +Place these at the top of your scripts to build fail-safe pipelines. +```bash +set -e # Immediately exit if any command returns a non-zero status +set -u # Treat unset/uninitialized variables as an error and exit +set -o pipefail # If any command in a pipeline fails, the whole pipeline fails +``` + +**Debug Mode** +Prints each command to the terminal before executing it. Great for troubleshooting. +```bash +set -x # Enable debug trace mode +set +x # Disable debug trace mode +``` + +**Trap Command** +Catches system signals (like script exit or CTRL+C) and forces a specific action. +```bash +# Deletes the temporary file whenever the script exits, even on failure +trap 'rm -f /tmp/temp_file.txt' EXIT +``` diff --git a/2026/day-22/day-22-notes.md b/2026/day-22/day-22-notes.md new file mode 100644 index 0000000000..4176d0e8a6 --- /dev/null +++ b/2026/day-22/day-22-notes.md @@ -0,0 +1,19 @@ +# Day 22: Git Workflow Notes + +**1. What is the difference between git add and git commit?** +- git add: It moves changes from your working directory to the staging area. It tells Git, "I want to include these specific changes in the next save." +- git commit: It takes everything in the staging area and permanently saves it as a snapshot in the local repository history with a descriptive message. + +**2. What does the staging area do? Why doesn't Git just commit directly?** +The staging area (or index) acts as a buffer or waiting room. It allows developers to logically group related changes into a single commit rather than forcing them to commit every modified file all at once. It helps maintain a clean, organized commit history. + +**3. What information does git log show you?** +It displays the chronological commit history of the active branch. It shows the unique commit ID (SHA-1 hash), the author's name and email, the timestamp, and the commit message. + +**4. What is the .git/ folder and what happens if you delete it?** +The .git/ folder is the brain and database of the repository. It stores all the snapshots, branches, configuration settings, and history. If you delete it, you lose the entire version control history, and the project becomes just a standard, untracked folder again. + +**5. What is the difference between a working directory, staging area, and repository?** +- Working Directory: The actual files on your machine that you are currently editing. +- Staging Area: The intermediate waiting room where you place files that are ready to be committed. +- Repository: The permanent local database where Git stores all your committed snapshots and historical versions. diff --git a/2026/day-22/day-22-solution.md b/2026/day-22/day-22-solution.md new file mode 100644 index 0000000000..ebcfd102ff --- /dev/null +++ b/2026/day-22/day-22-solution.md @@ -0,0 +1,45 @@ +# Day 22: Introduction to Git - Your First Repository + +## Objective +To understand the fundamentals of Version Control, set up a local Git repository, build a living Git commands reference, and master the Git workflow (Working Directory -> Staging Area -> Repository). + +--- + +## Task 1 & 2: Git Setup and Initialization +Configured the global Git identity and initialized a fresh local repository to track changes. + +**Commands Executed:** +```bash +git config --global user.name "rashid-khan681" +git config --global user.email "rk8539100@gmail.com" +mkdir devops-git-practice +cd devops-git-practice +git init +``` + +--- + +## Task 3, 4 & 5: Creating Reference & Building History +Created a `git-commands.md` file, staged it, and made multiple edits to simulate a real-world development workflow, generating a clean commit history. + +**Commands Executed:** +```bash +git add git-commands.md +git commit -m "docs: add setup and basic workflow commands" +# Made changes to the file +git add git-commands.md +git commit -m "docs: add git diff command for viewing changes" +# Made further changes +git commit -m "docs: add git log command for commit history" +``` + +**Proof of Execution (Git Log):** +![Git Log Output](./task-5-git-log.png) + +--- + +## Task 6: Understand the Git Workflow +Answered core architectural questions regarding the Staging Area, the `.git/` folder, and the difference between `git add` and `git commit`. + +**Output:** +The detailed answers have been saved in a dedicated file named `day-22-notes.md` within this directory. The living reference guide is saved as `git-commands.md`. diff --git a/2026/day-22/git-commands.md b/2026/day-22/git-commands.md new file mode 100644 index 0000000000..a7255908d7 --- /dev/null +++ b/2026/day-22/git-commands.md @@ -0,0 +1,23 @@ +# Git Commands Reference + +## Setup & Config +* `git config --global user.name "Name"` - Sets the author name to be attached to your commits. + * Example: `git config --global user.name "rashid-khan681"` +* `git config --global user.email "email"` - Sets the author email. + * Example: `git config --global user.email "rk8539100@gmail.com"` +* `git init` - Initializes a new, empty Git repository in the current folder. + * Example: `git init` + +## Basic Workflow +* `git status` - Shows the state of the working directory and staging area. + * Example: `git status` + +## Viewing Changes +* `git diff` - Shows unstaged changes between your working directory and the staging area. + * Example: `git diff` +* `git log` - Shows the commit history for the currently active branch. + * Example: `git log --oneline` + +## Undoing Changes +* `git checkout -- ` - Discards changes in the working directory. + * Example: `git checkout -- file.txt` diff --git a/2026/day-22/task-5-git-log.png b/2026/day-22/task-5-git-log.png new file mode 100644 index 0000000000..e8a1e812b7 Binary files /dev/null and b/2026/day-22/task-5-git-log.png differ diff --git a/2026/day-23/day-23-notes.md b/2026/day-23/day-23-notes.md new file mode 100644 index 0000000000..24685be1a2 --- /dev/null +++ b/2026/day-23/day-23-notes.md @@ -0,0 +1,36 @@ + +## Task 2: Branching Commands + +**Difference between `git switch` and `git checkout`:** + +`git checkout` is an older, overloaded command used for both switching branches AND restoring files. Because it did too many things, it caused confusion. Git introduced `git switch` purely and specifically for switching branches, making it a much safer and clearer command. + +## Task 3: Push to GitHub + +**Difference between `origin` and `upstream`:** + +- **`origin`**: This is the default alias Git gives to your own remote repository (e.g., your personal repo or your fork on GitHub). It's where your local changes are pushed by default. +- **`upstream`**: This refers to the original, central repository that you forked from. Developers track `upstream` to fetch updates made by the core team and keep their own `origin` fork in sync. + +## Task 4: Pull from GitHub + +**Difference between `git fetch` and `git pull`:** + +- **`git fetch`**: It only downloads the latest metadata and commits from the remote repository to your local machine, but it **does not** automatically merge them into your current working files. It's a safe way to review what has changed remotely before applying it. +- **`git pull`**: It performs two actions in a single command: it runs `git fetch` to download the changes, and then immediately runs `git merge` to integrate those changes directly into your active working directory. + +## Task 5: Clone vs Fork + +**1. What is the difference between clone and fork?** + +- **Clone**: A Git command that downloads an exact copy of a repository to your local machine. It connects your local environment to the remote repository. +- **Fork**: A GitHub-specific feature (not a Git command) that creates a personal copy of someone else's repository under your own GitHub account. You fork when you don't have write access to the original repo but want to propose changes. + +**2. When would you clone vs fork?** + +- **Clone**: When you have read/write access to a repository (like your own project or your team's internal repo) and want to work on it locally. +- **Fork**: When you want to contribute to a public open-source project where you do NOT have direct write access. + +**3. After forking, how do you keep your fork in sync with the original repo?** + +You add the original repository as a new remote called `upstream` (`git remote add upstream `). Then, you regularly run `git fetch upstream` and `git merge upstream/main` to sync their latest changes into your local branch, which you can then push to your own `origin` fork. diff --git a/2026/day-23/day-23-solution.md b/2026/day-23/day-23-solution.md new file mode 100644 index 0000000000..f9f1fc9ee0 --- /dev/null +++ b/2026/day-23/day-23-solution.md @@ -0,0 +1,25 @@ +# Day 23: Git Branching & Working with GitHub + +## Objective +To master Git branching, understand the difference between local and remote repositories, and learn how to push, pull, fork, and sync code using GitHub. + +## Tasks Completed +1. **Understanding Branches**: Answered core concepts about `HEAD`, isolated environments, and branching mechanisms. +2. **Branching Hands-On**: Created, switched (`git switch`), and deleted branches locally. +3. **Push to GitHub**: Created a new remote repository, connected it via `origin`, and pushed multiple branches. +4. **Pull from GitHub**: Simulated remote collaboration by editing code on the GitHub UI and syncing it locally via `git pull` (utilized `git stash` to resolve local conflict warnings). +5. **Clone vs Fork**: Detailed the architectural differences between a standard Git clone and a GitHub fork, including upstream syncing. + +## Outputs +- Detailed theoretical answers are stored in `day-23-notes.md`. +- The continuously updated command reference is in `git-commands.md`. + +## Proof of Execution +### Branching & Isolation +![Branching](./task-2-branching.png) + +### Pushing Branches to GitHub Remote +![GitHub Push](./task-3-github-push.png) + +### Pulling Cloud Changes to Local +![Git Pull](./task-4-git-pull.png) diff --git a/2026/day-23/git-commands.md b/2026/day-23/git-commands.md new file mode 100644 index 0000000000..82372cc138 --- /dev/null +++ b/2026/day-23/git-commands.md @@ -0,0 +1,13 @@ +# Git Commands Reference + +## Setup & Config +* `git config --global user.name "rashid-khan681"` - Sets author name. +* `git config --global user.email "rk8539100@gmail.com"` - Sets author email. +* `git init` - Initializes a new Git repository. + +## Basic Workflow +* `git status` - Shows the state of the working directory. +* `git add git-commands.md` - Stages a specific file. +* `git commit -m "docs: added setup and basic workflow commands"` - Commits changes. +* `git log --oneline` - Shows commit history. +* Note: This line was added directly from the GitHub UI. diff --git a/2026/day-23/task-2-branching.png b/2026/day-23/task-2-branching.png new file mode 100644 index 0000000000..834024f0f5 Binary files /dev/null and b/2026/day-23/task-2-branching.png differ diff --git a/2026/day-23/task-3-github-push.png b/2026/day-23/task-3-github-push.png new file mode 100644 index 0000000000..5d73d54d95 Binary files /dev/null and b/2026/day-23/task-3-github-push.png differ diff --git a/2026/day-23/task-4-git-pull.png b/2026/day-23/task-4-git-pull.png new file mode 100644 index 0000000000..0563ccb514 Binary files /dev/null and b/2026/day-23/task-4-git-pull.png differ diff --git a/2026/day-24/day-24-notes.md b/2026/day-24/day-24-notes.md new file mode 100644 index 0000000000..87b49c678d --- /dev/null +++ b/2026/day-24/day-24-notes.md @@ -0,0 +1,59 @@ +# Day 24: Advanced Git - Merge, Rebase, Stash & Cherry Pick + +## Task 1: Git Merge + +**1. What is a fast-forward merge?** +A fast-forward merge happens when the target branch (e.g., `main`) has no new commits since the feature branch was created. Git simply moves the `main` pointer forward to the latest commit of the feature branch without creating a new merge commit. + +**2. When does Git create a merge commit instead?** +Git creates a merge commit (3-way merge) when both the target branch (`main`) and the feature branch have diverged (i.e., both have independent commits). It combines the histories and creates a new unified "merge commit". + +**3. What is a merge conflict?** +A merge conflict occurs when Git cannot automatically resolve differences between two branchesβ€”typically because the exact same line in the same file was modified differently in both branches. The developer must manually edit the file to choose the correct code, stage, and commit the resolution. + +## Task 2: Git Rebase + +**1. What does rebase actually do to your commits?** +Rebase temporarily saves the commits of your feature branch, moves the base of your feature branch to the very tip of the target branch (like `main`), and then re-applies your saved commits one by one on top. It literally rewrites commit history by creating brand new commit hashes. + +**2. How is the history different from a merge?** +A merge preserves the exact chronological history and shows diverging paths coming together via a "merge commit". A rebase rewrites history to create a perfectly straight, linear line, making the project look like everything was developed sequentially. + +**3. Why should you never rebase commits that have been pushed and shared with others?** +Because rebase creates completely new commit hashes. If you rebase a branch that your teammates are already working on and push it, Git will see diverging histories. Your teammates will face chaotic merge conflicts, duplicate commits, and a broken local environment. Only rebase local, un-pushed branches! + +**4. When would you use rebase vs merge?** +- **Rebase:** Use it to clean up a local, private feature branch before raising a Pull Request, so the main project history stays clean and linear. +- **Merge:** Use it when combining completed feature branches into `main` or when collaborating on shared branches where preserving the exact chronological truth is necessary. + +## Task 3: Squash Commit vs Merge Commit + +**1. What does squash merging do?** +Squash merging takes all the individual commits from a feature branch, combines them into one single, comprehensive set of changes, and places them into the target branch as a single new commit. + +**2. When would you use squash merge vs regular merge?** +- **Squash Merge:** Use it when a feature branch has many messy, insignificant, or "work-in-progress" commits (like "fixing typo", "testing again"). It keeps the `main` branch history perfectly clean and readable. +- **Regular Merge:** Use it when every individual commit in the feature branch has significant historical value and you want to preserve the exact step-by-step development process. + +**3. What is the trade-off of squashing?** +The main trade-off is the loss of granular history. If you squash 20 commits into 1, you lose the detailed step-by-step log of how that feature was built, making it harder to revert a specific micro-change later. + +## Task 4: Git Stash + +**1. What is the difference between `git stash pop` and `git stash apply`?** +- **`git stash pop`**: Retrieves the most recently stashed changes, applies them to your working directory, and **deletes** that stash from the stash list. +- **`git stash apply`**: Retrieves and applies the stashed changes to your working directory but **keeps** the stash in the stash list for future use (great for applying the same stash to multiple branches). + +**2. When would you use stash in a real-world workflow?** +You use stash when you are in the middle of working on a feature, but suddenly you need to switch branches to fix a critical production bug. You don't want to make a half-baked commit, so you stash your current changes, fix the bug on the other branch, come back, and apply your stash to resume where you left off. + +## Task 5: Cherry Picking + +**1. What does cherry-pick do?** +Cherry-picking allows you to take a specific, individual commit from one branch and apply it to another branch, without merging the entire history of the source branch. + +**2. When would you use cherry-pick in a real project?** +If a developer accidentally commits a critical hotfix to a feature branch (which isn't ready to be merged yet), you can use `cherry-pick` to pull just that specific hotfix commit directly into the `main` or `production` branch to resolve the live issue immediately. + +**3. What can go wrong with cherry-picking?** +Cherry-picking creates a brand-new commit hash for the applied change. If you cherry-pick a commit and later try to merge the original branch that contained the same commit, Git might get confused, leading to nasty merge conflicts or duplicate commits in the history. diff --git a/2026/day-24/day-24-solution.md b/2026/day-24/day-24-solution.md new file mode 100644 index 0000000000..ce75e96e49 --- /dev/null +++ b/2026/day-24/day-24-solution.md @@ -0,0 +1,103 @@ +# Day 24: Advanced Git Master Solution + +## Step 1: Git Merge & Conflicts +**Objective:** Perform Fast-Forward merge, 3-Way merge, and manually resolve an intentional merge conflict. +```bash +# Simulating a conflict by editing the same line on both branches +git switch -c feature-conflict +echo "Line 1: Edited by feature branch" > conflict-test.txt +git commit -am "docs: edit line 1 in feature branch" + +git switch main +echo "Line 1: Edited by main branch" > conflict-test.txt +git commit -am "docs: edit line 1 in main branch" + +# Trigger conflict and resolve manually +git merge feature-conflict +echo "Line 1: Resolved text keeping both main and feature changes" > conflict-test.txt +git commit -am "fix: resolved merge conflict in conflict-test.txt" +``` +**Output Screenshot:** +![Merge Conflict Resolution](./task-1-merge-conflict.png) + +--- + +## Step 2: Git Rebase +**Objective:** Rewrite commit history to maintain a perfectly linear project timeline without merge loops. +```bash +# Branching off, committing, and moving main ahead +git switch -c feature-dashboard +echo "Dashboard UI" > dashboard-ui.txt +git add dashboard-ui.txt +git commit -m "feat: add dashboard UI" + +# Rebasing feature onto the new tip of main +git switch feature-dashboard +git rebase main + +# Viewing the linear history +git log --oneline --graph --all -n 12 +``` +**Output Screenshot:** +![Git Rebase Graph](./task-2-rebase-graph.png) + +--- + +## Step 3: Squash Commit vs Regular Merge +**Objective:** Condense multiple messy Work-In-Progress (WIP) commits into a single clean commit for `main`. +```bash +# Squashing 5 commits from feature-profile into 1 +git switch main +git merge --squash feature-profile +git commit -m "feat: add complete profile feature (squashed 5 commits into 1)" + +# Regular merge comparison +git switch main +git merge feature-settings --no-ff -m "Merge branch 'feature-settings' into main" +``` +**Output Screenshot:** +![Squash vs Regular Merge](./task-3-squash-merge.png) + +--- + +## Step 4: Git Stash +**Objective:** Temporarily save uncommitted changes to safely switch branches during emergencies. +```bash +# Saving uncommitted work to the stash locker +echo "Doing some complex work..." > urgent-work.txt +git add urgent-work.txt +git stash push -m "saving complex work temporarily" + +# Popping the stash after returning to the branch +git stash pop + +# Applying a specific stash from the list +git stash list +git stash apply stash@{1} +``` +**Output Screenshot:** +![Git Stash Operations](./task-4-stash.png) + +--- + +## Step 5: Git Cherry Picking +**Objective:** Extract a single, specific commit from another branch and apply it to the current branch. +```bash +# Grabbing the hash of a specific commit on feature-hotfix +CHERRY_PICK_HASH=$(git rev-parse HEAD) + +# Switching to main and applying ONLY that specific commit +git switch main +git cherry-pick $CHERRY_PICK_HASH +``` +**Output Screenshot:** +![Git Cherry Pick](./task-5-cherry-pick.png) + +--- + +## 5 Key Points What I Learned +1. **Merge Conflicts are Protections, Not Errors:** Git physically prevents code overwrites when lines clash, forcing the developer to manually verify and resolve the final state of the code. +2. **Rebase Keeps History Linear:** While `git merge` preserves the exact chronological branching timeline, `git rebase` rewrites history to place feature commits perfectly on top of the main branch, avoiding diamond-shaped merge loops. +3. **Squash Merging Cleans Up Garbage:** Using `--squash` is the best way to combine dozens of messy, minor WIP commits ("typo fix", "test again") into one pristine, meaningful commit on the `main` branch. +4. **Stash is the Ultimate Context-Switcher:** `git stash` safely locks away uncommitted changes so you can jump to another branch for an urgent fix without losing your current progress or making a broken commit. +5. **Cherry-Picking Isolates Hotfixes:** You don't have to merge an entire branch to get a specific fix. `git cherry-pick ` allows you to surgically extract exactly the one commit you need for production. diff --git a/2026/day-24/git-commands.md b/2026/day-24/git-commands.md new file mode 100644 index 0000000000..575a83052d --- /dev/null +++ b/2026/day-24/git-commands.md @@ -0,0 +1,23 @@ +# Git Commands Reference + +## Setup & Config +* `git config --global user.name "rashid-khan681"` - Sets author name. +* `git config --global user.email "rk8539100@gmail.com"` - Sets author email. +* `git init` - Initializes a new Git repository. + +## Basic Workflow +* `git status` - Shows the state of the working directory. +* `git add git-commands.md` - Stages a specific file. +* `git commit -m "docs: added setup and basic workflow commands"` - Commits changes. +* `git log --oneline` - Shows commit history. +* Note: This line was added directly from the GitHub UI. + +## Advanced Operations (Day 24) +* `git merge ` - Merges the specified branch into the current branch. +* `git merge --squash ` - Combines all commits from the source branch into a single staged change. +* `git rebase ` - Rewrites history by moving the base of your current branch to the tip of the specified branch. +* `git stash` or `git stash push -m "msg"` - Temporarily saves uncommitted work. +* `git stash list` - Lists all stashed changes. +* `git stash pop` - Applies the most recent stash and removes it from the list. +* `git stash apply stash@{n}` - Applies a specific stash but keeps it in the list. +* `git cherry-pick ` - Applies a specific commit from another branch to the current branch. diff --git a/2026/day-24/task-1-merge-conflict.png b/2026/day-24/task-1-merge-conflict.png new file mode 100644 index 0000000000..86001e0c19 Binary files /dev/null and b/2026/day-24/task-1-merge-conflict.png differ diff --git a/2026/day-24/task-2-rebase-graph.png b/2026/day-24/task-2-rebase-graph.png new file mode 100644 index 0000000000..61f7aefa10 Binary files /dev/null and b/2026/day-24/task-2-rebase-graph.png differ diff --git a/2026/day-24/task-3-squash-merge.png b/2026/day-24/task-3-squash-merge.png new file mode 100644 index 0000000000..34bfbcd8a8 Binary files /dev/null and b/2026/day-24/task-3-squash-merge.png differ diff --git a/2026/day-24/task-4-stash.png b/2026/day-24/task-4-stash.png new file mode 100644 index 0000000000..694c533898 Binary files /dev/null and b/2026/day-24/task-4-stash.png differ diff --git a/2026/day-24/task-5-cherry-pick.png b/2026/day-24/task-5-cherry-pick.png new file mode 100644 index 0000000000..3aaa4a9afa Binary files /dev/null and b/2026/day-24/task-5-cherry-pick.png differ diff --git a/2026/day-25/day-25-notes.md b/2026/day-25/day-25-notes.md new file mode 100644 index 0000000000..3a0d45c6a8 --- /dev/null +++ b/2026/day-25/day-25-notes.md @@ -0,0 +1,82 @@ +# Day 25: Git Reset, Revert & Branching Strategies + +## Task 1: Git Reset β€” Hands-On + +**Q: Use `git reset --soft` to go back one commit β€” what happens to the changes?** +*Answer:* When I ran this, the commit was deleted, but my file changes stayed intact and were left in the Staging Area (green status). It's like undoing just the commit message but keeping the work ready to commit again. + +**Q: Re-commit, then use `git reset --mixed` to go back one commit β€” what happens now?** +*Answer:* This deleted the commit and also unstaged my changes. My file modifications were still safe in my working directory, but they turned red in `git status` (not ready for commit yet). I observed that this is Git's default reset behavior. + +**Q: Re-commit, then use `git reset --hard` to go back one commit β€” what happens this time?** +*Answer:* This completely nuked the commit AND deleted my physical file changes. The code was permanently gone from my working directory. + +**Q: What is the difference between `--soft`, `--mixed`, and `--hard`?** +*Answer:* It's all about where my files end up after the commit is deleted. `--soft` keeps them staged, `--mixed` keeps them in the working directory (unstaged), and `--hard` deletes them entirely. + +**Q: Which one is destructive and why?** +*Answer:* `--hard` is highly destructive because it deletes the actual code I wrote in my local files, not just the commit history. If I haven't backed it up, it's gone forever. + +**Q: When would you use each one?** +*Answer:* I'd use `--soft` if I just messed up a commit message and want to quickly re-commit. I'd use `--mixed` if I want to unstage files and modify my code before committing again. I'd use `--hard` only when my recent code is complete garbage and I want to start fresh from the last good commit. + +**Q: Should you ever use `git reset` on commits that are already pushed?** +*Answer:* Absolutely NOT. If I reset pushed commits, I rewrite history. When my team tries to pull, it will break their local repositories and cause massive merge conflicts. + +--- + +## Task 2: Git Revert β€” Hands-On + +**Q: Revert commit Y (the middle one) β€” what happens?** +*Answer:* Git created a brand new commit that did the exact opposite of what Commit Y did (it removed the file I added). It didn't delete the old commit; it just pushed a new "undo" commit on top. + +**Q: Check `git log` β€” is commit Y still in the history?** +*Answer:* Yes! When I checked the log, Commit Y was still sitting exactly where it was. The new `Revert "Commit Y..."` was added at the top of the history safely. + +**Q: How is `git revert` different from `git reset`?** +*Answer:* `reset` deletes history by moving backward. `revert` preserves history by moving forward and adding a new commit that cancels out the old changes. + +**Q: Why is revert considered safer than reset for shared branches?** +*Answer:* Because it doesn't rewrite the past. Since it just adds a new commit, my team can simply run `git pull` to safely get my "undo" changes without their local repo breaking or complaining about mismatched histories. + +**Q: When would you use revert vs reset?** +*Answer:* I use `reset` for my local, unpushed branches when I want to clean up my messy history. I use `revert` for public/shared branches (like `main`) when I need to undo a bug that has already been pushed to GitHub. + +--- + +## Task 3: Reset vs Revert Summary + +| Feature | `git reset` | `git revert` | +| :--- | :--- | :--- | +| **What it does?** | Rewinds and removes commits from history. | Adds a new commit that undoes the changes. | +| **Destructive?** | Yes (especially `--hard`). | No (History keeps moving forward). | +| **Safe for Shared/Pushed?** | ❌ NO. Breaks history for other devs. | βœ… YES. Safe for active collaboration. | +| **When to use?** | Fixing local mistakes before pushing. | Undoing bugs on public/shared branches. | + +--- + +## Task 4: Branching Strategies + +### 1. GitFlow +* **How it works:** Strict structure using `main`, `develop`, `feature`, `release`, and `hotfix` branches. +* **When to use:** Enterprise software with scheduled, versioned releases (e.g., v1.0, v2.0). +* **Pros & Cons:** Highly organized and safe, but can be slow and bureaucratic. + +### 2. GitHub Flow +* **How it works:** Extremely simple. One always-deployable `main` branch, and developers create temporary `feature` branches for PRs. +* **When to use:** Agile teams deploying multiple times a day (Continuous Deployment). +* **Pros & Cons:** Fast and lightweight, but requires rigorous automated testing. + +### 3. Trunk-Based Development +* **How it works:** Developers push code directly to the main trunk (`main`) multiple times a day using feature flags. Short-lived branches only. +* **When to use:** High-performing tech giants (Netflix, Google). +* **Pros & Cons:** Zero "merge hell", but requires elite-level CI/CD pipelines. + +**Q: Which strategy would you use for a startup shipping fast?** +*Answer:* I would choose GitHub Flow or Trunk-Based Development. They remove bottlenecks and allow rapid deployment. + +**Q: Which strategy would you use for a large team with scheduled releases?** +*Answer:* I would use GitFlow. It provides the strict structure needed to manage multiple environments and versioned releases safely. + +**Q: Which one does your favorite open-source project use?** +*Answer:* Most open-source projects on GitHub (like React or Kubernetes) use a variation of **GitHub Flow** (specifically, the Fork and Pull Request model). diff --git a/2026/day-25/git-commands.md b/2026/day-25/git-commands.md new file mode 100644 index 0000000000..89ec0d3fd3 --- /dev/null +++ b/2026/day-25/git-commands.md @@ -0,0 +1,37 @@ +# Master Git Commands Cheat Sheet (Days 22-25) + +## 1. Setup & Configuration +* `git config --global user.name "Your Name"`: Set username. +* `git config --global user.email "email@domain.com"`: Set email. + +## 2. Basic Workflow +* `git status`: Check the state of the working directory. +* `git add ` / `git add .`: Stage changes. +* `git commit -m "message"`: Commit staged changes. +* `git log --oneline --graph --all`: View visual commit history. + +## 3. Branching & Remote +* `git branch `: Create a new branch. +* `git switch ` or `git checkout `: Switch to a branch. +* `git switch -c `: Create and switch instantly. +* `git clone `: Download a remote repo. +* `git fetch`: Download remote changes without merging. +* `git pull origin main`: Fetch and merge changes. +* `git push origin `: Upload local commits to remote. + +## 4. Advanced Git (Merge, Rebase, Squash) +* `git merge `: Combines histories (creates a diamond loop). +* `git merge --squash `: Condenses multiple commits into one before merging. +* `git rebase `: Moves feature commits to the tip of the base branch (creates a clean linear line). + +## 5. Lifesavers (Stash & Cherry-Pick) +* `git stash`: Temporarily locks away uncommitted changes. +* `git stash pop`: Brings back the stashed changes. +* `git cherry-pick `: Surgically extracts a specific commit and applies it to the current branch. + +## 6. Damage Control (Reset, Revert, Reflog) +* `git reset --soft HEAD~1`: Undoes commit, keeps files staged. +* `git reset --mixed HEAD~1`: Undoes commit, unstages files (Default). +* `git reset --hard HEAD~1`: NUKES commit and deletes local changes. +* `git revert `: Safely undoes a commit by creating a new inverse commit. +* `git reflog`: The ultimate safety net. Shows every action taken in Git, allowing you to recover even from a hard reset. diff --git a/2026/day-26/day-26-notes.md b/2026/day-26/day-26-notes.md new file mode 100644 index 0000000000..9552c7d051 --- /dev/null +++ b/2026/day-26/day-26-notes.md @@ -0,0 +1,39 @@ +# Day 26: GitHub CLI (gh) – Manage GitHub from Your Terminal + +## Task 1: Install and Authenticate +* **What authentication methods does `gh` support?** + *Answer:* `gh` primarily supports authentication via web browser login (`gh auth login`), personal access tokens (PATs) passed via environment variables or terminal prompts, and SSH keys. It safely stores credentials in the system's secure credential store. + +## Task 2: Working with Repositories +* **Observations & Hands-On:** + * Created public repos instantly with README using `gh repo create --public --readme`. + * Cloned repos directly with `gh repo clone` without navigating to the browser. + * Inspected repo details, listed all repositories with `gh repo list`, opened them in the browser via `gh repo view --web`, and safely cleaned up test repos using `gh repo delete`. + +## Task 3: Issues +* **Created issue:** Tested creating issues with a title, body, and labels directly from the CLI (`gh issue create`). +* **Managed issues:** Listed open issues (`gh issue list`), viewed specific issues by number (`gh issue view`), and closed them (`gh issue close`). +* **How could you use `gh issue` in a script or automation?** + *Answer:* By passing the `--json` flag (e.g., `gh issue list --json number,title`), I can pipe the output into tools like `jq` to build automation scripts. For example, automatically labeling incoming bugs, assigning issues to specific engineers based on keywords, or generating a tracking issue if a cron job or CI/CD pipeline crashes. + +## Task 4: Pull Requests +* **Workflow tested:** Created a new branch, made local modifications, pushed to remote, and raised a pull request entirely from the terminal (`gh pr create --fill`). +* **PR Management:** Listed open PRs (`gh pr list`), inspected reviews and checks (`gh pr view`), checked out a PR locally for testing (`gh pr checkout`), and merged it using squash method (`gh pr merge --squash`). +* **What merge methods does `gh pr merge` support?** + *Answer:* It supports standard merge (`--merge`), squash and merge (`--squash`), and rebase and merge (`--rebase`). +* **How would you review someone else's PR using `gh`?** + *Answer:* I can check out their branch locally with `gh pr checkout ` to test the code on my machine, inspect the checks status using `gh pr checks`, and submit my review via `gh pr review --approve` or `--request-changes`. + +## Task 5: GitHub Actions & Workflows (Preview) +* **Explored:** Listed workflow runs (`gh run list`), inspected specific run details and failed logs (`gh run view --log-failed`), and manually triggered workflows (`gh workflow run`). +* **How could `gh run` and `gh workflow` be useful in a CI/CD pipeline?** + *Answer:* They allow developers and SREs to monitor pipeline health, debug failing tests by pulling logs directly to the terminal (`--log-failed`), and trigger manual or parameter-driven deployments without ever needing to open the GitHub web dashboard. + +## Task 6: Useful `gh` Tricks +* **Tricks integrated into workflow:** + * `gh api`: To make raw REST/GraphQL requests against GitHub's API. + * `gh gist`: To instantly share configuration snippets or logs. + * `gh release`: To automate tagging and publishing release artifacts. + * `gh alias`: To map frequently used long commands to short custom shortcuts. + * `gh search repos`: To query repositories across GitHub directly from the command line. +## Live Terminal Test Check diff --git a/2026/day-26/git-commands.md b/2026/day-26/git-commands.md new file mode 100644 index 0000000000..de223168b8 --- /dev/null +++ b/2026/day-26/git-commands.md @@ -0,0 +1,43 @@ +# Master GitHub CLI (gh) Reference Cheat Sheet (Day 26) + +## 1. Authentication & Setup +* `gh auth login`: Authenticate with GitHub account via browser or token. +* `gh auth status`: Verify active logged-in account and token permissions. +* `gh config set editor `: Configure default text editor for the CLI. + +## 2. Repository Management (`gh repo`) +* `gh repo create --public --readme`: Create a new remote repo with a README. +* `gh repo clone `: Clone a repository quickly. +* `gh repo view --web`: Open current repository in the browser. +* `gh repo list`: List your repositories. +* `gh repo delete `: Delete a repository. +* `gh repo fork `: Fork any public repository. + +## 3. Issues Management (`gh issue`) +* `gh issue create --title "Title" --body "Desc" --label "bug"`: Create an issue. +* `gh issue list`: List open issues. +* `gh issue view `: View specific issue details and comments. +* `gh issue close `: Close an issue. +* `gh issue reopen `: Reopen a closed issue. + +## 4. Pull Requests (`gh pr`) +* `gh pr create --fill`: Create a PR automatically using commit messages. +* `gh pr list`: List open pull requests. +* `gh pr view `: Check PR status, checks, and reviewers. +* `gh pr checkout `: Checkout a PR branch locally for testing. +* `gh pr review --approve`: Approve a pull request. +* `gh pr merge --squash`: Merge a PR using squash method. + +## 5. GitHub Actions & Workflows (`gh run` & `gh workflow`) +* `gh run list`: List recent workflow runs. +* `gh run view `: View details of a specific workflow run. +* `gh run view --log-failed`: View logs specifically for failed runs. +* `gh workflow list`: List all available workflows in the repo. +* `gh workflow run `: Trigger a workflow manually. + +## 6. Advanced CLI Utilities (`gh api`, `gh gist`, `gh release`, `gh alias`) +* `gh api `: Make raw GitHub REST/GraphQL API requests. +* `gh gist create `: Create a GitHub Gist from a file. +* `gh release create --notes "Release notes"`: Create a GitHub release. +* `gh alias set ''`: Create custom command shortcuts. +* `gh search repos `: Search public GitHub repos from the terminal. diff --git a/2026/day-27/day-27-notes.md b/2026/day-27/day-27-notes.md new file mode 100644 index 0000000000..b823f7474b --- /dev/null +++ b/2026/day-27/day-27-notes.md @@ -0,0 +1,10 @@ +# Day 27: GitHub Profile Makeover + +## Before & After +* **Before State:** *(Missed the screenshot before diving into CLI automation!️)* Previously, all scripts were buried deep inside 90 daily folders. +* **After State:** `after-profile.png` (Attached in folder) - Clean, categorized, and recruiter-ready. + +## 3 Things I Improved & Why +1. **Repository Organization:** Extracted standalone scripts into dedicated `shell-scripts`, `python-scripts`, and `devops-notes` repos. **Why:** Categorized repos showcase specific skills instantly. +2. **Strategic Pinning:** Validated that my pinned repos highlight high-value architectural projects. **Why:** It shifts my identity from a "student" to an "action-oriented engineer". +3. **Profile README Audit:** Confirmed my bio clearly states my Cloud/DevOps focus. **Why:** Clean developer resumes convert profile views into networking opportunities. diff --git a/2026/day-28/day-28-notes.md b/2026/day-28/day-28-notes.md new file mode 100644 index 0000000000..ec72995265 --- /dev/null +++ b/2026/day-28/day-28-notes.md @@ -0,0 +1,186 @@ +# Day 28: Milestone Revision & Honest Self-Audit (Days 1–27) + +**Author:** Rashid Khan +**Track:** #90DaysOfDevOps (2026) +**Milestone:** Systems Engineering, Automation & Version Control Consolidation + +--- + +## Task 1: Self-Assessment Checklist + +### A. Linux Systems Administration +* [x] **Can do confidently** | Navigate the file system, create/move/delete files and directories (`mkdir -p`, `mv`, `rm -rf`, `cp -r`) +* [x] **Can do confidently** | Manage processes β€” list, inspect, and kill (`ps aux`, `top`, `pgrep`, `kill -15`, `kill -9`) +* [x] **Can do confidently** | Work with systemd β€” start, stop, enable, restart, and inspect status (`systemctl status`, `journalctl -u -f`) +* [x] **Can do confidently** | Read and edit text files using vi/vim or nano +* [x] **Can do confidently** | Troubleshoot CPU, memory, and disk issues using `top`, `free -h -t`, `df -h`, `du -sh` +* [x] **Can do confidently** | Explain the Linux file system hierarchy (`/`, `/etc`, `/var/log`, `/home`, `/tmp`, `/opt`, `/bin`) +* [x] **Can do confidently** | Create users and groups, manage passwords (`useradd`, `passwd`, `usermod -aG`) +* [x] **Can do confidently** | Set file permissions using chmod (numeric: `755`, `644`; symbolic: `chmod +x`, `u+rw`) +* [x] **Can do confidently** | Change file ownership with `chown` and `chgrp` +* [ ] **Need to revisit** | Create and manage LVM volumes (PV, VG, LV physical-to-logical lifecycle) +* [x] **Can do confidently** | Check network connectivity β€” `ping`, `curl -Iv`, `dig +short`, `ip addr`, `ss -tulpn` +* [x] **Can do confidently** | Explain DNS resolution, IP addressing, subnets, CIDR blocks, and standard ports (22, 80, 443) + +### B. Shell Scripting & Automation +* [x] **Can do confidently** | Write a script with shebang (`#!/bin/bash`), variables, positional arguments (`$1`, `$2`), and user input (`read -p`) +* [x] **Can do confidently** | Use `if/elif/else` and `case` statements for conditional routing +* [x] **Can do confidently** | Write `for`, `while`, and `until` loops for batch operations +* [x] **Can do confidently** | Define and call modular functions with arguments and return exit codes +* [ ] **Need to revisit** | Advanced text processing with `awk`, `sed`, `sort`, `uniq` for structured log extraction +* [x] **Can do confidently** | Handle script failures defensively with `set -e`, `set -u`, `set -o pipefail`, and `trap` +* [x] **Can do confidently** | Schedule automation scripts with `crontab -e` + +### C. Git & GitHub Architecture +* [x] **Can do confidently** | Initialize a repo, stage (`git add`), commit, and inspect logs (`git log --oneline --graph`) +* [x] **Can do confidently** | Create, switch, and delete branches (`git checkout -b`, `git branch -d`) +* [x] **Can do confidently** | Push to and pull from remote tracking repositories (`origin`, `tws`) +* [x] **Can do confidently** | Explain clone (local copy of remote) vs fork (server-side copy under your namespace) +* [x] **Can do confidently** | Merge branches β€” understand fast-forward merges vs three-way merge commits +* [ ] **Need to revisit** | Rebase a branch (`git rebase main`) vs merge, and interactive squash workflows +* [x] **Can do confidently** | Use `git stash` and `git stash pop` to preserve work-in-progress state +* [x] **Can do confidently** | Cherry-pick specific commits across branches (`git cherry-pick `) +* [x] **Can do confidently** | Explain squash merge (combining multiple feature commits into one) vs regular merge +* [x] **Can do confidently** | Use `git reset` (`--soft`, `--mixed`, `--hard`) and `git revert` safely +* [x] **Can do confidently** | Explain GitFlow, GitHub Flow, and Trunk-Based Development +* [x] **Can do confidently** | Use GitHub CLI (`gh`) to manage repos, clones, deletions, and pull requests headless + +--- + +## Task 2: Revisit Weak Spots (My Hands-On Remediation) + +### Weak Spot 1: Logical Volume Management (LVM) +* **What tripped me up initially:** The three-tier abstraction (`PV -> VG -> LV`) felt abstract compared to mounting a regular drive. I had trouble visualizing why we need a Volume Group in between. +* **How I practiced and fixed it:** + I walked through the entire creation and live-resizing workflow step-by-step: + ```bash + # 1. Initialize physical disk + sudo pvcreate /dev/sdb + # 2. Pool physical disks into a Volume Group + sudo vgcreate app_vg /dev/sdb + # 3. Carve out a 5GB slice for application data + sudo lvcreate -L 5G -n app_data app_vg + # 4. Format and verify + sudo mkfs.ext4 /dev/app_vg/app_data + # 5. Live volume extension without unmounting active services + sudo lvextend -L +2G /dev/app_vg/app_data + sudo resize2fs /dev/app_vg/app_data + ``` +* **My takeaway:** Standard disk partitions are rigid wallsβ€”expanding them requires taking servers offline. LVM acts like liquid storage; you can pour more disk space into a Volume Group and expand the Logical Volume on a running system without breaking services. + +### Weak Spot 2: Advanced Text Parsing (`awk` & `sed`) +* **What tripped me up initially:** While `grep` is intuitive for matching single error lines, using `awk` column syntax (`$1`, `$9`) and `sed` stream substitution felt unnatural without referencing documentation. +* **How I practiced and fixed it:** + I practiced isolating specific telemetry fields on web server access logs: + ```bash + # Extract unique client IPs and rank them by frequency + awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -n 5 + + # In-place search and replace across config files without opening nano + sed -i 's/127.0.0.1:5432/db.internal:5432/g' config.env + ``` +* **My takeaway:** `grep` is a filter, but `awk` is a data processor. During production log triage, combining `awk '{print $col}' | sort | uniq -c` isolates spikes and bad actors much faster than reading logs line-by-line. + +### Weak Spot 3: Git Rebase vs. Merge in Real Workflows +* **What tripped me up initially:** Handling merge conflicts during a `git merge` is straightforward because Git creates a merge commit. During `git rebase`, conflicts occur commit-by-commit, which caused confusion when resolving and continuing. +* **How I practiced and fixed it:** + I practiced resolving step-by-step rebase conflicts on test branches: + ```bash + git checkout feature/script + git fetch origin + git rebase origin/main + # When conflict paused execution: + # 1. Manually resolve conflict markers in the file + # 2. Stage the resolution: git add + # 3. Continue the replay: git rebase --continue + ``` +* **My takeaway:** Merge preserves actual history as it happened, but leaves non-linear branch knots. Rebase unplugs your commits, updates the base to the latest commit, and replays your work on top. Rule of thumb: Rebase personal feature branches before PR submission; never rebase shared branches like `main`. + +--- + +## Task 3: Quick-Fire Questions (Practical Experience & Mental Models) + +### 1. What does `chmod 755 script.sh` do? +It assigns octal permission bits across Owner, Group, and Others: +* **7 (4+2+1) for Owner:** Full Read, Write, and Execute (`rwx`). The author can modify and run the script. +* **5 (4+0+1) for Group:** Read and Execute (`r-x`). Team members can view and run it, but cannot overwrite it. +* **5 (4+0+1) for Others:** Read and Execute (`r-x`). Any user on the OS can execute it without altering code. + +### 2. What is the difference between a process and a service? +* **Process:** Any single execution instance of a program currently running in RAM with an assigned PID (e.g., executing `./script.sh` or running `top`). If the terminal session ends, the process terminates unless daemonized. +* **Service:** A managed daemon running persistently in the background, supervised by `systemd` (PID 1). It includes automatic restart rules, dependency ordering, boot enablement, and centralized logging. + +### 3. How do you find which process is using port 8080? +Run socket statistics filtering for active listeners: +```bash +sudo ss -tulpn | grep :8080 +# Alternative: +sudo lsof -i :8080 +``` +This shows the exact Process Name and PID holding the socket, allowing you to gracefully terminate it (`kill -15 `) if it blocks another deployment. + +### 4. What does `set -euo pipefail` do in a shell script? +This is defensive bash scripting to prevent silent automation failures in production: +* `-e`: Immediately terminates the script if any command exits with a non-zero status. +* `-u`: Crashes execution if an uninitialized variable is referenced (prevents accidental disasters like `rm -rf /$UNSET_VAR`). +* `-o pipefail`: Prevents masked pipeline errors. In `cmd1 | cmd2`, bash normally only checks the exit code of `cmd2`. With `pipefail`, if `cmd1` fails, the pipeline fails. + +### 5. What is the difference between `git reset --hard` and `git revert`? +* `git reset --hard `: Moves the branch pointer back in time, discarding all staged and unstaged work from that commit forward. It rewrites Git history destructively. +* `git revert `: Leaves existing commit history intact and generates a brand-new commit that applies the exact inverse diff of the target commit. It is safe for shared public branches. + +### 6. What branching strategy would you recommend for a team of 5 developers shipping weekly? +**GitHub Flow:** +A team of 5 developers shipping weekly does not need the complexity of long-lived `release`, `develop`, and `hotfix` branches found in GitFlow. Maintain a single deployable `main` branch. Developers cut short-lived feature branches (`feature/auth`), open PRs with automated checks, get peer reviews, and merge directly into `main`. + +### 7. What does `git stash` do and when would you use it? +`git stash` stores modified, uncommitted work (both staged and unstaged) in temporary storage and cleans the working tree back to `HEAD`. +* **When I use it:** While midway through editing a script, an urgent production bug requires an immediate branch switch. Running `git stash` shelves uncommitted work, allowing a clean checkout to fix the bug, followed by `git stash pop` upon returning. + +### 8. How do you schedule a script to run every day at 3 AM? +Open the cron configuration with `crontab -e` and append: +```cron +0 3 * * * /bin/bash /home/ubuntu/scripts/backup.sh >> /var/log/backup.log 2>&1 +``` +*(Syntax: 0th minute, 3rd hour, any day of month, any month, any day of week)* + +### 9. What is the difference between `git fetch` and `git pull`? +* `git fetch`: Queries the remote repository and downloads new commits and branch refs to your local tracking references (`origin/main`), without modifying your local working files. +* `git pull`: Executes `git fetch` and immediately follows it with a `git merge`, pulling changes directly into your active branch. + +### 10. What is LVM and why would you use it instead of regular partitions? +LVM (Logical Volume Manager) adds an abstraction layer over raw storage, grouping physical disks into a shared Volume Group. +* **Why use it:** Standard disk partitions are hard boundaries. If a partition fills up, expanding it requires shutting down the server and repartitioning. LVM allows expanding a logical volume and its filesystem live with zero server downtime. + +--- + +## Task 4: Work Organization & Showcase Verification + +* [x] **Chronological Commits:** Days 01 through 27 organized and committed inside `2026/` in chronological order. +* [x] **Git Reference Guide:** Centralized `git-commands.md` reference maintained. +* [x] **Shell Automation Portfolio:** Bash scripts migrated into the dedicated `shell-scripts` showcase repository. +* [x] **Profile Hygiene:** Redundant practice repos removed via `gh repo delete`, and GitHub profile pinned repositories curated. + +--- + +## Task 5: Teach It Back + +### Topic: File Permissions Explained to a New Linux User +Think of your Linux computer like an office building where every file is a confidential folder inside a room. Linux assigns three basic permission keys to that folder: +1. **Read (r):** You can open the folder and read what is written on the paper. +2. **Write (w):** You can grab a pen and add, edit, or scribble over the pages. +3. **Execute (x):** If the paper contains step-by-step instructions (a script), you have the badge required to step up and run it. + +Linux asks: **Who are you?** +* **The Owner (User):** The person who created the document. +* **The Department (Group):** The immediate colleagues sitting in the same office team. +* **The Public (Others):** A visitor walking in from the street. + +When you see `chmod 755 backup.sh`, it means: +* The Owner has full keys (`7` = read + write + execute). +* The Team and the Public have restricted keys (`5` = read + execute, but **cannot edit** the text). + +--- + +## Milestone Verdict +Foundational systems engineering, shell scripting, Git hygiene, and terminal troubleshooting consolidation complete. Ready for Docker containerization modules. diff --git a/2026/day-29/day-29-docker-basics.md b/2026/day-29/day-29-docker-basics.md new file mode 100644 index 0000000000..175932c611 --- /dev/null +++ b/2026/day-29/day-29-docker-basics.md @@ -0,0 +1,230 @@ +# Day 29: Introduction to Docker & Container Mechanics + +**Author:** Rashid Khan +**Track:** #90DaysOfDevOps (2026) +**Milestone:** Day 29 Task Documentation & Hands-on Lab Log + +--- + +## Task 1: What is Docker? + +### 1. What is a container and why do we need them? +Before running containers on my EC2 instance today, I thought containers were just stripped-down virtual machines. Seeing them live completely changed my mental model: + +* **What it is in my own words:** A container is simply an isolated, sandboxed Linux process running directly on the host machine's kernel. It bundles my application code along with its exact dependencies, system libraries, and runtime configs so that nothing from the outside host can tamper with it, and it cannot leak outside unless I explicitly map ports or storage. +* **Why we need them:** During my earlier days working with Linux services, package versions constantly conflicted. If my host machine had one version of a library and an application required an older release, system utilities would break. Containers eliminate the classic excuse: *"it works fine on my local machine, why is production crashing?"* If an image runs on my machine, it runs identically on an EC2 server or a staging cluster. +* **Under the Hood (Linux Kernel Features):** Docker doesn't emulate hardware; it relies on two native Linux subsystems: + * **Namespaces (Isolation Walls):** Restricts what the process can *see*. The PID namespace gives the container its own isolated process tree (making its entrypoint process PID 1). The NET namespace provides private virtual network interfaces and routing tables. The MNT namespace isolates filesystem mount points. + * **Control Groups / cgroups (Resource Limits):** Restricts what the process can *use*. It prevents a memory leak or an unconstrained CPU spike inside a container from starving the host EC2 instance. + +### 2. Containers vs. Virtual Machines β€” What's the Real Difference? +Here is how I distinguish them based on my hands-on setup: + +| Feature | Virtual Machines (What I used previously) | Containers / Docker (What I ran today) | +| :--- | :--- | :--- | +| **Architecture** | Relies on a Hypervisor (Type 1 or 2) to emulate physical hardware | Runs directly on Host OS via the Docker container engine | +| **Kernel / OS** | Packs an entire Guest OS (boots its own dedicated kernel) | Shares the host machine's Linux kernel directly | +| **Boot Latency** | 1 to 3 minutes (virtual BIOS boot, kernel init, systemd startup) | Sub-second / milliseconds (it just spawns a regular process) | +| **Resource Usage** | Static memory/CPU reservation upfront, whether used or idle | Consumes only active memory/CPU cycles on demand | +| **Disk Footprint** | 10 GB to 50 GB disk images (`.iso`, `.vmdk`) | 10 MB to 200 MB lightweight layered OCI images | +| **Performance** | Performance penalty due to hardware instruction emulation | Near-native bare-metal execution speeds directly on host CPU | + +### 3. Docker Architecture in My Own Words +The Docker architecture is built on a client-server model consisting of 5 distinct components: + +* **Docker Client (`docker` CLI):** The command-line interface on my terminal where I type commands like `docker run` or `docker ps`. It doesn't actually create containers; it just packages my request into a REST API call. +* **Docker Daemon (`dockerd`):** The background service constantly listening on `/var/run/docker.sock`. When the client speaks, the daemon does the heavy lifting: it coordinates with `containerd`/`runc`, builds network bridges, pulls image layers, and manages process life. +* **Docker Image:** A read-only snapshot or blueprint. It is immutable and built out of read-only stacked filesystem layers. +* **Docker Container:** An active, running instance spawned from an image. Docker adds a thin read/write layer on top of the image's read-only layers so the container can write temporary files and runtime logs. +* **Docker Registry:** The distribution library where images live. By default, my daemon pulled from **Docker Hub**, but in production setups, private registries like AWS ECR are used. + +```text + [ MY EC2 TERMINAL ] + Docker Client (CLI) + | + | (REST API via /var/run/docker.sock) + v + [ DOCKER DAEMON (dockerd) ] <======== Pulls Image ========> [ DOCKER REGISTRY ] + - Listens on UNIX socket (Docker Hub) + - Manages Namespaces & Cgroups + | + |-- Spawns & Manages --> [ CONTAINER: test-nginx ] (Process + R/W Layer) + |-- Spawns & Manages --> [ CONTAINER: my-web-server ] (Host:8080 -> Cont:80) + |-- Stores & Caches --> [ IMAGES: ubuntu, nginx, hello-world ] +``` + +--- + +## Task 2: Install Docker & Verification + +### 1. Installation Workflow on EC2 +I set up the Docker engine on my Ubuntu EC2 terminal, started the daemon, and handled group permissions so I wouldn't need to prefix every command with `sudo`: + +```bash +# 1. Update package database and install Docker engine +sudo apt update +sudo apt install -y docker.io + +# 2. Start Docker service and enable it across reboots +sudo systemctl enable --now docker + +# 3. Add my 'ubuntu' user to the docker group +sudo usermod -aG docker $USER + +# 4. Activate group membership without logging out +newgrp docker + +# 5. Verify the version +docker --version +``` +*Output confirmed:* Docker Engine version verified and active on Ubuntu host. + +### 2. Running `hello-world` & Deconstructing the Trace +I executed: +```bash +docker run hello-world +``` + +**What the Output Explained (Step-by-Step Under the Hood):** +1. My CLI client contacted the local Docker daemon at `/var/run/docker.sock`. +2. The daemon checked local storage for `hello-world:latest`, found nothing, and output: + `Unable to find image 'hello-world:latest' locally`. +3. The daemon queried Docker Hub, pulled the image manifest and layers, and verified the hash digest. +4. The daemon built an isolated container execution space using Linux namespaces and invoked the greeting binary inside the image. +5. The binary printed the *"Hello from Docker!"* confirmation block to stdout, sent it back through the client to my terminal, and the process exited immediately. + +--- + +## Task 3: Run Real Containers (Hands-on Execution & Triage) + +### 1. Running an Ubuntu Container in Interactive Mode (`-it`) +To experience container isolation firsthand, I launched a minimal Ubuntu container: +```bash +docker run -it ubuntu bash +``` + +**My Observations Inside the Container Shell:** +* **Isolated Environment:** Running `cat /etc/os-release` showed Ubuntu inside the container, regardless of my host customizations. +* **Private Hostname:** My terminal prompt switched to `root@009638b9df2a:/#`. The hostname matched the unique container ID hash, not my EC2 private IP. +* **Process Tree Isolation:** When I ran `ps aux`, there was no `systemd`, no `cron`, and no host processes visibleβ€”only PID 1 (which was `/bin/bash`) and my own `ps` command. +* **Stopping Mechanism:** As soon as I typed `exit`, PID 1 terminated, which automatically stopped the container because a container only lives as long as its main foreground process. + +### 2. Running Nginx & Real-World Port Conflict Triage +Next, I attempted to spin up a web server exposed on standard HTTP port 80: +```bash +docker run -d -p 80:80 --name test-nginx nginx +``` + +**The Error I Encountered:** +```text +docker: Error response from daemon: failed to bind host port 0.0.0.0:80/tcp: address already in use +``` + +**My Diagnosis & Fix:** +1. I realized my EC2 host was already running a native Nginx service from my previous Week 1 Linux systemd practice. +2. I removed the dead container state first so the name wouldn't collide later: + ```bash + docker rm test-nginx + ``` +3. I audited port 80 on the host to confirm who was occupying it: + ```bash + sudo ss -tulpn | grep :80 + ``` +4. I stopped and disabled the host systemd service to free socket 80: + ```bash + sudo systemctl stop nginx + sudo systemctl disable nginx + ``` +5. I re-ran the container: + ```bash + docker run -d -p 80:80 --name test-nginx nginx + ``` +6. **Verification:** Ran `curl -I http://localhost` and got a clean `HTTP/1.1 200 OK` response. + +### 3. Container Lifecycle Management Commands +I practiced tracking states and cleaning up stopped containers: + +```bash +# View only actively running containers +docker ps + +# View all containers including stopped ones (hello-world, ubuntu, test-nginx) +docker ps -a + +# Gracefully terminate the running nginx container (sends SIGTERM) +docker stop test-nginx + +# Permanently delete the container and its writable scratch layer +docker rm test-nginx + +# Verify cleanup +docker ps -a +``` + +--- + +## Task 4: Deep Dive Exploration (Flags, Logs & Exec) + +### 1. Detached Mode (`-d`) vs. Foreground Mode +* When I ran `docker run -it ubuntu bash`, my terminal was locked inside the container shell. +* When I ran `docker run -d ...`, Docker decoupled the execution from my shell, printed the 64-character long container ID, and returned my shell prompt immediately. The container ran quietly as a background daemon process. + +### 2. Custom Naming (`--name`) & Port Mapping (`-p host:container`) +I launched an Nginx instance on non-standard port 8080 with an identifiable custom name: +```bash +docker run -d -p 8080:80 --name my-web-server nginx +``` +* **Why `--name my-web-server` matters:** Instead of dealing with random autogenerated names like `flamboyant_morse`, I can target my container directly in scripts using `my-web-server`. +* **Why `-p 8080:80` matters:** The container listens on internal port 80. Docker iptables/nftables rules forward traffic hitting the host's port 8080 into the container's private port 80. + +### 3. Inspecting Container Logs (`docker logs`) +Because detached containers don't dump text onto the terminal, debugging requires inspecting the container's stdout/stderr stream: +```bash +# Dump historical logs +docker logs my-web-server + +# Inspect the last 10 log entries with exact timestamps +docker logs --tail 10 -t my-web-server + +# Live stream incoming HTTP access logs in real-time (like tail -f) +docker logs -f my-web-server +``` + +### 4. Running Commands Inside a Running Container (`docker exec`) +Instead of stopping a container or rebuilding an image to make a quick check, I used `docker exec` to inject commands directly into its running namespace: + +* **One-off diagnostic check:** + ```bash + docker exec my-web-server nginx -v + ``` +* **Interactive Shell & Live Web Page Modification:** + ```bash + docker exec -it my-web-server bash + ``` + Inside the container shell, I attempted to rewrite the index file: + ```bash + echo "

Hello from Rashid's Docker Container (Day 29)

" > /usr/share/nginx/html/index.htm + exit + ``` +* **The Typo & Troubleshooting Lesson:** + When I tested with `curl http://localhost:8080`, I still saw the default *"Welcome to nginx!"* page! + *Root Cause:* I accidentally created `index.htm` instead of `index.html`. Nginx's internal routing checks `index.html` first. + *The Direct Fix:* I fixed it cleanly from the outside using `docker exec`: + ```bash + docker exec my-web-server bash -c 'echo "

Hello from Rashid Docker Container (Day 29)

" > /usr/share/nginx/html/index.html' + ``` + Testing again with `curl http://localhost:8080` immediately returned my custom heading: + `

Hello from Rashid Docker Container (Day 29)

`. + +### 5. Final Lab Teardown +```bash +docker stop my-web-server +docker rm my-web-server +``` + +--- + +## Key Engineering Takeaways from Day 29 +1. **Containers are isolated processes, not hardware:** Seeing `ps aux` inside Ubuntu having only PID 1 proved that containers simply share the host kernel under strict namespace boundaries. +2. **Port binding requires vigilance:** A container cannot bind to a host port if a native daemon or another container already owns that socket. Triage tools (`ss -tulpn`) remain essential. +3. **Immutability vs State:** Any changes made inside a running container using `docker exec` live only in the temporary read/write layer. Once I run `docker rm`, those manual edits are destroyed. To make permanent changes, I will need Dockerfiles. diff --git a/2026/day-30/01-image-size-comparison.png b/2026/day-30/01-image-size-comparison.png new file mode 100644 index 0000000000..99f9f72beb Binary files /dev/null and b/2026/day-30/01-image-size-comparison.png differ diff --git a/2026/day-30/02-nginx-image-layers.png b/2026/day-30/02-nginx-image-layers.png new file mode 100644 index 0000000000..864ca1460f Binary files /dev/null and b/2026/day-30/02-nginx-image-layers.png differ diff --git a/2026/day-30/03-container-lifecycle-states.png b/2026/day-30/03-container-lifecycle-states.png new file mode 100644 index 0000000000..5dfb6b052d Binary files /dev/null and b/2026/day-30/03-container-lifecycle-states.png differ diff --git a/2026/day-30/04-live-triage.png b/2026/day-30/04-live-triage.png new file mode 100644 index 0000000000..bd4929af90 Binary files /dev/null and b/2026/day-30/04-live-triage.png differ diff --git a/2026/day-30/05-docker-system-cleanup.png b/2026/day-30/05-docker-system-cleanup.png new file mode 100644 index 0000000000..8bd6a2f9a3 Binary files /dev/null and b/2026/day-30/05-docker-system-cleanup.png differ diff --git a/2026/day-30/day-30-images.md b/2026/day-30/day-30-images.md new file mode 100644 index 0000000000..8274a5d685 --- /dev/null +++ b/2026/day-30/day-30-images.md @@ -0,0 +1,42 @@ +# Day 30: Docker Images, Layers & Container Lifecycle + +Today, I deep-dived into the core of how Docker images are built, how they share space, and how to triage running containers directly from the host. + +### Task 1: Docker Images & Size Comparison +**Action:** Pulled `nginx`, `ubuntu`, and `alpine` images to compare their footprints. Investigated an image using `docker inspect` and forcefully removed a base image locked by a dead container (`docker rm -f` followed by `docker rmi`). +**Observation:** `alpine` is ~13MB compared to `ubuntu` (~160MB) because Alpine strips out heavy `glibc` libraries for a minimal `musl libc` environment, drastically reducing the attack surface. + +![Task 1 - Image Size Comparison](./01-image-size-comparison.png) + +### Task 2: Unpacking Image Layers +**Action:** Ran `docker image history nginx` to inspect the internal architecture of a Docker image. +**Observation:** Images are stacked Lego blocks. The base OS (`Debian 13 trixie`) took 87.5MB, and the main Nginx installation took 95.9MB. The `0B` layers are just metadata instructions (ENV, EXPOSE). Docker caches and shares these heavy OS layers across different containers to save disk space and network bandwidth. + +![Task 2 - Nginx Image Layers](./02-nginx-image-layers.png) + +### Task 3: The Container Lifecycle +**Action:** Manually transitioned a container through its complete lifecycle: +1. `docker create` -> **Created** (Ready on disk, no CPU/RAM used) +2. `docker start` -> **Up** (Process actively running) +3. `docker pause` -> **Paused** (Frozen in RAM, 0% CPU usage) +4. `docker stop` -> **Exited (0)** (Graceful shutdown via `SIGTERM`) +5. `docker kill` -> **Exited (137)** (Force killed via `SIGKILL`) + +![Task 3 - Container Lifecycle States](./03-container-lifecycle-states.png) + +### Task 4: Live Triage of a Running Container +**Action:** Deployed a detached Nginx container (`docker run -d -p 8080:80`) and executed live triage: +- Watched live traffic logs using `docker logs -f` and caught a standard 404 for a missing `favicon.ico`. +- Extracted the internal Docker bridge IP (`172.17.0.2`) using `docker inspect`. +- Ran commands inside the container without entering the shell: `docker exec live-web cat /etc/os-release`. + +![Task 4 - Live Triage & Exec](./04-live-triage.png) + +### Task 5: System Cleanup +**Action:** Automated the cleanup of the Docker environment to prevent disk space hoarding on the host machine. +- Stopped all running processes dynamically: `docker stop $(docker ps -q)` +- Removed all dead containers: `docker rm $(docker ps -aq)` +- Nuked all unused layers, caches, and images: `docker system prune -a --volumes` +**Result:** Successfully reclaimed 139.2MB of disk space. + +![Task 5 - Docker System Cleanup](./05-docker-system-cleanup.png) diff --git a/2026/day-31/01-first-image-build.png b/2026/day-31/01-first-image-build.png new file mode 100644 index 0000000000..1c572ae38c Binary files /dev/null and b/2026/day-31/01-first-image-build.png differ diff --git a/2026/day-31/02-dockerfile-instructions.png b/2026/day-31/02-dockerfile-instructions.png new file mode 100644 index 0000000000..a258db985b Binary files /dev/null and b/2026/day-31/02-dockerfile-instructions.png differ diff --git a/2026/day-31/03-cmd-vs-entrypoint.png b/2026/day-31/03-cmd-vs-entrypoint.png new file mode 100644 index 0000000000..7814e88c07 Binary files /dev/null and b/2026/day-31/03-cmd-vs-entrypoint.png differ diff --git a/2026/day-31/04-nginx-webapp-browser.png b/2026/day-31/04-nginx-webapp-browser.png new file mode 100644 index 0000000000..5a86674389 Binary files /dev/null and b/2026/day-31/04-nginx-webapp-browser.png differ diff --git a/2026/day-31/04-nginx-webapp-cli.png b/2026/day-31/04-nginx-webapp-cli.png new file mode 100644 index 0000000000..b35da31252 Binary files /dev/null and b/2026/day-31/04-nginx-webapp-cli.png differ diff --git a/2026/day-31/05-dockerignore-verify.png b/2026/day-31/05-dockerignore-verify.png new file mode 100644 index 0000000000..e9308bd4ee Binary files /dev/null and b/2026/day-31/05-dockerignore-verify.png differ diff --git a/2026/day-31/06-build-cache-opt.png b/2026/day-31/06-build-cache-opt.png new file mode 100644 index 0000000000..7dfacdbd91 Binary files /dev/null and b/2026/day-31/06-build-cache-opt.png differ diff --git a/2026/day-31/day-31-dockerfile.md b/2026/day-31/day-31-dockerfile.md new file mode 100644 index 0000000000..0a3fee28fa --- /dev/null +++ b/2026/day-31/day-31-dockerfile.md @@ -0,0 +1,122 @@ +# Day 31: Writing Production-Grade Dockerfiles & Layer Optimization & and build custom images. + +Today's session was an incredible hands-on experience on my EC2 instance. Instead of just pulling pre-built images, I wrote my own custom Dockerfiles from scratch. I made mistakes (like adding rogue spaces in package names), debugged them using Vim, and finally understood how Docker layer caching actually works in a production setup. + +Below is the complete breakdown of all 6 tasks I executed today. + +## Task 1: Building My First Custom Dockerfile +The first task was about setting a solid foundation. I built a custom Ubuntu image, installed `curl`, and configured it to output a custom startup message. + +**Dockerfile:** +```dockerfile +FROM ubuntu +RUN apt-get update && apt-get install -y curl +CMD ["echo", "Hello from my custom image, Rashid!"] +``` + +**Execution & Proof:** +```bash +$ docker build -t my-ubuntu:v1 . +$ docker run my-ubuntu:v1 +Hello from my custom image, Rashid! +``` +![First Image Build](./01-first-image-build.png) + +--- + +## Task 2: Mastering Dockerfile Instructions +Here, I utilized core Dockerfile instructions (`FROM`, `WORKDIR`, `COPY`, `RUN`, `EXPOSE`, `CMD`) to construct an Alpine-based image. + +*Learning moment:* Initially, I mistakenly typed `apk add --no-cache python 3` (with a space). The package manager got confused and threw a `3 (no such package)` error. I quickly realized package names must be exact (`python3`) and fixed it via Vim on the terminal. + +**Dockerfile:** +```dockerfile +FROM alpine:latest +WORKDIR /app +COPY host-data.txt . +RUN apk add --no-cache python3 +EXPOSE 8000 +CMD ["cat", "host-data.txt"] +``` + +**Execution & Proof:** +```bash +$ docker run task2-image:v1 +This file was successfully copied from the host machine into the Docker container. +``` +![Dockerfile Instructions](./02-dockerfile-instructions.png) + +--- + +## Task 3: The Ultimate Interview Question - CMD vs ENTRYPOINT +In this task, I practically tested the difference between these two critical instructions: +* **`CMD`** acts as a polite default argumentβ€”if you pass a new command during `docker run`, it completely overrides the default. +* **`ENTRYPOINT`** acts like a strict ruleβ€”it cannot be easily overridden; instead, any extra arguments passed during runtime are appended to it. + +**Execution (CMD Override vs ENTRYPOINT Append):** +```bash +# Bypassing CMD +$ docker run test-cmd echo "Wait, I just bypassed the CMD!" +Wait, I just bypassed the CMD! + +# Appending to ENTRYPOINT +$ docker run test-entrypoint "You cannot bypassme, I will append this!" +Strict Entrypoint says: You cannot bypassme, I will append this! +``` +![CMD vs ENTRYPOINT](./03-cmd-vs-entrypoint.png) + +--- + +## Task 4: Deploying "DataPilot - Cluster Monitor" (Interactive Web App) +Simple text prints are boring, so I built a futuristic, dark-themed, hacker-style interactive dashboard (`index.html`) named after my DataPilot project. It features real-time CPU load and network traffic updates using CSS/JS. I mounted this onto an `nginx:alpine` image and deployed it live on EC2 port 8080. + +**Execution:** +```bash +$ docker run -d -p 8080:80 my-website:v1 +``` + +**Terminal Proof:** +![Nginx WebApp CLI](./04-nginx-webapp-cli.png) + +**Live Browser View:** +![Nginx WebApp Dashboard](./04-nginx-webapp-browser.png) + +--- + +## Task 5: Securing the Build with .dockerignore +To optimize and secure the build context, I implemented a `.dockerignore` file. It successfully acted as a shield, preventing intentionally created heavy folders (`node_modules`), hidden git configs (`.git`), and secret files (`secret.env`) from ever reaching the Docker daemon. + +**Verification:** +```bash +$ docker run test-ignore:v1 +total 16 +drwxr-xr-x 1 root root 4096 Sep 7 12:48 . +drwxr-xr-x 1 root root 4096 Sep 7 12:48 .. +-rw-rw-r-- 1 root root 31 Sep 7 12:45 .dockerignore +-rw-rw-r-- 1 root root 60 Sep 7 12:47 Dockerfile +-rw-rw-r-- 1 root root 0 Sep 7 12:42 app.js +``` +![Dockerignore Verification](./05-dockerignore-verify.png) + +--- + +## Task 6: Build Optimization & Layer Caching Magic +This is where I learned how to write production-grade Dockerfiles. By placing infrequently changed layers (like OS packages and library installations) at the TOP, and frequently changing layers (like source code) at the BOTTOM, I optimized the build time. + +When I modified the source code and triggered a second build, Docker skipped re-downloading the heavy installations and instantly pulled from the cache (`---> Using cache`), dropping the build time to milliseconds! + +**Second Build Output Proof:** +```bash +Step 3/5 : RUN apk add --no-cache curl + ---> Using cache + ---> d8cdcd5bc977 +``` +![Build Optimization Cache](./06-build-cache-opt.png) + +--- + +## Top 3 Key Learnings + +1. **Debugging is Part of the Process:** I experienced firsthand how a tiny syntax error (like `python 3` instead of `python3`) can halt a build. Fixing it directly via Vim on the EC2 terminal reinforced my Linux command-line confidence. +2. **CMD vs ENTRYPOINT is No Longer Confusing:** Seeing `ENTRYPOINT` append my string while `CMD` got completely wiped out by my custom command made this classic interview concept permanently stick in my head. +3. **Order Matters for Caching:** Docker layers are like a stack. Changing anything invalidates all layers below it. Putting `RUN apt-get update` at the top and `COPY . .` at the very bottom is the ultimate secret to lightning-fast builds. diff --git a/2026/day-31/my-first-image/Dockerfile b/2026/day-31/my-first-image/Dockerfile new file mode 100644 index 0000000000..99e77518d3 --- /dev/null +++ b/2026/day-31/my-first-image/Dockerfile @@ -0,0 +1,3 @@ +FROM ubuntu +RUN apt-get update && apt-get install -y curl +CMD ["echo", "Hello from my custom image, Rashid!"] diff --git a/2026/day-31/task-2-instructions/Dockerfile b/2026/day-31/task-2-instructions/Dockerfile new file mode 100644 index 0000000000..5dc100fa8e --- /dev/null +++ b/2026/day-31/task-2-instructions/Dockerfile @@ -0,0 +1,11 @@ +FROM alpine:latest + +WORKDIR /app + +COPY host-data.txt . + +RUN apk add --no-cache python3 + +EXPOSE 8000 + +CMD ["cat", "host-data.txt"] diff --git a/2026/day-31/task-2-instructions/host-data.txt b/2026/day-31/task-2-instructions/host-data.txt new file mode 100644 index 0000000000..89f8e2d7a1 --- /dev/null +++ b/2026/day-31/task-2-instructions/host-data.txt @@ -0,0 +1 @@ +This file was successfully copied from the host machine into the Docker container. diff --git a/2026/day-31/task-3-cmd-vs-entrypoint/Dockerfile.cmd b/2026/day-31/task-3-cmd-vs-entrypoint/Dockerfile.cmd new file mode 100644 index 0000000000..5b91900fc1 --- /dev/null +++ b/2026/day-31/task-3-cmd-vs-entrypoint/Dockerfile.cmd @@ -0,0 +1,3 @@ +FROM alpine:latest +CMD ["echo", "I am the default CMD message"] + diff --git a/2026/day-31/task-3-cmd-vs-entrypoint/Dockerfile.entrypoint b/2026/day-31/task-3-cmd-vs-entrypoint/Dockerfile.entrypoint new file mode 100644 index 0000000000..c4a8c7ff68 --- /dev/null +++ b/2026/day-31/task-3-cmd-vs-entrypoint/Dockerfile.entrypoint @@ -0,0 +1,4 @@ +FROM alpine:latest + +ENTRYPOINT ["echo", "Strict Entrypoint says:"] + diff --git a/2026/day-31/task-4-webapp/Dockerfile b/2026/day-31/task-4-webapp/Dockerfile new file mode 100644 index 0000000000..d07b33dc47 --- /dev/null +++ b/2026/day-31/task-4-webapp/Dockerfile @@ -0,0 +1,3 @@ +FROM nginx:alpine +COPY index.html /usr/share/nginx/html/ +EXPOSE 80 diff --git a/2026/day-31/task-4-webapp/index.html b/2026/day-31/task-4-webapp/index.html new file mode 100644 index 0000000000..32f53b08fa --- /dev/null +++ b/2026/day-31/task-4-webapp/index.html @@ -0,0 +1,44 @@ + + + + + + DataPilot - Cluster Monitor + + + +
+

πŸš€ DataPilot | Autonomous Cluster Monitor

+

System Status: LIVE - SECURE TUNNEL

+
+
+

CPU Load (EC2)

42%

+

Active AI Agents

7 Nodes

+

Network Traffic

1.2 GB/s

+
+
+ > Initializing DataPilot Core Engine...
+ > Connecting to AWS EC2 instances...
+ > Authentication successful. Tunnel established.
+
+ + + diff --git a/2026/day-31/task-5-dockerignore/.dockerignore b/2026/day-31/task-5-dockerignore/.dockerignore new file mode 100644 index 0000000000..b8966b67e4 --- /dev/null +++ b/2026/day-31/task-5-dockerignore/.dockerignore @@ -0,0 +1,5 @@ +node_modules +.git +*.md +*.env + diff --git a/2026/day-31/task-5-dockerignore/Dockerfile b/2026/day-31/task-5-dockerignore/Dockerfile new file mode 100644 index 0000000000..7a6a092096 --- /dev/null +++ b/2026/day-31/task-5-dockerignore/Dockerfile @@ -0,0 +1,5 @@ +FROM alpine:latest +WORKDIR /app +COPY . . +CMD ["ls", "-la"] + diff --git a/2026/day-31/task-5-dockerignore/README.md b/2026/day-31/task-5-dockerignore/README.md new file mode 100644 index 0000000000..e69de29bb2 diff --git a/2026/day-31/task-5-dockerignore/app.js b/2026/day-31/task-5-dockerignore/app.js new file mode 100644 index 0000000000..e69de29bb2 diff --git a/2026/day-31/task-5-dockerignore/secret.env b/2026/day-31/task-5-dockerignore/secret.env new file mode 100644 index 0000000000..e69de29bb2 diff --git a/2026/day-31/task-6-optimization/Dockerfile b/2026/day-31/task-6-optimization/Dockerfile new file mode 100644 index 0000000000..a55abe8f63 --- /dev/null +++ b/2026/day-31/task-6-optimization/Dockerfile @@ -0,0 +1,9 @@ +FROM alpine:latest +WORKDIR /app + +RUN apk add --no-cache curl + +COPY app.js . + +CMD ["cat", "app.js"] + diff --git a/2026/day-31/task-6-optimization/app.js b/2026/day-31/task-6-optimization/app.js new file mode 100644 index 0000000000..885ee85aa6 --- /dev/null +++ b/2026/day-31/task-6-optimization/app.js @@ -0,0 +1,2 @@ +console.log('This is version 2 with a new feature!'); + diff --git a/2026/day-32/day-32-volumes-networking.md b/2026/day-32/day-32-volumes-networking.md new file mode 100644 index 0000000000..068c90c6dc --- /dev/null +++ b/2026/day-32/day-32-volumes-networking.md @@ -0,0 +1,102 @@ +# Day 32: Docker Volumes & Networking + +## Overview +Today’s deep dive was all about fixing two major real-world problems in the containerized ecosystem: **Data Persistence** and **Container Communication**. By default, containers are ephemeral (data is lost when destroyed) and isolated (they cannot easily resolve each other by name). Here is how I solved both today. + +--- + +## Task 1: The Problem (Ephemeral Containers) +I started by testing the default behavior of containers to see exactly what happens to data when a container is removed. + +**What I did:** +I spun up a Postgres container, jumped into the `psql` shell, and created a demo table with some data. +```bash +docker run --name pg-test -e POSTGRES_PASSWORD=root -d postgres:alpine +docker exec -it pg-test psql -U postgres +# SQL: CREATE TABLE demo (id INT, name TEXT); INSERT INTO demo VALUES (1, 'rashid'); +``` +Then, I forcefully removed the container (`docker rm -f pg-test`), spun up a new one using the exact same image, and tried to fetch my table. + +**The Result & Why:** +I got `ERROR: relation "demo" does not exist`. The data was wiped out completely because it was stored in the container's writable layer, which gets destroyed alongside the container. The ephemeral nature of containers is real! + +![Task 1 Proof](./task-01-container-data-lost.png) + +--- + +## Task 2: Named Volumes (The Solution) +To persist data, I utilized Docker Named Volumes. + +**A real debugging moment:** +Initially, I mounted the volume to `/var/lib/postgresql/data`, but the container instantly crashed. Instead of panicking, I checked the logs (`docker logs pg-vol-test`). The logs clearly told me that for Postgres 18+ images, the mount needs to be directly at `/var/lib/postgresql`. Logs are truly a developer's best friend! + +**The Fix:** +```bash +docker volume create pg-data +docker run --name pg-vol-test -v pg-data:/var/lib/postgresql -e POSTGRES_PASSWORD=root -d postgres:alpine +``` +I recreated the data, destroyed the container, and spun up a brand new one attached to the same `pg-data` volume. This time, the data survived! + +![Task 2 Proof](./task-02-named-volumes.png) + +--- + +## Task 3: Bind Mounts +To understand the difference between Named Volumes and Bind Mounts, I mapped a local directory from my EC2 host directly into an Nginx container. + +```bash +docker run -d --name nginx-web -p 8081:80 -v ~/nginx-bind-test:/usr/share/nginx/html nginx:alpine +``` +**The Difference:** +* **Named Volumes** are managed entirely by Docker (in `/var/lib/docker/volumes/`). They are best for strict data persistence like databases. +* **Bind Mounts** link an exact file/folder path on the host to the container. I updated `index.html` locally on my Ubuntu host, and the changes reflected instantly on the browser without restarting the container. Great for local dev! + +![Task 3 Proof](./task-03-bind-mounts.png) + +--- + +## Task 4: Docker Networking Basics +I inspected the default `bridge` network to understand container isolation. + +**The Experiment:** +I spun up two Alpine containers and tried to ping `container2` from `container1` by its name. +```bash +docker exec container1 ping -c 2 container2 +``` +**Result:** `ping: bad address 'container2'`. +However, when I fetched the exact IP of `container2` and pinged the IP, it worked flawlessly. +**Why?** The default bridge network does not support automatic DNS resolution. Containers can only communicate via IP on the default bridge, which is terrible for production since IPs change. + +![Task 4 Proof](./task-04-networking-basics.png) + +--- + +## Task 5: Custom Networks +To fix the DNS issue, I created a custom bridge network called `my-app-net`. + +```bash +docker network create my-app-net +docker run -d --name my-container1 --network my-app-net alpine sleep 3600 +docker run -d --name my-container2 --network my-app-net alpine sleep 3600 +``` + +**The Magic:** +When I pinged `my-container2` directly by its name, it succeeded instantly with `0% packet loss`. +**Why it works:** Custom Docker networks come with an embedded DNS server. It automatically resolves container names to their IPs, allowing name-based communication. + +![Task 5 Proof](./task-05-custom-network.png) + +--- + +## Task 6: Put It Together +I combined both concepts to simulate a real-world microservice setup: +1. Created a custom network `final-app-net`. +2. Ran a backend DB (Postgres) on this network, attaching a persistent volume to keep data safe. +3. Ran a frontend App (Alpine) on the same network to verify connectivity. + +```bash +docker run --name final-app --network final-app-net -it --rm alpine ping -c 3 final-db +``` +The app container successfully resolved and pinged the database by name. Storage is persistent, and networking is seamless! + +![Task 6 Proof](./task-06-put-it-together.png) diff --git a/2026/day-32/task-01-container-data-lost.png b/2026/day-32/task-01-container-data-lost.png new file mode 100644 index 0000000000..dfa2fa40f7 Binary files /dev/null and b/2026/day-32/task-01-container-data-lost.png differ diff --git a/2026/day-32/task-02-named-volumes.png b/2026/day-32/task-02-named-volumes.png new file mode 100644 index 0000000000..2c6206c8ca Binary files /dev/null and b/2026/day-32/task-02-named-volumes.png differ diff --git a/2026/day-32/task-03-bind-mounts.png b/2026/day-32/task-03-bind-mounts.png new file mode 100644 index 0000000000..93269da3d6 Binary files /dev/null and b/2026/day-32/task-03-bind-mounts.png differ diff --git a/2026/day-32/task-04-networking-basics.png b/2026/day-32/task-04-networking-basics.png new file mode 100644 index 0000000000..ae330c0170 Binary files /dev/null and b/2026/day-32/task-04-networking-basics.png differ diff --git a/2026/day-32/task-05-custom-network.png b/2026/day-32/task-05-custom-network.png new file mode 100644 index 0000000000..f682ea94ad Binary files /dev/null and b/2026/day-32/task-05-custom-network.png differ diff --git a/2026/day-32/task-06-put-it-together.png b/2026/day-32/task-06-put-it-together.png new file mode 100644 index 0000000000..b644e471ac Binary files /dev/null and b/2026/day-32/task-06-put-it-together.png differ diff --git a/2026/day-33/compose-basics/docker-compose.yml b/2026/day-33/compose-basics/docker-compose.yml new file mode 100644 index 0000000000..3f53d3717b --- /dev/null +++ b/2026/day-33/compose-basics/docker-compose.yml @@ -0,0 +1,6 @@ +services: + web: + image: nginx:alpine + ports: + - "8080:80" + diff --git a/2026/day-33/day-33-compose.md b/2026/day-33/day-33-compose.md new file mode 100644 index 0000000000..ba49046758 --- /dev/null +++ b/2026/day-33/day-33-compose.md @@ -0,0 +1,129 @@ +# Day 33 - Docker Compose: Multi-Container Basics + +Yesterday, I spent time manually creating networks, provisioning volumes, and typing out long `docker run` commands to link containers. Today's goal? Automate all of that manual labor into a single YAML file using Docker Compose. + +Here is the log of how I tackled today's challenge, including the hiccups and the real-world fixes. + +--- + +## Task 1: Install & Verify + +Right off the bat, I hit a roadblock. Running `sudo apt-get install docker-compose-plugin` on my Ubuntu EC2 instance failed because the package couldn't be located in the default `apt` repos. + +Instead of getting stuck, I took the manual route. I pulled the latest binary directly from Docker's official GitHub releases using `curl`, gave it execution permissions, and moved it to the CLI plugins folder. + +```bash +mkdir -p ~/.docker/cli-plugins/ +curl -SL [https://github.com/docker/compose/releases/latest/download/docker-compose-linux-$(uname](https://github.com/docker/compose/releases/latest/download/docker-compose-linux-$(uname) -m) -o ~/.docker/cli-plugins/docker-compose +chmod +x ~/.docker/cli-plugins/docker-compose +docker compose version +``` + +**Proof of Installation:** +![Task 1: Install & Verify](./task-01-compose-version.png) + +--- + +## Task 2: Your First Compose File + +Time to test the waters. I created a directory `compose-basics` and wrote my very first `docker-compose.yml`. The goal was simple: spin up an Nginx container and map it to port 8080. + +**`docker-compose.yml`:** +```yaml +services: + web: + image: nginx:alpine + ports: + - "8080:80" +``` + +Spun it up in detached mode using `docker compose up -d`, verified it was serving pages locally with `curl http://localhost:8080`, and then cleanly tore the whole environment down with `docker compose down`. Just like thatβ€”no manual network cleanup needed. + +**Proof of Execution:** +![Task 2: First Compose File](./task-02-first-compose.png) + +--- + +## Task 3: Two-Container Setup (WordPress + MySQL) + +This was the main event. I needed to run WordPress and MySQL together. The beauty of Compose is that I didn't even have to define a network manuallyβ€”Compose automatically created a default bridge network, allowing WordPress to find MySQL simply by its service name (`db`). + +I also configured a named volume (`db_data`) so that even if the containers get destroyed, the database records survive. + +**`docker-compose.yml`:** +```yaml +services: + db: + image: mysql:5.7 + restart: always + environment: + MYSQL_ROOT_PASSWORD: rootpassword + MYSQL_DATABASE: wordpress + MYSQL_USER: wpuser + MYSQL_PASSWORD: wppassword + volumes: + - db_data:/var/lib/mysql + + wordpress: + image: wordpress:latest + restart: always + ports: + - "8081:80" + environment: + WORDPRESS_DB_HOST: db + WORDPRESS_DB_USER: wpuser + WORDPRESS_DB_PASSWORD: wppassword + WORDPRESS_DB_NAME: wordpress + depends_on: + - db + +volumes: + db_data: +``` + +**The Struggle:** After running `up -d`, the containers were healthy, but my browser couldn't reach port 8081. While configuring my AWS EC2 Security Group to allow inbound TCP on 8081, I accidentally caused a broken pipe and locked myself out of SSH! A quick instance reboot and re-adding the port 22 rule got me back in. + +**Persistence Test:** I stopped the environment (`docker compose down`) and brought it back up. My WordPress setup was completely intact thanks to the named volume! + +**Proof of Setup:** +![Task 3: WordPress + MySQL Stack](./task-03-wordpress-mysql.png) + +--- + +## Task 4: Compose Commands + +I spent some time getting familiar with the daily-driver operational commands. Since MySQL spits out a massive amount of logs on initialization, I used the `--tail` flag to keep my terminal clean. + +**Commands practiced:** +* **Start detached:** `docker compose up -d` +* **Check status:** `docker compose ps` +* **View specific logs:** `docker compose logs --tail 10 db` (Pro-tip for avoiding log floods) +* **Stop gracefully:** `docker compose stop` (keeps networks and containers intact) +* **Nuke everything:** `docker compose down` (removes containers and networks, but keeps volumes) + +**Proof of Commands in Action:** +![Task 4: Compose Commands](./task-04-compose-commands.png) + +--- + +## Task 5: Environment Variables + +Hardcoding passwords inside a YAML file is a terrible idea for version control. To fix this, I created a `.env` file in the same directory to hold my secrets securely. + +**`.env` file:** +```env +MYSQL_ROOT_PASSWORD=supersecretroot +MYSQL_DATABASE=wordpress +MYSQL_USER=wpuser +MYSQL_PASSWORD=supersecretwp +``` + +I then updated the `docker-compose.yml` to interpolate these values using `${VARIABLE_NAME}`. Before spinning it up, I ran `docker compose config` to verify that Docker was correctly picking up the variables from the `.env` file. It worked flawlessly. + +*(Note: Added `.env` to `.gitignore` to ensure these secrets never hit the repository).* + +**Proof of Variable Interpolation:** +![Task 5: Environment Variables](./task-05-env-vars.png) + +--- +*Documented as part of the #90DaysOfDevOps Challenge.* diff --git a/2026/day-33/task-01-compose-version.png b/2026/day-33/task-01-compose-version.png new file mode 100644 index 0000000000..b52b911365 Binary files /dev/null and b/2026/day-33/task-01-compose-version.png differ diff --git a/2026/day-33/task-02-first-compose.png b/2026/day-33/task-02-first-compose.png new file mode 100644 index 0000000000..55da86e195 Binary files /dev/null and b/2026/day-33/task-02-first-compose.png differ diff --git a/2026/day-33/task-03-wordpress-mysql.png b/2026/day-33/task-03-wordpress-mysql.png new file mode 100644 index 0000000000..b5defd5ec2 Binary files /dev/null and b/2026/day-33/task-03-wordpress-mysql.png differ diff --git a/2026/day-33/task-04-compose-commands.png b/2026/day-33/task-04-compose-commands.png new file mode 100644 index 0000000000..74dc284a93 Binary files /dev/null and b/2026/day-33/task-04-compose-commands.png differ diff --git a/2026/day-33/task-05-env-vars.png b/2026/day-33/task-05-env-vars.png new file mode 100644 index 0000000000..8bc9284e9c Binary files /dev/null and b/2026/day-33/task-05-env-vars.png differ diff --git a/2026/day-33/wordpress-mysql/.gitignore b/2026/day-33/wordpress-mysql/.gitignore new file mode 100644 index 0000000000..4c49bd78f1 --- /dev/null +++ b/2026/day-33/wordpress-mysql/.gitignore @@ -0,0 +1 @@ +.env diff --git a/2026/day-33/wordpress-mysql/docker-compose.yml b/2026/day-33/wordpress-mysql/docker-compose.yml new file mode 100644 index 0000000000..861e30cb73 --- /dev/null +++ b/2026/day-33/wordpress-mysql/docker-compose.yml @@ -0,0 +1,34 @@ +# Wordpress & MySQL Stack with Persistent Storage +# This compose file sets up a Wordpress frontend connected to a MySQL Backend, ensuring data persistence using named volumes. + + +services: + db: + image: mysql:5.7 + restart: always + environment: + MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD} + MYSQL_DATABASE: ${MYSQL_DATABASE} + MYSQL_USER: ${MYSQL_USER} + MYSQL_PASSWORD: ${MYSQL_PASSWORD} + volumes: + - db_data:/var/lib/mysql + + + wordpress: + image: wordpress:latest + restart: always + ports: + - "8081:80" + environment: + WORDPRESS_DB_HOST: db + WORDPRESS_DB_USER: ${MYSQL_USER} + WORDPRESS_DB_PASSWORD: ${MYSQL_PASSWORD} + WORDPRESS_DB_NAME: ${MYSQL_DATABASE} + depends_on: + - db + + +volumes: + db_data: + diff --git a/2026/day-34/app-stack/Dockerfile b/2026/day-34/app-stack/Dockerfile new file mode 100644 index 0000000000..19f6fefeda --- /dev/null +++ b/2026/day-34/app-stack/Dockerfile @@ -0,0 +1,10 @@ + + +FROM node:18-alpine +WORKDIR /app +COPY package.json ./ +RUN npm install +COPY index.js . +EXPOSE 3000 +CMD ["node", "index.js"] + diff --git a/2026/day-34/app-stack/docker-compose.yml b/2026/day-34/app-stack/docker-compose.yml new file mode 100644 index 0000000000..971632ddaa --- /dev/null +++ b/2026/day-34/app-stack/docker-compose.yml @@ -0,0 +1,53 @@ +# This is Master Orchestration Docker-Compose.yml File + +networks: + day34-net: + driver: bridge + +volumes: + db-data: + +services: + web: + build: . + restart: on-failure + ports: + - "3000:3000" + networks: + - day34-net + labels: + - "tier=frontend" + depends_on: + db: + condition: service_healthy + cache: + condition: service_started + + db: + image: postgres:15-alpine + restart: always + environment: + POSTGRES_USER: postgres + POSTGRES_PASSWORD: password + POSTGRES_DB: testdb + networks: + - day34-net + volumes: + - db-data:/var/lib/postgresql/data + labels: + - "tier=database" + healthcheck: + test: ["CMD-SHELL", "pg_isready -U postgres -d testdb"] + interval: 5s + timeout: 5s + retries: 5 + + cache: + image: redis:alpine + networks: + - day34-net + labels: + - "tier=cache" + + + diff --git a/2026/day-34/app-stack/index.js b/2026/day-34/app-stack/index.js new file mode 100644 index 0000000000..69132d3d73 --- /dev/null +++ b/2026/day-34/app-stack/index.js @@ -0,0 +1,34 @@ +const express = require('express'); +const { Pool } = require('pg'); +const { createClient } = require('redis'); + +const app = express(); + +// Database connection +const pool = new Pool({ + host: 'db', + user: 'postgres', + password: 'password', + database: 'testdb' +}); + +// Redis connection +const redisClient = createClient({ + url: 'redis://cache:6379' +}); + +app.get('/', async (req, res) => { + try { + if (!redisClient.isOpen) await redisClient.connect(); + let visits = await redisClient.get('visits') || 0; + visits = parseInt(visits) + 1; + await redisClient.set('visits', visits); + + const dbRes = await pool.query('SELECT NOW()'); + res.send(`

Day 34: App Updated Successfully!

Visits (from Redis): ${visits}

Database Time (from Postgres): ${dbRes.rows[0].now}

`); + } catch (err) { + res.status(500).send(`Error: ${err.message}`); + } +}); + +app.listen(3000, () => console.log('App running on port 3000')); diff --git a/2026/day-34/app-stack/package.json b/2026/day-34/app-stack/package.json new file mode 100644 index 0000000000..6f45826281 --- /dev/null +++ b/2026/day-34/app-stack/package.json @@ -0,0 +1,10 @@ + + +{ + "name": "day34-app", + "dependencies": { + "express": "^4.19.2", + "pg": "^8.11.3", + "redis": "^4.6.13" + } +} diff --git a/2026/day-34/app-stack/task-01-app-stack.png b/2026/day-34/app-stack/task-01-app-stack.png new file mode 100644 index 0000000000..a1054a1896 Binary files /dev/null and b/2026/day-34/app-stack/task-01-app-stack.png differ diff --git a/2026/day-34/app-stack/task-02-healthchecks.png b/2026/day-34/app-stack/task-02-healthchecks.png new file mode 100644 index 0000000000..81e18cbc79 Binary files /dev/null and b/2026/day-34/app-stack/task-02-healthchecks.png differ diff --git a/2026/day-34/app-stack/task-03-restart-policy.png b/2026/day-34/app-stack/task-03-restart-policy.png new file mode 100644 index 0000000000..9da837f93b Binary files /dev/null and b/2026/day-34/app-stack/task-03-restart-policy.png differ diff --git a/2026/day-34/app-stack/task-04-custom-dockerfile.png b/2026/day-34/app-stack/task-04-custom-dockerfile.png new file mode 100644 index 0000000000..c72cd2be2f Binary files /dev/null and b/2026/day-34/app-stack/task-04-custom-dockerfile.png differ diff --git a/2026/day-34/app-stack/task-05-networks-volumes.png b/2026/day-34/app-stack/task-05-networks-volumes.png new file mode 100644 index 0000000000..a0c789595a Binary files /dev/null and b/2026/day-34/app-stack/task-05-networks-volumes.png differ diff --git a/2026/day-34/app-stack/task-06-scaling-error.png b/2026/day-34/app-stack/task-06-scaling-error.png new file mode 100644 index 0000000000..6f57f83fd5 Binary files /dev/null and b/2026/day-34/app-stack/task-06-scaling-error.png differ diff --git a/2026/day-34/day-34-compose-advanced.md b/2026/day-34/day-34-compose-advanced.md new file mode 100644 index 0000000000..77cdb0d090 --- /dev/null +++ b/2026/day-34/day-34-compose-advanced.md @@ -0,0 +1,42 @@ +# Day 34: Advanced Docker Compose – Real-World Debugging & App Stacks + +Today was all about getting out of the basic Docker zone and building a production-like environment. I typed out the YAML files manually using `vim` to build that muscle memory, hit a few intentional errors, and deep-dived into how Docker engine actually thinks. + +## Task 1: Build Your Own App Stack +Built a 3-tier Node.js stack from scratch. Wrote the `Dockerfile` and `package.json`, and hooked it up with a PostgreSQL database and Redis cache. +Instead of pulling pre-built images for the app, I configured `docker-compose.yml` to build the web container dynamically. +![Task 1 - App Stack](./app-stack/task-01-app-stack.png) + +## Task 2: depends_on & Healthchecks +**The Problem:** Node.js crashes if it tries to connect to Postgres before the DB is fully ready. +**The Fix:** I added a `healthcheck` (`pg_isready`) to the DB service and set the web service to wait for `condition: service_healthy`. +*Terminal Proof:* I literally watched the logs wait. The DB hit `Healthy` at 5.8s, and immediately at 5.9s, the web container started. Perfect synchronization! +![Task 2 - Healthchecks](./app-stack/task-02-healthchecks.png) + +## Task 3: Restart Policies (The Trick Question) +**Q: Manually kill the database container β€” does it come back?** +Nope! I ran `docker kill app-stack-db-1`. When I checked `docker compose ps`, it was gone. I had to run `docker compose ps -a` to catch it sitting in an `Exited (137)` state. +*Why?* Because an explicit `docker kill` command acts as an "Admin Override". Docker assumes that if I explicitly killed it, I don't want it back, completely ignoring the `restart: always` policy. + +**Q: When would you use each restart policy?** +* **`restart: always`**: I'd use this for my core infrastructure (like Postgres or Redis). If the container crashes due to memory issues or the EC2 server reboots, Docker will force it back up. +* **`restart: on-failure`**: Better for background tasks. It only restarts if the exit code is non-zero (an error). If the code finishes cleanly (code 0), it stays stopped. +![Task 3 - Restart Policy](./app-stack/task-03-restart-policy.png) + +## Task 4: Custom Dockerfiles in Compose +I edited `index.js` via `vim` to change the UI text to `

Day 34: App Updated Successfully!

`. +Instead of tearing down the whole DB and Cache, I ran a targeted rebuild: `docker compose up -d --build web`. +Docker's caching magic kicked in (`---> Using cache`), completely skipped the heavy `npm install` step, and pushed my new code live in under 15 seconds. +![Task 4 - Custom Dockerfile](./app-stack/task-04-custom-dockerfile.png) + +## Task 5: Named Networks & Volumes +Time to make data permanent. I dropped the default network and created a custom bridge network (`day34-net`) for better isolation. More importantly, I bound the Postgres data directory to a named volume (`db-data`). Verified it by running `docker volume ls | grep db-data`. Now, even if I wipe the containers, my database records survive. +![Task 5 - Networks & Volumes](./app-stack/task-05-networks-volumes.png) + +## Task 6: Scaling (Bonus) +**Q: What happens? What breaks? Why doesn't simple scaling work with port mapping?** +I tried to push the limits and scale the web tier using `docker compose up -d --scale web=3`. +It instantly threw an error: `Bind for :::3000 failed: port is already allocated`. +*The reason:* I hardcoded `"3000:3000"` in my compose file. The first container `web-1` grabbed port 3000 on my EC2 instance. When `web-2` and `web-3` tried to spin up, they crashed because that exact host port was already locked. +*Lesson:* To properly scale, you have to omit the host port mapping (e.g., just `- "3000"`) and let Docker assign random available ports on the host. +![Task 6 - Scaling Error](./app-stack/task-06-scaling-error.png) diff --git a/2026/day-35/Dockerfile b/2026/day-35/Dockerfile new file mode 100644 index 0000000000..ff72562793 --- /dev/null +++ b/2026/day-35/Dockerfile @@ -0,0 +1,35 @@ + +# ---------- Stage 1: Builder ---------- + +FROM golang:1.20-alpine AS builder + +WORKDIR /app + +# Copy code +COPY main.go . + +# Build the app (CGO_ENABLED=0 is a pro-tip to make it run anywhere without C dependencies) +RUN CGO_ENABLED=0 GOOS=linux go build -o myapp main.go + +# ---------- Stage 2: Production ---------- + +# Best practice 1: Use specific base image tag (not 'latest') +FROM alpine:3.18 + +# Best Practice 2: Create a non-root user and group +RUN addgroup -S appgroup && adduser -S appuser -G appgroup + +WORKDIR /home/appuser/ + +# Copy ONLY the built binary from the 'builder' stage +COPY --from=builder /app/myapp . + +# Best Practice 3: Switch to the non-root user +USER appuser + +# Expose port and run +EXPOSE 8080 + +CMD ["./myapp"] + + diff --git a/2026/day-35/day-35-multistage-hub.md b/2026/day-35/day-35-multistage-hub.md new file mode 100644 index 0000000000..460fd3ed13 --- /dev/null +++ b/2026/day-35/day-35-multistage-hub.md @@ -0,0 +1,174 @@ +# Day 35: Multi-Stage Builds, Docker Hub & Image Security Best Practices + +Today's session was all about moving away from bloated, insecure container images and building production-grade, optimized, and secure microservices. Instead of reading theory, I faced real-world infrastructure constraints (like running out of disk space on an AWS EC2 instance) and solved them directly from the terminal. + +--- + +## Environment & Setup +* Build Server: AWS EC2 (Ubuntu instance with tight storage constraints) +* Verification Environment: Local MacOS (Docker Desktop) +* App Stack: Custom Golang lightweight web server (main.go) + +--- + +## Task 1: The Problem with Large Images (Single-Stage) + +To understand why multi-stage builds are critical in production, I started by writing a simple Go web server and containerizing it using a single-stage approach. + +### The Code (main.go) +```go +package main + +import ( + "fmt" + "net/http" +) + +func main() { + http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, "Hello Day 35! This is a heavy single-stage build.") + }) + fmt.Println("Server starting on port 8080...") + http.ListenAndServe(":8080", nil) +} +``` + +### The Single-Stage Dockerfile +```dockerfile +FROM golang:1.20-alpine +WORKDIR /app +COPY main.go . +RUN go build -o myapp main.go +EXPOSE 8080 +CMD ["./myapp"] +``` + +### Terminal Execution & Reality Check +When I ran the build command: +```bash +docker build -t day35-single-stage . +docker images | grep day35-single-stage +``` +* Real Challenge: Initially, attempting a standard heavyweight Go base image crashed my AWS EC2 free-tier instance with a `no space left on device` error because the extracted filesystem layers consumed all 1.2GB of available free space! After a system prune, I switched to `golang:1.20-alpine`, but the final single-stage image still sat at an alarming 492 MB just to serve a basic "Hello World" response. + +![Task 1 - Large Single Stage Image](./task-01-large-image.png) + +--- + +## Task 2: Multi-Stage Build Optimization + +To slash the image size, I rewrote the Dockerfile into two separate distinct stages: +1. Builder Stage: Uses the heavy Go SDK compiler to build the standalone binary (`CGO_ENABLED=0`). +2. Production Stage: Discards the SDK entirely and copies only the compiled binary into a bare-minimum Alpine OS. + +### The Multi-Stage Dockerfile +```dockerfile +# --------- STAGE 1: Builder --------- +FROM golang:1.20-alpine AS builder +WORKDIR /app +COPY main.go . +RUN CGO_ENABLED=0 GOOS=linux go build -o myapp main.go + +# --------- STAGE 2: Production --------- +FROM alpine:latest +WORKDIR /root/ +COPY --from=builder /app/myapp . +EXPOSE 8080 +CMD ["./myapp"] +``` + +### Build & Comparison +```bash +docker build -t day35-multi-stage . +docker images | grep day35 +``` + +* The Result: The image size dropped from 492 MB down to 7.65 MB (actual unique disk space used). That is a massive 98.4% reduction in size! This drastically cuts down deployment time, security vulnerability surface area, and registry storage costs. + +![Task 2 - Multi Stage Build Size Comparison](./task-02-multi-stage.png) + +--- + +## Task 3: Pushing & Pulling via Docker Hub + +Once the image was optimized, I distributed it via Docker Hub using secure CLI authentication (Personal Access Token). + +### Commands Executed: +```bash +# Authenticate securely using PAT +docker login + +# Tag the image with Docker Hub username +docker tag day35-multi-stage:latest rashidkhan6685/day35-go-app:latest + +# Push to Docker Hub registry +docker push rashidkhan6685/day35-go-app:latest + +# Verify by clearing local cache and pulling back down +docker rmi rashidkhan6685/day35-go-app:latest +docker pull rashidkhan6685/day35-go-app:latest +``` + +* Observation: Verified cross-environment compatibility by pulling the image seamlessly onto my local MacOS environment and inspecting it inside Docker Desktop. (Note: Docker Desktop retains the original build timestamp metadata from EC2, showing the creation history correctly). + +![Task 3 - Docker Hub Push & Pull Terminal Evidence](./task-03-docker-hub-push.png) + +--- + +## Task 4: Docker Hub Repository & Documentation + +Maintained production hygiene by configuring the repository directly on the Docker Hub UI: +* Added a clear, concise short description ("Day 35 #90DaysOfDevOps: Multi-stage Golang image reduced from 492MB to 7.65MB"). +* Verified the tag versioning (`latest`), repository digest, and compressed storage size (7.3 MB). + +![Task 4 - Docker Hub Repository UI](./task-04-docker-hub-repo.png) + +--- + +## Task 5: Security Best Practices (Non-Root User & Pinned Tags) + +By default, Docker containers run as the `root` user, creating a severe vulnerability window if an attacker manages to break out of the container (Container Escape). I applied production hardening rules: + +1. Pinned Base Image Tags: Avoided `latest` in production and used a specific stable version (`alpine:3.18`). +2. Non-Root User Creation: Added system user and group directives inside the Dockerfile. +3. User Switching: Enforced the execution context to shift away from root prior to runtime. + +### Final Production Dockerfile: +```dockerfile +# --------- STAGE 1: Builder --------- +FROM golang:1.20-alpine AS builder +WORKDIR /app +COPY main.go . +RUN CGO_ENABLED=0 GOOS=linux go build -o myapp main.go + +# --------- STAGE 2: Production --------- +FROM alpine:3.18 + +# Create dedicated non-root user and group +RUN addgroup -S appgroup && adduser -S appuser -G appgroup + +WORKDIR /home/appuser/ +COPY --from=builder /app/myapp . + +# Switch active execution context to non-root user +USER appuser + +EXPOSE 8080 +CMD ["./myapp"] +``` + +### Security Verification: +```bash +docker build -t day35-best-practices . +docker run --rm day35-best-practices whoami +``` +* Output: Successfully returned `appuser` instead of `root`, proving that process isolation is fully active. + +![Task 5 - Security Verification Output](./task-05-best-practices.png) + +--- + +## Key Takeaways +* Heavy base images will quietly choke your cloud servers (like my EC2 disk space crash). Multi-stage builds are non-negotiable for production. +* Always use explicit tags (`alpine:3.18`) instead of `latest` for reproducible builds. +* Running containers as a non-root `appuser` is a mandatory security baseline to prevent container-escape privilege escalations. diff --git a/2026/day-35/main.go b/2026/day-35/main.go new file mode 100644 index 0000000000..9e4051454d --- /dev/null +++ b/2026/day-35/main.go @@ -0,0 +1,14 @@ +package main + +import ( + "fmt" + "net/http" +) + +func main() { + http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, "Hello Day 35! This is a heavy single-stage build.") + }) + fmt.Println("Server starting on port 8080...") + http.ListenAndServe(":8080", nil) +} diff --git a/2026/day-35/task-01-large-image.png b/2026/day-35/task-01-large-image.png new file mode 100644 index 0000000000..a4bffbe091 Binary files /dev/null and b/2026/day-35/task-01-large-image.png differ diff --git a/2026/day-35/task-02-multi-stage.png b/2026/day-35/task-02-multi-stage.png new file mode 100644 index 0000000000..ff141f69d1 Binary files /dev/null and b/2026/day-35/task-02-multi-stage.png differ diff --git a/2026/day-35/task-03-docker-hub-push.png b/2026/day-35/task-03-docker-hub-push.png new file mode 100644 index 0000000000..ede85259de Binary files /dev/null and b/2026/day-35/task-03-docker-hub-push.png differ diff --git a/2026/day-35/task-04-docker-hub-repo.png b/2026/day-35/task-04-docker-hub-repo.png new file mode 100644 index 0000000000..bd5d7cbd7b Binary files /dev/null and b/2026/day-35/task-04-docker-hub-repo.png differ diff --git a/2026/day-35/task-05-best-practices.png b/2026/day-35/task-05-best-practices.png new file mode 100644 index 0000000000..1e7791c762 Binary files /dev/null and b/2026/day-35/task-05-best-practices.png differ diff --git a/2026/day-36/architecture-diagram.png b/2026/day-36/architecture-diagram.png new file mode 100644 index 0000000000..a2e8232cd7 Binary files /dev/null and b/2026/day-36/architecture-diagram.png differ diff --git a/2026/day-36/day-36-docker-project.md b/2026/day-36/day-36-docker-project.md new file mode 100644 index 0000000000..42523b4fcf --- /dev/null +++ b/2026/day-36/day-36-docker-project.md @@ -0,0 +1,91 @@ +# Day 36 – Docker Project: End-to-End Dockerization + +## Task 1: Pick Your App +**Project Repository:** [Nexus Prime Dashboard on GitHub](https://github.com/rashid-khan681/nexus-dashboard) + +**App Chosen:** Nexus Prime Dashboard (Full-Stack 3-Tier Application) + +**Why?** +- I chose this project because it represents a real-world, production-level architecture. It consists of a Next.js frontend, a Node.js/Express backend (with Socket.io for live telemetry), and a PostgreSQL database. It challenges me to handle multi-stage builds, internal networking, persistent volumes, and dynamic WebSocket routing in a containerized AWS EC2 environment. + +![Nexus Prime Architecture](architecture-diagram.png) + +--- + +## Task 2: Write the Dockerfile +I created Dockerfiles for both the frontend and backend. Here is the **Frontend Dockerfile**, which utilizes a multi-stage build and a non-root user (`appuser`) for enhanced security and reduced image size. + +![Dockerfiles Setup](task-02-dockerfiles.png) +![Frontend Multi-stage Build](task-02-frontend-multi.png) + +```dockerfile +# STAGE 1: Dependencies +FROM node:22-alpine AS deps +WORKDIR /app +COPY package*.json ./ +RUN npm install + +# STAGE 2: Builder +FROM node:22-alpine AS builder +WORKDIR /app +COPY --from=deps /app/node_modules ./node_modules +COPY . . +RUN npm run build + +# STAGE 3: Runner +FROM node:22-alpine AS runner +WORKDIR /app + +ENV NODE_ENV production + +# Create a non-root user and group +RUN addgroup -S appgroup && adduser -S appuser -G appgroup + +COPY --from=builder --chown=appuser:appgroup /app/public ./public +COPY --from=builder --chown=appuser:appgroup /app/.next/standalone ./ +COPY --from=builder --chown=appuser:appgroup /app/.next/static ./.next/static + +USER appuser +EXPOSE 3000 +CMD ["node", "server.js"] +``` + +--- + +## Task 3: Add Docker Compose +The `docker-compose.yml` ties everything together. It features a custom bridge network (`nexus-net`), a persistent volume for the database (`pg_data`), and `pg_isready` healthchecks to ensure the backend only starts when the DB is fully ready. + +![Docker Compose Up](task-03-compose-up.png) + +--- + +## Task 4: Ship It +The images were successfully built, tagged, and pushed to my public Docker Hub repository. + +![Docker Hub Repositories](task-04-docker-hub.png) + +**Docker Hub Links:** +* **Backend:** [rashidkhan6685/nexus-backend](https://hub.docker.com/r/rashidkhan6685/nexus-backend) +* **Frontend:** [rashidkhan6685/nexus-frontend](https://hub.docker.com/r/rashidkhan6685/nexus-frontend) + +**Final Image Sizes:** +By using `node:22-alpine` and multi-stage builds, the frontend image was reduced significantly to under 299MB, and the backend to around 391MB. + +--- + +## Task 5: Test the Whole Flow (Remote AWS EC2 Deployment) +Instead of just testing locally, I deployed this entirely on a remote **AWS EC2 Ubuntu Instance**. I pulled the images directly from Docker Hub and ran them using `docker compose up -d`. + +![Remote EC2 Deployment](task-05-remote-deploy.png) + +### Challenges Faced & How I Solved Them + +1. **"No space left on device" (EBS Volume Limit):** + * *Issue:* AWS default EC2 instances provide an 8GB EBS volume. Frequent multi-stage builds caching quickly filled the disk. + * *Solution:* Implemented rigorous Docker cleanup commands (`docker builder prune -a -f` and `docker image prune -a -f`) before builds to clear dangling images and cache. +2. **The Hardcoded IP / WebSocket Trap:** + * *Issue:* Initially, the Next.js frontend was attempting to connect to `http://localhost:5000` for the WebSocket stream. In a remote EC2 setup, this failed because `localhost` referred to the client's browser, not the server. + * *Solution:* Re-engineered the client connection to dynamically resolve the host IP using `window.location.hostname`. This eliminated the need for hardcoded `.env` IPs and allowed the code to automatically adapt to any new AWS Public IP. +3. **Security Group & Port Conflicts:** + * *Issue:* The dashboard wouldn't connect because Node.js was serving both REST APIs and Socket.io on port 5000, but the frontend was trying to ping port 5001. + * *Solution:* Debugged container logs (`docker compose logs backend`), identified the single exposed port, audited AWS Security Groups to allow inbound TCP on port 5000, and corrected the frontend code to route all traffic to the unified port. diff --git a/2026/day-36/task-02-dockerfiles.png b/2026/day-36/task-02-dockerfiles.png new file mode 100644 index 0000000000..b38b9ec435 Binary files /dev/null and b/2026/day-36/task-02-dockerfiles.png differ diff --git a/2026/day-36/task-02-frontend-multi.png b/2026/day-36/task-02-frontend-multi.png new file mode 100644 index 0000000000..82aaacf644 Binary files /dev/null and b/2026/day-36/task-02-frontend-multi.png differ diff --git a/2026/day-36/task-03-compose-up.png b/2026/day-36/task-03-compose-up.png new file mode 100644 index 0000000000..54e556417a Binary files /dev/null and b/2026/day-36/task-03-compose-up.png differ diff --git a/2026/day-36/task-04-docker-hub.png b/2026/day-36/task-04-docker-hub.png new file mode 100644 index 0000000000..f3c477aec3 Binary files /dev/null and b/2026/day-36/task-04-docker-hub.png differ diff --git a/2026/day-36/task-05-remote-deploy.png b/2026/day-36/task-05-remote-deploy.png new file mode 100644 index 0000000000..d5d760ffc6 Binary files /dev/null and b/2026/day-36/task-05-remote-deploy.png differ diff --git a/2026/day-37/day-37-revision.md b/2026/day-37/day-37-revision.md new file mode 100644 index 0000000000..f1d3f2f43d --- /dev/null +++ b/2026/day-37/day-37-revision.md @@ -0,0 +1,67 @@ +# Day 37: Docker Revision & Self-Assessment + +## 1. Self-Assessment Checklist +I have marked this honestly based on my actual hands-on execution from Day 29 to Day 36 on my AWS EC2 instance: + +- [x] Run a container from Docker Hub (interactive + detached) +- [x] List, stop, remove containers and images +- [x] Explain image layers and how caching works +- [x] Write a Dockerfile from scratch with FROM, RUN, COPY, WORKDIR, CMD +- [x] Explain CMD vs ENTRYPOINT +- [x] Build and tag a custom image +- [x] Create and use named volumes +- [x] Use bind mounts +- [x] Create custom networks and connect containers +- [x] Write a docker-compose.yml for a multi-container app +- [x] Use environment variables and .env files in Compose +- [x] Write a multi-stage Dockerfile +- [x] Push an image to Docker Hub +- [x] Use healthchecks and depends_on + +*Self-Check Verdict:* **100% Confident**. (I executed all of these end-to-end while deploying the Nexus Prime dashboard). + +## 2. Quick-Fire Questions (From Real Experience) + +**1. What is the difference between an image and a container?** +- An image is an immutable blueprint built from read-only filesystem layers (like the heavy 1.8GB Node image I built). A container is the actual live, isolated Linux process running that blueprint with a thin read/write layer on top. + +**2. What happens to data inside a container when you remove it?** +- It is permanently destroyed. I experienced this firsthand on Day 32: I created a `demo` table in a Postgres container, ran `docker rm -f`, and spun up a new one. My query through `ERROR: relation "demo" does not exist`. Data must be backed by Volumes to survive! + +**3. How do two containers on the same custom network communicate?** +- They talk seamlessly using Docker's embedded DNS by referencing their container/service names. Docker features an embedded DNS server that automatically translates container names into their private IP addresses. I proved this when the default bridge threw a `ping: bad address` error, but pinging `my-container2` worked instantly with 0% packet loss on my custom `my-app-net`. + +**4. What does `docker compose down -v` do differently from `docker compose down`?** +- A standard `down` only removes the containers and networks, keeping your data safe. Adding the `-v` flag violently destroys your persistent named volumes too. The critical difference is that docker compose down -v deletes your persistent data volumes, whereas docker compose down leaves your data completely untouched. I used `-v` specifically when my EC2 EBS volume was choked and I needed to deep clean my server. + +**5. Why are multi-stage builds useful?** +- Multi-stage builds allow you to create drastically **smaller, more secure, and production-ready Docker images** using a single Dockerfile (via multiple `FROM` statements). + +Here is exactly why they are so valuable: +* **Drastically Smaller Image Sizes:** Instead of baking heavy build tools and compilers into the final image, you compile the app in Stage 1 and copy *only the final executable* into a completely clean, lightweight Stage 2 (like `alpine`). (e.g., A Go app drops from 800MB to just 15MB). +* **Enhanced Security:** Every build tool left in an image increases its attack surface. By leaving those tools behind in the build stage, your final runtime image contains only what is absolutely necessary, significantly lowering vulnerability risks. +* **Separation of Concerns:** You can logically organize tasks into distinct stages (e.g., Stage 1: Build -> Stage 2: Test -> Stage 3: Production). + +**Quick Comparison: Before vs. After** +| Feature | Traditional Build | Multi-Stage Build | +| :--- | :--- | :--- | +| **Dockerfile Count** | Often required multiple files (`Dockerfile.dev`, `Dockerfile.prod`) | **One** single, easy-to-read Dockerfile | +| **Final Image Size** | Large (includes build tools, caches, and source code) | **Minimal** (includes only compiled binaries/assets) | +| **Security Risk** | Higher (larger attack surface, more packages) | **Lower** (only runtime dependencies are present) | + +**6. What is the difference between COPY and ADD?** +- `COPY` strictly copies files/directories from the host into the container. `ADD` does the same but can also extract `.tar` archives and download from URLs. Best practice is to stick to `COPY` for predictable builds, which I used consistently in my Dockerfiles. + +**7. What does `-p 8080:80` mean?** +- The flag `-p 8080:80` tells Docker to forward traffic from a specific port on your host machine to a port inside the container. This process is called **port mapping** or **port publishing**. + +Here is the exact breakdown of how it works: +* **`-p` (or `--publish`)**: This tells Docker that you want to open a network port so outside traffic can reach the container. +* **`8080` (Host Port)**: The port on your *actual computer* (host machine) that you will connect to in your browser or app (e.g., `http://localhost:8080`). +* **`80` (Container Port)**: The port that the application inside the Docker container is actively *listening on* (usually the default port for web servers like Nginx or Apache). + +When a request is made, it follows this exact path: +`Your Browser ──> Host Machine (Port 8080) ──> Docker Bridge ──> Container (Port 80)` + +**8. How do you check how much disk space Docker is using?** +- By running `docker system df` to see a high-level summary of the space used by images, containers, local volumes, and the build cache. For a deeper, itemized breakdown showing exactly which specific container or image is consuming space, we append the verbose flag: `docker system df -v`. diff --git a/2026/day-37/docker-cheatsheet.md b/2026/day-37/docker-cheatsheet.md new file mode 100644 index 0000000000..09ead762e3 --- /dev/null +++ b/2026/day-37/docker-cheatsheet.md @@ -0,0 +1,51 @@ +# Ultimate Docker & Compose Cheat Sheet +Compiled from real-world EC2 deployments, multi-stage builds, and troubleshooting. + +## Container Commands +* `docker run -d -p 8080:80 --name web nginx` - Starts a container in detached mode with port mapping. +* `docker ps -a` - Lists all containers (both running and Exited/137). +* `docker stop ` - Gracefully shuts down a container (SIGTERM). +* `docker rm -f ` - Force deletes a running or stopped container. +* `docker exec -it bash` - Jumps into a running container's shell for live triage. +* `docker logs --tail 20 -f ` - Streams the last 20 lines of logs in real-time. + +## Image Commands +* `docker build -t my-app:v1 .` - Builds an image from the Dockerfile in the current directory. +* `docker pull ` - Fetches an image from Docker Hub to local cache. +* `docker push ` - Uploads your tagged local image to a Docker Hub registry. +* `docker tag local-image:latest user/repo:v1` - Tags an image properly for pushing to a remote registry. +* `docker images` - Lists all local images and their physical disk sizes. +* `docker rmi -f ` - Forcefully removes an image from local storage. + +## Volume Commands (Data Persistence) +* `docker volume create ` - Creates a persistent named volume. +* `docker volume ls` - Lists all Docker-managed volumes. +* `docker volume inspect ` - Shows volume details including the physical mount path on the host. +* `docker volume rm ` - Deletes a specific volume. + +## Network Commands (Container Communication) +* `docker network create ` - Creates a custom bridge network for automatic DNS resolution. +* `docker network ls` - Lists all networks (bridge, host, none, and custom). +* `docker network inspect ` - Displays network details including attached containers and internal IPs. +* `docker network connect ` - Attaches a running container to a network. + +## Docker Compose Commands +* `docker compose up -d --build` - Builds images and spins up the entire stack in the background. +* `docker compose down -v` - Destroys the stack, including networks and named volumes. +* `docker compose ps` - Shows the health and status of all services in the current stack. +* `docker compose logs -f db` - Streams real-time logs for a specific service (like 'db'). +* `docker compose config` - Validates the YAML file and verifies .env variable injection. + +## Cleanup Commands (Disk Space Savers) +* `docker system df` - Shows exactly how much disk space Docker images, containers, and volumes are eating. +* `docker system prune -a --volumes -f` - The ultimate nuke command: clears all unused containers, images, networks, and volumes. +* `docker builder prune -a -f` - Clears hidden BuildKit cache to prevent "no space left on device" errors. + +## Core Dockerfile Instructions +* `FROM` - Defines the base OS image (e.g., node:22-alpine). +* `RUN` - Executes shell commands during the build phase (e.g., npm install). +* `COPY` - Transfers files from the host into the container filesystem. +* `WORKDIR` - Sets the active directory inside the container for subsequent commands. +* `EXPOSE` - Documents the port the container listens on (e.g., 5000). +* `CMD` - The default startup command that can be overridden at runtime. +* `ENTRYPOINT` - The strict startup executable that appends extra runtime arguments instead of being overridden. diff --git a/2026/day-38/day-38-yaml.md b/2026/day-38/day-38-yaml.md new file mode 100644 index 0000000000..d63e8db6ed --- /dev/null +++ b/2026/day-38/day-38-yaml.md @@ -0,0 +1,112 @@ +# Day 38: YAML Basics + +## Task 1 & 2: Key-Value Pairs and Lists +In this task, I created `person.yml` to understand basic key-value pairs and the two ways to write lists in YAML (Block style and Flow style). + +### `person.yml` +```yaml +--- +name: Rashid Khan +role: DevOps Engineer +experience_year: 1 +learning: true + +tools: + - Docker + - Linux + - Git + - Kubernetes + - Terraform + +hobbies: [Coding, Cloud Computing, Automation] +``` + +**Notes:** +* **Block style list:** Items are written on new lines starting with a dash and a space (`- item`). Best for readability and long arrays. +* **Flow style (inline) list:** Items are written in a single line inside square brackets (`[item1, item2]`). Best for compact, short lists. + +--- + +## Task 3 & 4: Nested Objects and Multi-line Strings +Here, I created `server.yml` to understand hierarchical nesting (using standard 2-space indentation) and handling long strings or bash scripts. + +### `server.yml` +```yaml +--- +server: + name: web-server-01 + ip: 192.168.1.10 + port: 80 + +database: + host: db.local + name: user_db + credentials: + user: admin + password: supersecretpassword + +# The '|' (pipe) character preserves line breaks exactly as they are written. +# Goal: Use this when exact formatting matters, like in a multi-line bash script. +startup_script: | + #!/bin/bash + echo "Starting the web server..." + systemctl start nginx + echo "Server is up and running!" + +# The '>' (Greater-than) character folds multiple lines into a single long line. +# Goal: Use this for long text paragraphs where new lines don't matter, to keep the file readable. +server_description: > + This is the primary web server for the application. + It handles all incoming HTTP traffic and routes it to the appropriate backend services. +``` + +**Notes on Multi-line strings:** +* Use `|` (Pipe) when you want to **preserve newlines** (e.g., Bash scripts in CI/CD pipelines). +* Use `>` (Fold) when you want to **ignore newlines** and fold everything into one long string (e.g., text descriptions). + +--- + +## Task 5: Validate Your YAML +YAML is extremely strict about spacing. Tabs are completely forbidden. To prove this, I intentionally injected a Tab character into `server.yml` and validated it using `yamllint` on my AWS EC2 instance. + +### Terminal Output (The Sabotage & Fix) +```bash +ubuntu@ip-172-31-29-81:~/90DaysOfDevOpsChallengeRashidKhan/2026/day-38$ yamllint server.yml +server.yml + 3:81 error line too long (81 > 80 characters) (line-length) + 12:80 error trailing spaces (trailing-spaces) + 13:81 error line too long (99 > 80 characters) (line-length) + 18:81 error line too long (89 > 80 characters) (line-length) + 21:1 error syntax error: could not find expected ':' (syntax) +``` +**Conclusion:** The fatal error at line 21 (`syntax error: could not find expected ':'`) occurred because of the hidden Tab character (`\t`). The parser immediately rejected the file. Removing the tab fixed the syntax error. + +--- + +## Task 6: Spot the Difference + +**Block 1 - Correct** +```yaml +name: devops +tools: + - docker + - kubernetes +``` + +**Block 2 - Broken** +```yaml +name: devops +tools: +- docker + - kubernetes +``` + +**What is wrong with Block 2?** +The indentation is completely inconsistent. In YAML, list items belonging to the same array must be vertically aligned perfectly. In Block 2, `- docker` has 0 spaces of indentation, while `- kubernetes` has 2 spaces. This breaks the hierarchy, and the YAML parser will throw an indentation/syntax error. + +--- + +## 3 Key Learnings Today +1. **Spaces Only, Never Tabs:** YAML parsers will violently reject any file containing a tab character. The 2-space indentation rule is the golden standard. +2. **Handling Scripts in Configs:** The pipe (`|`) block style is an absolute necessity for writing multi-line CI/CD shell scripts directly inside YAML pipelines without losing line breaks. +3. **Pre-commit Linting is Mandatory:** Running `yamllint` locally before pushing configuration code saves hours of troubleshooting confusing pipeline failures in production. diff --git a/2026/day-39/day-39-cicd-concepts.md b/2026/day-39/day-39-cicd-concepts.md new file mode 100644 index 0000000000..8aad0a5b8b --- /dev/null +++ b/2026/day-39/day-39-cicd-concepts.md @@ -0,0 +1,81 @@ +# Day 39: Understanding CI/CD Concepts + +Before writing any pipeline code, it is critical to understand the architecture, problems solved, and exact terminology behind Continuous Integration and Continuous Delivery/Deployment. + +## Task 1: The Problem (Manual Deployments & Integration) +* **What can go wrong with 5 developers pushing manually?** + In the industry, this is known as **"Integration Hell."** Manual deployments lack automated quality gates. Developers overwrite configurations, miss database migrations, or deploy untested code. The result is silent production breakages and extended downtime. +* **What does "it works on my machine" mean?** + This is officially called **Configuration Drift**. A developer's local environment (OS versions, hidden `.env` files, global dependencies) slowly drifts away from the actual production server state. It works locally due to unrecorded dependencies but crashes in production, making triage a guessing game between Dev and Ops teams. +* **How many times a day can a team safely deploy manually?** + According to **DORA Metrics** (the standard for DevOps performance), teams relying on manual deployments are categorized as "Low Performers." They can safely deploy perhaps once a month or once a week. Manual deployments inherently carry a high Change Failure Rate (46%–60%). + +--- + +## Task 2: CI vs CD Definitions +* **Continuous Integration (CI):** + The automated process where developers frequently merge code into a central repository. On every push, automated build scripts and unit tests execute immediately. It acts as a quality gate to catch syntax errors and broken dependencies early. + * *Example:* A developer pushes code; GitHub Actions instantly runs `pytest` and blocks the Pull Request if existing functions break. +* **Continuous Delivery (CD):** + An extension of CI where the codebase is always maintained in a deployable state. Artifacts are automatically built and deployed to a Staging environment. However, promoting the release to the live Production environment requires a **manual human approval**. + * *Example:* QA verifies the code on `staging.myapp.com`. A Release Manager then manually clicks "Deploy to Prod" in Jenkins. +* **Continuous Deployment (CD):** + The ultimate level of automation. Every code change that successfully passes all automated test gates is deployed directly to the live Production servers with **zero human intervention**. + * *Example:* A Netflix developer pushes a UI patch. It passes automated tests and goes live to millions of users 15 minutes later without management approval. + +--- + +## Task 3: Pipeline Anatomy +Real-world pipelines execute using these hierarchical components: +* **Trigger:** An event listener that initiates the pipeline (e.g., a webhook for `git push`, or a time-based CRON schedule). +* **Stage:** A macro logical phase (Build, Test, Deploy). Stages run sequentially; a failure in 'Test' halts execution, preventing 'Deploy'. +* **Job:** A defined unit of work executing on a single runner. Jobs within the same stage can run concurrently in parallel, but a single job cannot be split across multiple servers. +* **Step:** The smallest executable unit within a job (e.g., running `npm install` or checking out code). Steps run sequentially and share the same filesystem. +* **Runner:** The actual compute infrastructure (VM, Docker container, or EC2 instance) executing the job. +* **Artifact:** A persistent output file (compiled binary, `.zip` archive, or Docker image) generated by a successful job, passed to subsequent stages. + +--- + +## Task 4: CI/CD Pipeline Diagram +*Scenario: Push -> Test -> Docker Build -> Staging Deploy* + + [ Developer ] + β”‚ (git push origin main) + β–Ό + =================================================== + CI/CD PIPELINE (GitHub Actions) + =================================================== + β”‚ + β”Œβ”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β” + β”‚ STAGE 1 β”‚ + β”‚ Unit Test β”‚ + β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ + β”‚ (If tests pass) + β”Œβ”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β” + β”‚ STAGE 2 β”‚ ────(Push)────► [ Docker Registry ] + β”‚ Build App β”‚ + β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ + β”‚ (Pull image) + β”Œβ”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β” + β”‚ STAGE 3 β”‚ + β”‚ Deploy β”‚ + β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ + β”‚ (SSH / Update) + ======β–Ό============================================ + [ Staging Server (AWS EC2) ] + +--- + +## Task 5: Explore in the Wild (Open-Source Analysis) +Using the terminal, I fetched the actual production CI/CD workflow of the popular Python framework **FastAPI**. + +* **Repository Analyzed:** `tiangolo/fastapi` +* **Workflow File:** `.github/workflows/test.yml` +* **What triggers it?** + It utilizes the `on:` keyword to define webhook triggers. Specifically, it executes on `push` and `pull_request` events targeting the `master` branch. It also has a cron schedule `0 0 * * 1` to run weekly on Mondays. +* **How many jobs does it have?** + The top of the file defines initial jobs like `changes` (using a filter to check modified files) before dynamically spinning up test jobs. +* **What does it do?** + It executes the core CI process. It checks out the code, evaluates permissions, and sets up dependencies to rigorously test FastAPI against multiple Python versions and operating systems via matrix strategies. + +![Task 5 Output](./task-05-explore.png) diff --git a/2026/day-39/fastapi-test.yml b/2026/day-39/fastapi-test.yml new file mode 100644 index 0000000000..c4fdadc901 --- /dev/null +++ b/2026/day-39/fastapi-test.yml @@ -0,0 +1,332 @@ +name: Test + +on: + push: + branches: + - master + pull_request: + schedule: + # cron every week on monday + - cron: "0 0 * * 1" + +permissions: {} + +env: + UV_NO_SYNC: true + INLINE_SNAPSHOT_DEFAULT_FLAGS: review + +jobs: + changes: + runs-on: ubuntu-latest + # Required permissions + permissions: + pull-requests: read + # Set job outputs to values from filter step + timeout-minutes: 5 + outputs: + src: ${{ steps.filter.outputs.src }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + # For pull requests it's not necessary to checkout the code but for the main branch it is + - uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3 + id: filter + with: + filters: | + src: + - .github/workflows/test.yml + - docs_src/** + - fastapi/** + - scripts/** + - tests/** + - .python-version + - pyproject.toml + - uv.lock + + test: + needs: + - changes + if: needs.changes.outputs.src == 'true' || github.ref == 'refs/heads/master' + timeout-minutes: 10 + strategy: + matrix: + os: [ windows-latest, macos-latest ] + python-version: [ "3.14", "3.14t" ] + deprecated-tests: [ "no-deprecation" ] + uv-resolution: + - highest + starlette-src: + - starlette-pypi + - starlette-git + include: + - os: macos-latest + python-version: "3.10" + coverage: coverage + uv-resolution: lowest-direct + deprecated-tests: "no-deprecation" + - os: windows-latest + python-version: "3.12" + coverage: coverage + uv-resolution: lowest-direct + deprecated-tests: "no-deprecation" + - os: ubuntu-latest + python-version: "3.13" + coverage: coverage + uv-resolution: highest + deprecated-tests: "no-deprecation" + - os: ubuntu-latest + python-version: "3.13" + uv-resolution: highest + codspeed: codspeed + deprecated-tests: "no-deprecation" + - os: ubuntu-latest + python-version: "3.13" + uv-resolution: highest + deprecated-tests: "no-deprecation" + without-httpx2: true + - os: ubuntu-latest + python-version: "3.14" + coverage: coverage + uv-resolution: highest + starlette-src: starlette-git + deprecated-tests: "test-deprecation" + - os: ubuntu-latest + python-version: "3.14t" + coverage: coverage + uv-resolution: highest + deprecated-tests: "no-deprecation" + fail-fast: false + runs-on: ${{ matrix.os }} + env: + UV_PYTHON: ${{ matrix.python-version }} + UV_RESOLUTION: ${{ matrix.uv-resolution }} + STARLETTE_SRC: ${{ matrix.starlette-src }} + steps: + - name: Dump GitHub context + env: + GITHUB_CONTEXT: ${{ toJson(github) }} + run: echo "$GITHUB_CONTEXT" + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Set up Python + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: ${{ matrix.python-version }} + - name: Setup uv + uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1 + with: + version: "latest-known" + enable-cache: true + cache-dependency-glob: | + pyproject.toml + uv.lock + - name: Install Dependencies + run: uv sync --no-dev --group tests --extra all + - name: Ensure that we have the lowest supported Pydantic version + if: matrix.uv-resolution == 'lowest-direct' + run: uv pip install "pydantic==2.9.0" + - name: Install Starlette from source + if: matrix.starlette-src == 'starlette-git' + run: uv pip install "git+https://github.com/Kludex/starlette@main" + - name: Install deprecated libraries just for testing + if: matrix.deprecated-tests == 'test-deprecation' + run: uv pip install orjson ujson + - name: Uninstall httpx2 to run tests with httpx + if: matrix.without-httpx2 == 'true' + run: uv pip uninstall httpx2 + - name: Reinstall SQLAlchemy without Cython extensions + if: matrix.python-version == '3.14t' && matrix.os == 'ubuntu-latest' + run: "DISABLE_SQLALCHEMY_CEXT=1 uv pip install --force-reinstall --no-binary :all: sqlalchemy" + - run: mkdir coverage + - name: Test + run: uv run --no-sync bash scripts/test-cov.sh $PYTEST_OPTIONS + env: + COVERAGE_FILE: coverage/.coverage.${{ runner.os }}-py${{ matrix.python-version }}-${{ matrix.deprecated-tests}} + CONTEXT: ${{ runner.os }}-py${{ matrix.python-version }}-${{ matrix.deprecated-tests}} + PYTEST_OPTIONS: ${{ (matrix.without-httpx2 == 'true') && '-W ignore::UserWarning' || '' }} + # Do not store coverage for all possible combinations to avoid file size max errors in Smokeshow + - name: Store coverage files + if: matrix.coverage == 'coverage' + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: coverage-${{ runner.os }}-${{ matrix.python-version }}-${{ matrix.deprecated-tests}}-${{ hashFiles('**/coverage/.coverage.*') }} + path: coverage + include-hidden-files: true + + benchmark: + needs: + - changes + if: needs.changes.outputs.src == 'true' || github.ref == 'refs/heads/master' + runs-on: ubuntu-latest + timeout-minutes: 5 + env: + UV_PYTHON: "3.13" + UV_RESOLUTION: highest + steps: + - name: Dump GitHub context + env: + GITHUB_CONTEXT: ${{ toJson(github) }} + run: echo "$GITHUB_CONTEXT" + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Set up Python + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.13" + - name: Setup uv + uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1 + with: + version: "latest-known" + enable-cache: true + cache-dependency-glob: | + pyproject.toml + uv.lock + - name: Install Dependencies + run: uv sync --no-dev --group tests --extra all + - name: CodSpeed benchmarks + uses: CodSpeedHQ/action@373d6868929f444bc08d901fd0eb0ad52a8875ea # v5.2.1 + with: + mode: simulation + run: uv run --no-sync pytest tests/benchmarks --codspeed + - name: CodSpeed memory benchmark + uses: CodSpeedHQ/action@373d6868929f444bc08d901fd0eb0ad52a8875ea # v5.2.1 + with: + mode: memory + run: uv run --no-sync pytest tests/memory_benchmarks --codspeed + + regression-test: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - name: Check out the pull request + if: github.event_name == 'pull_request' + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.sha }} + path: pr + persist-credentials: false + fetch-depth: 0 + - name: Find changed tests + if: github.event_name == 'pull_request' + id: changed-tests + working-directory: pr + env: + BASE_SHA: ${{ github.event.pull_request.base.sha }} + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + run: | + git diff --name-only --diff-filter=AM -z "$BASE_SHA" "$HEAD_SHA" -- tests \ + | while IFS= read -r -d '' file; do + case "$(basename "$file")" in + test_*.py) printf '%s\0' "$file" ;; + esac + done > "$RUNNER_TEMP/changed-tests" + if [ -s "$RUNNER_TEMP/changed-tests" ]; then + echo "found=true" >> "$GITHUB_OUTPUT" + git diff --binary "$BASE_SHA" "$HEAD_SHA" -- tests \ + > "$RUNNER_TEMP/tests.patch" + else + echo "found=false" >> "$GITHUB_OUTPUT" + echo "No added or modified test files; regression proof is not applicable." + fi + - name: Check out the base revision + if: steps.changed-tests.outputs.found == 'true' + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.event.pull_request.base.sha }} + path: base + persist-credentials: false + - name: Set up Python + if: steps.changed-tests.outputs.found == 'true' + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version-file: "base/.python-version" + - name: Setup uv + if: steps.changed-tests.outputs.found == 'true' + uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1 + with: + version: "latest-known" + enable-cache: true + - name: Run the changed tests against the base code + if: steps.changed-tests.outputs.found == 'true' + working-directory: base + run: | + git apply "$RUNNER_TEMP/tests.patch" + uv sync --locked --no-dev --group tests --extra all + set +e + xargs -0 uv run --no-sync pytest -- < "$RUNNER_TEMP/changed-tests" + status=$? + set -e + if [ "$status" -eq 0 ]; then + echo "::warning::The changed tests already pass on the base revision. Check whether the fix is still needed." + echo "### Regression proof: base already passes :warning:" >> "$GITHUB_STEP_SUMMARY" + echo "The changed tests pass without the pull request's code changes." >> "$GITHUB_STEP_SUMMARY" + else + echo "The changed tests fail on the base revision as expected (pytest exit code $status)." + echo "### Regression proof: base fails as expected :white_check_mark:" >> "$GITHUB_STEP_SUMMARY" + fi + coverage-combine: + needs: + - test + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - name: Dump GitHub context + env: + GITHUB_CONTEXT: ${{ toJson(github) }} + run: echo "$GITHUB_CONTEXT" + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version-file: ".python-version" + - name: Setup uv + uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1 + with: + version: "latest-known" + enable-cache: true + cache-dependency-glob: | + pyproject.toml + uv.lock + - name: Install Dependencies + run: uv sync --locked --no-dev --group tests --extra all + - name: Get coverage files + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + pattern: coverage-* + path: coverage + merge-multiple: true + - run: ls -la coverage + - run: uv run coverage combine coverage + - run: uv run coverage html --title "Coverage for ${{ github.sha }}" + - name: Store coverage HTML + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: coverage-html + path: htmlcov + include-hidden-files: true + - run: uv run coverage report --fail-under=100 + + # https://github.com/marketplace/actions/alls-green#why + test-alls-green: # This job does nothing and is only used for the branch protection + if: always() + needs: + - test + - coverage-combine + - benchmark + - regression-test + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - name: Dump GitHub context + env: + GITHUB_CONTEXT: ${{ toJson(github) }} + run: echo "$GITHUB_CONTEXT" + - name: Decide whether the needed jobs succeeded or failed + uses: re-actors/alls-green@b5b5b37504aa4183270bd3d855c52a67f212be35 # v1.3.0 + with: + jobs: ${{ toJSON(needs) }} + allowed-skips: coverage-combine,test,benchmark diff --git a/2026/day-39/task-05-explore.png b/2026/day-39/task-05-explore.png new file mode 100644 index 0000000000..4f5b64640f Binary files /dev/null and b/2026/day-39/task-05-explore.png differ diff --git a/2026/day-40/day-40-first-workflow.md b/2026/day-40/day-40-first-workflow.md new file mode 100644 index 0000000000..4596d815b8 --- /dev/null +++ b/2026/day-40/day-40-first-workflow.md @@ -0,0 +1,72 @@ +# Day 40: Your First GitHub Actions Workflow + +This document records the execution of my first GitHub Actions Continuous Integration (CI) pipeline, moving from theoretical architecture (Day 39) to practical implementation on an active repository. + +## Task 1 & 2: Set Up and Hello Workflow +Instead of an empty practice repository, I utilized my existing forked repository (`e-commerce-github-actions`) to simulate a real-world CI/CD environment. + +Below is the initial configuration designed to verify runner allocation and basic execution: + + # ===================================================================== + # GOAL: Execute the first basic GitHub Actions pipeline to verify CI setup + # TRIGGER: Executes automatically on every push to any branch + # ===================================================================== + name: Hello Workflow + + on: + push: + + jobs: + # ------------------------------------------------------------------- + # JOB 1: A simple greeting job to test runner execution and checkout + # ------------------------------------------------------------------- + greet: + runs-on: ubuntu-latest + steps: + - name: Checkout Code + uses: actions/checkout@v4 + + - name: Print Greeting + run: echo "Hello from GitHub Actions!" + +**Verification:** +The pipeline was triggered automatically on push. GitHub provisioned an `ubuntu-latest` runner, checked out the code, and successfully executed the bash command. + +![Green Pipeline Run](./task-02-green-run.png) + +--- + +## Task 3: Understand the Anatomy +Based on official GitHub Actions architecture documentation, here is the technical breakdown of the workflow configuration keys: + +* **`on:`** Defines the trigger mechanism. It listens for specific webhook events (like `push` or `pull_request`) or time-based CRON schedules to initiate execution. +* **`jobs:`** A macro-level container for a set of steps. By default, multiple jobs within a workflow execute concurrently in parallel to optimize overall execution time. +* **`runs-on:`** Specifies the exact infrastructure environment. Declaring `ubuntu-latest` provisions a fresh, ephemeral Ubuntu Linux virtual machine hosted by GitHub for the duration of the job. +* **`steps:`** The granular, executable units of work within a job. Steps strictly execute sequentially one after the other. They run on the same runner and share the exact same local filesystem, memory, and environment variables. +* **`uses:`** Invokes a predefined, reusable Action. Instead of writing custom shell scripts for common tasks, this calls packaged code (e.g., `actions/checkout@v4` pulls the repository source code into the runner's workspace). +* **`run:`** Executes raw shell commands directly on the runner's operating system (e.g., `npm install` or `ls`). +* **`name:`** A human-readable identifier attached to a job or step. It does not affect execution logic but is critical for observability and debugging in the UI logs. + +--- + +## Task 4: Add More Steps (Dynamic Context Injection) +Pipelines must react to the context of the code being pushed. By injecting additional shell commands and GitHub Context variables (`${{ ... }}`), the pipeline becomes dynamic. + +* Executing `date` confirmed the internal system time of the ephemeral runner. +* Using `${{ github.ref_name }}` dynamically extracted the exact branch name (`main`) that triggered the webhook, which is critical for branch-specific deployment routing. +* Running `ls -la` proved that the `actions/checkout` step successfully pulled the remote code into the runner's isolated workspace. +* Using `${{ runner.os }}` verified the underlying operating system. + +![Expanded Logs](./task-04-expanded-logs.png) + +--- + +## Task 5: Break It On Purpose (Error Handling) +In real-world CI/CD, pipelines act as quality gates. To understand failure states, I intentionally injected a syntax error into the bash script (`echho` instead of `echo`). + +**Observations of a Failed Pipeline:** +1. **Immediate Halt:** The runner hit the unrecognized command, logged an `exit 127` (Command not found) non-zero exit code, and immediately halted execution. +2. **UI Feedback:** The GitHub Actions tab marked the run with a Red Cross (❌). If this were a Pull Request, branch protection rules would automatically block the code from being merged into `main`. +3. **Debugging:** By clicking into the failed step, the logs explicitly highlighted the exact line where the shell failed, making remediation straightforward. + +![Pipeline Failure Analysis](./task-05-failure.png) diff --git a/2026/day-40/task-02-green-run.png b/2026/day-40/task-02-green-run.png new file mode 100644 index 0000000000..a6e6c9164e Binary files /dev/null and b/2026/day-40/task-02-green-run.png differ diff --git a/2026/day-40/task-04-expanded-logs.png b/2026/day-40/task-04-expanded-logs.png new file mode 100644 index 0000000000..84882f2b57 Binary files /dev/null and b/2026/day-40/task-04-expanded-logs.png differ diff --git a/2026/day-40/task-05-failure.png b/2026/day-40/task-05-failure.png new file mode 100644 index 0000000000..70b5b875da Binary files /dev/null and b/2026/day-40/task-05-failure.png differ diff --git a/2026/day-41/day-41-triggers.md b/2026/day-41/day-41-triggers.md new file mode 100644 index 0000000000..42e6d2fd19 --- /dev/null +++ b/2026/day-41/day-41-triggers.md @@ -0,0 +1,153 @@ +# Day 41: GitHub Actions Triggers & Matrix Builds + +This document outlines the implementation of various GitHub Actions triggers, parallel execution using matrix strategies, and workflow control mechanisms. + +## Task 1: Trigger on Pull Request +- **Objective:** Execute pipeline strictly on Pull Requests targeting the `main` branch. +- **Implementation Notes:** Initially encountered cross-repo merge permission issues by targeting the upstream repository (`LondheShubham153:main`). Resolved by correcting the base repository to the local fork (`rashid-khan681/e-commerce-github-actions`). Noted interference from legacy DevSecOps `on: push` workflows which were safely ignored. + +```yaml +name: PR Check Workflow +on: + pull_request: + branches: + - main + types: [opened, synchronize] +jobs: + pr-verification: + runs-on: ubuntu-latest + steps: + - name: Checkout Code + uses: actions/checkout@v4 + - name: Print Branch Name + run: echo "PR check running for branch: ${{ github.head_ref }}" +``` +**Proof of Execution:** +![PR Trigger](./task-01-pr-trigger.png) + +--- + +## Task 2: Scheduled Trigger +- **Objective:** Run a job every day at midnight UTC using standard cron syntax. +- **Question:** What is the cron expression for every Monday at 9 AM? +- **Answer:** `0 9 * * 1` (0th minute, 9th hour, every day of the month, every month, 1 = Monday). + +```yaml +name: Midnight Cron Job +on: + schedule: + - cron: '0 0 * * *' +jobs: + nightly-backup-mock: + runs-on: ubuntu-latest + steps: + - name: Execute Nightly Task + run: echo "Starting nightly database backup at 00:00 UTC..." +``` +**Proof of Registration:** +![Cron Job](./task-02-cron.png) + +--- + +## Task 3: Manual Trigger (`workflow_dispatch`) +- **Objective:** Trigger workflows manually with custom environment inputs. +- **Implementation Notes:** Encountered a context parsing bug where the variable rendered as an empty string. Diagnosed that `environment` is a reserved keyword in GitHub Actions environments. Resolved by refactoring the input key to `target_env`. + +```yaml +name: Manual Deploy Workflow +on: + workflow_dispatch: + inputs: + target_env: + description: 'Target Environment (staging/production)' + required: true + default: 'staging' + type: choice + options: + - staging + - production +jobs: + manual-deploy: + runs-on: ubuntu-latest + steps: + - name: Checkout Code + uses: actions/checkout@v4 + - name: Print Target Environment + run: echo "Deploying application to the ${{ inputs.target_env }} environment!" +``` +**Proof of Execution:** +![Manual Trigger](./task-03-manual.png) + +--- + +## Task 4: Matrix Builds +- **Objective:** Run the exact same pipeline parallelly across multiple Python versions and Operating Systems. +- **Question:** How many total jobs run when matrix includes 3 Python versions and 2 OS? +- **Answer:** 6 jobs run in parallel (3 versions Γ— 2 OS). +- **Implementation Notes:** Logged a Node.js 20 deprecation warning linked to `actions/setup-python@v5`. This is a library-side dependency issue and does not affect the matrix execution integrity. + +```yaml +name: Matrix Build Pipeline +on: + push: + branches: [ main ] +jobs: + multi-env-test: + runs-on: ${{ matrix.os }} + strategy: + matrix: + os: [ubuntu-latest, windows-latest] + python-version: ["3.10", "3.11", "3.12"] + steps: + - name: Checkout Code + uses: actions/checkout@v4 + - name: Setup Python + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + - name: Print Environment Info + run: python -c "import sys; print(f'Running Python {sys.version.split()[0]} on ${{ matrix.os }}')" +``` +**Proof of Execution:** +![Matrix Build](./task-04-matrix.png) + +--- + +## Task 5: Exclude & Fail-Fast +- **Objective:** Remove specific combinations from the matrix and control workflow failure behavior. +- **Question:** What does `fail-fast: true` (the default) do vs `false`? +- **Answer:** +* `fail-fast: true`: Immediately cancels all other running jobs in the matrix if a single job fails to save compute resources. +* `fail-fast: false`: Allows all running jobs in the matrix to finish their execution independent of individual job failures, useful for comprehensive matrix testing. + +```yaml +name: Matrix Build Pipeline +on: + push: + branches: [ main ] +jobs: + multi-env-test: + runs-on: ${{ matrix.os }} + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, windows-latest] + python-version: ["3.10", "3.11", "3.12"] + exclude: + - os: windows-latest + python-version: "3.10" + steps: + - name: Checkout Code + uses: actions/checkout@v4 + - name: Setup Python + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + - name: Intentional Error (To test fail-fast) + if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11' + run: | + echo "Simulating a crash..." + exit 1 +``` +**Proof of Execution:** +![Fail Fast](./task-05-fail-fast.png) diff --git a/2026/day-41/task-01-pr-trigger.png b/2026/day-41/task-01-pr-trigger.png new file mode 100644 index 0000000000..28f8d69a17 Binary files /dev/null and b/2026/day-41/task-01-pr-trigger.png differ diff --git a/2026/day-41/task-02-cron.png b/2026/day-41/task-02-cron.png new file mode 100644 index 0000000000..f2af60eafc Binary files /dev/null and b/2026/day-41/task-02-cron.png differ diff --git a/2026/day-41/task-03-manual.png b/2026/day-41/task-03-manual.png new file mode 100644 index 0000000000..81e9d934d4 Binary files /dev/null and b/2026/day-41/task-03-manual.png differ diff --git a/2026/day-41/task-04-matrix.png b/2026/day-41/task-04-matrix.png new file mode 100644 index 0000000000..b3cc37909e Binary files /dev/null and b/2026/day-41/task-04-matrix.png differ diff --git a/2026/day-41/task-05-fail-fast.png b/2026/day-41/task-05-fail-fast.png new file mode 100644 index 0000000000..2e0f97c25e Binary files /dev/null and b/2026/day-41/task-05-fail-fast.png differ diff --git a/2026/day-42/day-42-runners.md b/2026/day-42/day-42-runners.md new file mode 100644 index 0000000000..880c140fef --- /dev/null +++ b/2026/day-42/day-42-runners.md @@ -0,0 +1,58 @@ +# Day 42 - GitHub Actions: GitHub-Hosted vs Self-Hosted Runners + +Today I completed the hands-on tasks to understand where GitHub Actions jobs actually execute, exploring both GitHub-hosted runners and setting up my own AWS EC2 self-hosted runner. + +## Task 1: GitHub-Hosted Runners +I created a workflow with 3 jobs (`ubuntu-latest`, `windows-latest`, `macos-latest`) running in parallel. Each job successfully printed the OS name, runner hostname, and current user. + +![Task 1 - GitHub Hosted Runners](./task-01-hosted.png) + +**Notes:** +* **What is a GitHub-hosted runner?** It is a temporary virtual machine provided by GitHub that spins up just to run your job and gets destroyed immediately after. +* **Who manages it?** GitHub manages it completely. + +--- + +## Task 2: Explore What's Pre-installed +I ran a step on the `ubuntu-latest` runner to print the versions of Docker, Python, Node, and Git to see what is already available. + +![Task 2 - Pre-installed Tools](./task-02-preinstalled.png) + +**Notes:** +* **Why does it matter that runners come with tools pre-installed?** It saves a lot of execution time. If we had to write commands to download and install tools like Node or Docker on every single pipeline run, it would make our CI/CD process very slow. + +--- + +## Task 3: Set Up a Self-Hosted Runner +I created an AWS EC2 instance (Ubuntu) and followed the GitHub instructions to download and configure it as a self-hosted runner for my repository. + +**Proof:** Verified that my EC2 runner appeared in the Runners list with a green dot and Idle status. +![Task 3 - Self-Hosted Runner Idle](./task-03-idle-runner.png) + +--- + +## Task 4: Use Your Self-Hosted Runner +I created `.github/workflows/self-hosted.yml` and set `runs-on: self-hosted`. The job successfully ran on my EC2 machine. It printed my EC2 hostname, the working directory, and created a `my-proof.txt` file which I manually verified inside the server's terminal. + +--- + +## Task 5: Labels +I added a custom label `my-linux-runner` to my EC2 self-hosted runner. Then, I updated my workflow to use `runs-on: [self-hosted, my-linux-runner]` and triggered it. It successfully picked up the job based on the label. + +![Task 5 - Targeted Label Run](./task-05-labels-run.png) + +**Notes:** +* **Why are labels useful when you have multiple self-hosted runners?** Labels help route specific jobs to the correct machines. If you have different servers (e.g., one for testing, one for database, one with a GPU), labels ensure the job runs exactly on the hardware it is meant for. + +--- + +## Task 6: GitHub-Hosted vs Self-Hosted + +| Feature | GitHub-Hosted | Self-Hosted | +| :--- | :--- | :--- | +| **Who manages it?** | GitHub | Me (Developer / DevOps Engineer) | +| **Cost** | Free (up to monthly limit) | I pay for the server running it (AWS EC2 bill) | +| **Pre-installed tools** | Yes (Docker, Git, Node, Python, etc.) | No, I have to install everything manually | +| **Good for** | Standard CI/CD, quick and easy setups | Custom hardware, accessing private VPCs, large workloads | +| **Security concern** | Safe (machine is destroyed after run) | Risky (a bad workflow script could delete files on my actual server) | + diff --git a/2026/day-42/task-01-hosted.png b/2026/day-42/task-01-hosted.png new file mode 100644 index 0000000000..f25aaaad81 Binary files /dev/null and b/2026/day-42/task-01-hosted.png differ diff --git a/2026/day-42/task-02-preinstalled.png b/2026/day-42/task-02-preinstalled.png new file mode 100644 index 0000000000..f9dfe77227 Binary files /dev/null and b/2026/day-42/task-02-preinstalled.png differ diff --git a/2026/day-42/task-03-idle-runner.png b/2026/day-42/task-03-idle-runner.png new file mode 100644 index 0000000000..3d14d48b54 Binary files /dev/null and b/2026/day-42/task-03-idle-runner.png differ diff --git a/2026/day-42/task-04-selfhosted-run.png b/2026/day-42/task-04-selfhosted-run.png new file mode 100644 index 0000000000..86f8287e31 Binary files /dev/null and b/2026/day-42/task-04-selfhosted-run.png differ diff --git a/2026/day-42/task-05-labels-run.png b/2026/day-42/task-05-labels-run.png new file mode 100644 index 0000000000..42df59499f Binary files /dev/null and b/2026/day-42/task-05-labels-run.png differ diff --git a/2026/day-43/day-43-jobs-steps.md b/2026/day-43/day-43-jobs-steps.md new file mode 100644 index 0000000000..ed3d04cc4a --- /dev/null +++ b/2026/day-43/day-43-jobs-steps.md @@ -0,0 +1,57 @@ +# Day 43: Jobs, Steps, Env Vars & Conditionals in GitHub Actions + +Today, I leveled up my CI/CD game by moving from simple "dumb scripts" to **Smart and Dynamic Workflows**. I learned how to control the flow of a pipeline using dependencies, environment variables, job outputs, and conditional statements. + +--- + +## Task 1: Multi-Job Workflow (Dependency Chain) +Created a workflow with three jobs (`build`, `test`, `deploy`). I used the `needs` keyword to ensure strict execution order. If `build` fails, `test` won't run. If `test` fails, `deploy` is blocked. + +![Multi-Job Workflow Dependency](task-01-multi-job.png) + +--- + +## Task 2: Environment Variables +Practiced variable scoping at three different levels: +1. **Workflow Level:** Accessible to all jobs (e.g., `APP_NAME`). +2. **Job Level:** Accessible only inside a specific job (e.g., `ENVIRONMENT`). +3. **Step Level:** Accessible only inside a specific step (e.g., `VERSION`). +Also fetched dynamic GitHub Context variables like `${{ github.actor }}` and `${{ github.sha }}`. + +![Environment Variables Output](task-02-env-vars.png) + +--- + +## Task 3: Job Outputs (Passing Data Between Jobs) +Since every GitHub Actions job runs on a fresh, isolated Virtual Machine, they don't share memory. I created a job that generates today's date, saves it to `$GITHUB_OUTPUT`, and passes it to a completely independent second job. + +**Why pass outputs between jobs?** +In real-world scenarios, Job 1 might build the code and generate a dynamic version tag (like `v1.2.4`). Job 2 needs this exact tag to push the Docker image to a registry or deploy it to a server. Outputs bridge this gap between isolated VMs. + +![Job Outputs Passing Data](task-03-job-outputs.png) + +--- + +## Task 4: Conditionals & Error Handling +Used the `if` condition to make pipelines intelligent: +* Ran steps only if the branch was `main`. +* Intentionally failed a step but used `continue-on-error: true` to prevent pipeline crash. +* Used `if: failure()` to create a step that *only* runs when a previous step fails (perfect for sending Slack/Email alerts). + +![Conditionals and Alerts](task-04-conditionals.png) + +--- + +## Task 5: The Smart Pipeline +Combined all concepts into a final `smart-pipeline.yml`. +* `lint` and `test` jobs run in **parallel** to save time. +* `summary` job runs only after both succeed. +* Used context variables to dynamically print the commit message and check branch types. + +![Smart Pipeline Graph & Logs](task-05-smart-pipeline.png) + +--- + +## Key Takeaways +* **`needs:`** acts as the traffic controller of the pipeline. It defines strict prerequisites, ensuring downstream jobs only run when upstream jobs succeed. +* **`outputs:`** is the communication bridge. It allows securely passing dynamic, generated data across completely separate Virtual Machines (runners) during a single workflow run. \ No newline at end of file diff --git a/2026/day-43/task-01-multi-job.png b/2026/day-43/task-01-multi-job.png new file mode 100644 index 0000000000..4665867207 Binary files /dev/null and b/2026/day-43/task-01-multi-job.png differ diff --git a/2026/day-43/task-02-env-vars.png b/2026/day-43/task-02-env-vars.png new file mode 100644 index 0000000000..06c1190a3d Binary files /dev/null and b/2026/day-43/task-02-env-vars.png differ diff --git a/2026/day-43/task-03-job-outputs.png b/2026/day-43/task-03-job-outputs.png new file mode 100644 index 0000000000..ab86ebd4c7 Binary files /dev/null and b/2026/day-43/task-03-job-outputs.png differ diff --git a/2026/day-43/task-04-conditionals.png b/2026/day-43/task-04-conditionals.png new file mode 100644 index 0000000000..46f30fd395 Binary files /dev/null and b/2026/day-43/task-04-conditionals.png differ diff --git a/2026/day-43/task-05-smart-pipeline.png b/2026/day-43/task-05-smart-pipeline.png new file mode 100644 index 0000000000..4d46b70a52 Binary files /dev/null and b/2026/day-43/task-05-smart-pipeline.png differ diff --git a/2026/day-44/day-44-secrets-artifacts.md b/2026/day-44/day-44-secrets-artifacts.md new file mode 100644 index 0000000000..a3a4d242db --- /dev/null +++ b/2026/day-44/day-44-secrets-artifacts.md @@ -0,0 +1,69 @@ +# Day 44 - Secrets, Artifacts & Running Real Tests in CI + +Today's pipeline starts doing real work β€” storing sensitive values securely, saving build outputs, and running actual tests. + +--- + +## Task 1: GitHub Secrets +I created a repository secret named `MY_SECRET_MESSAGE` and wrote a workflow to read and print it. GitHub automatically masked the direct output with `***`. + +**Question: Why should you never print secrets in CI logs?** +- **Answer:** You should never print secrets directly because anyone with read access to the repository's action logs can see them if masking fails, or it can be logged in third-party systems. Printing them directly in a bash command (`run: echo ${{ secrets.XYZ }}`) also opens up the risk of command injection. + +*Proof of GitHub masking the secret:* +![Task 1 Secrets](task-1-secrets.png) + +--- + +## Task 2: Use Secrets as Environment Variables +Instead of printing directly, I mapped the secret to an environment variable (`MY_ENV_SECRET`) within the step. I also added `DOCKER_USERNAME` and `DOCKER_TOKEN` as secrets to the repository for future use. + +*Proof of securely loading secrets via Environment Variables:* +![Task 2 Env Secrets](task-2-env-secrets.png) + +--- + +## Task 3: Upload Artifacts +I created a step that generates a `test-report.txt` file and used `actions/upload-artifact@v4` to save it. + +**Verification:** Yes, after the workflow ran, the `daily-test-report` artifact was successfully generated, visible, and downloadable as a `.zip` file from the Actions summary tab. + +*Proof of Artifact available for download in GitHub Actions:* +![Task 3 Upload Artifact](task-3-upload-artifact.png) + +--- + +## Task 4: Download Artifacts Between Jobs +I created a multi-job workflow where `Job 1` generates a file and uploads it, and `Job 2` (which depends on Job 1) downloads that artifact and prints its contents. + +**Question: When would you use artifacts in a real pipeline?** +- **Answer:** Since every GitHub Actions job runs on a fresh, isolated virtual machine that gets destroyed after the job finishes, artifacts are crucial for: +1. Passing built/compiled files (like `.jar`, `.exe`, or docker builds) from a Build Job to a Deploy Job. +2. Saving test reports, code coverage, or error logs so developers can download and analyze them after the pipeline finishes. + +*Proof of Job 2 successfully downloading and reading the artifact from Job 1:* +![Task 4 Download Artifact](task-4-download-artifact.png) + +--- + +## Task 5: Run Real Tests in CI +I added a Python unit test (`tests/math_operations.py`) to the repository and created a workflow to run it. + +I intentionally broke the logic to ensure the pipeline goes **Red (Failed)**, then fixed the logic to verify the pipeline goes **Green (Passed)**. + +*Proof of failing test run (Pipeline breaks as expected):* +![Task 5a Test Fail](task-5a-test-fail.png) + +*Proof of passing test run (Pipeline fixed and green):* +![Task 5b Test Pass](task-5b-test-pass.png) + +--- + +## Task 6: Caching +I implemented `actions/cache@v4` to simulate caching dependencies. I ran it twice to observe the time difference between a cache-miss (downloading dependencies) and a cache-hit (restoring dependencies instantly). + +**Question: What is being cached and where is it stored?** +- **Answer:** Specific directories (like `node_modules` for Node.js or `~/.cache/pip` for Python) are zipped and cached. These files are stored securely on GitHub's internal cloud storage infrastructure. They are linked to a unique cache `key`, and if the key matches in future runs, GitHub restores the files directly to the runner VM, bypassing the internet download and saving time. + +*Proof of Caching execution in the workflow:* +![Task 6 Caching](task-6-caching.png) diff --git a/2026/day-44/task-1-secrets.png b/2026/day-44/task-1-secrets.png new file mode 100644 index 0000000000..2d2839ae57 Binary files /dev/null and b/2026/day-44/task-1-secrets.png differ diff --git a/2026/day-44/task-2-env-secrets.png b/2026/day-44/task-2-env-secrets.png new file mode 100644 index 0000000000..47b18e62a6 Binary files /dev/null and b/2026/day-44/task-2-env-secrets.png differ diff --git a/2026/day-44/task-3-upload-artifact.png b/2026/day-44/task-3-upload-artifact.png new file mode 100644 index 0000000000..1608b2642b Binary files /dev/null and b/2026/day-44/task-3-upload-artifact.png differ diff --git a/2026/day-44/task-4-download-artifact.png b/2026/day-44/task-4-download-artifact.png new file mode 100644 index 0000000000..b4497517c4 Binary files /dev/null and b/2026/day-44/task-4-download-artifact.png differ diff --git a/2026/day-44/task-5a-test-fail.png b/2026/day-44/task-5a-test-fail.png new file mode 100644 index 0000000000..1e5e1919cc Binary files /dev/null and b/2026/day-44/task-5a-test-fail.png differ diff --git a/2026/day-44/task-5b-test-pass.png b/2026/day-44/task-5b-test-pass.png new file mode 100644 index 0000000000..dd3077fb5d Binary files /dev/null and b/2026/day-44/task-5b-test-pass.png differ diff --git a/2026/day-44/task-6-caching.png b/2026/day-44/task-6-caching.png new file mode 100644 index 0000000000..71b068d8b4 Binary files /dev/null and b/2026/day-44/task-6-caching.png differ diff --git a/2026/day-45/day-45-docker-cicd.md b/2026/day-45/day-45-docker-cicd.md new file mode 100644 index 0000000000..07db68510c --- /dev/null +++ b/2026/day-45/day-45-docker-cicd.md @@ -0,0 +1,83 @@ +# Day 45: Docker Build & Push in GitHub Actions + +## Task 1: Prepare +Verified the Python Flask e-commerce application and the minimalistic production Dockerfile. Configured GitHub repository secrets `DOCKER_USERNAME` and `DOCKER_TOKEN` for Docker Hub authentication. + +![Task 1 - Prepare](task-1-prepare.png) + +## Task 2: Build the Docker Image in CI +Created the `.github/workflows/docker-publish.yml` to trigger on pushes to the `main` branch. The workflow checks out the code, sets up Docker Buildx, and builds the container image. + +**Complete Workflow YAML:** +```yaml +name: Docker Build and Publish + +on: + push: + branches: + - main + - test-pipeline + +jobs: + build-and-push: + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + - name: Login to Docker Hub + uses: docker/login-action@v3 + with: + username: ${{ secrets.DOCKER_USERNAME }} + password: ${{ secrets.DOCKER_TOKEN }} + + - name: Extract short SHA + id: vars + run: echo "sha_short=$(git rev-parse --short HEAD)" >> $GITHUB_ENV + + - name: Build and push + uses: docker/build-push-action@v5 + with: + context: . + push: ${{ github.ref == 'refs/heads/main' }} + tags: | + rashidkhan6685/my-ecommerce-app:latest + rashidkhan6685/my-ecommerce-app:sha-${{ env.sha_short }} +``` + +![Task 2 - Build](task-2-build.png) + +## Task 3: Push to Docker Hub +Configured the workflow to securely log into Docker Hub using secrets and push the image with dual tags (`latest` and `sha-`). + +**Docker Hub Link:** https://hub.docker.com/r/rashidkhan6685/my-ecommerce-app + +![Task 3 - Docker Hub](task-3-docker-hub.png) + +## Task 4: Only Push on Main +Added branch protection using the condition `push: ${{ github.ref == 'refs/heads/main' }}`. Verified that pushing to a feature branch triggers the build successfully in the runner cache but explicitly aborts the push to Docker Hub. + +![Task 4 - Branch Skip](task-4-branch-skip.png) + +## Task 5: Add a Status Badge +Obtained the workflow status badge URL from the Actions tab and embedded it into the repository's `README.md` to dynamically display a passing green build status. + +![Task 5 - Status Badge](task-5-status-badge.png) + +## Task 6: Pull and Run It +Pulled the automated cloud-built image to the local machine and verified the application execution. Overcame the local Apple Silicon architecture mismatch by enforcing the `--platform linux/amd64` flag during extraction and execution. + +**Question: What is the full journey from git push to a running container?** +- **Answer:** +1. **Push:** A developer pushes code changes to the GitHub `main` branch. +2. **Trigger:** GitHub Actions detects the push event and provisions an Ubuntu runner. +3. **Checkout & Auth:** The runner checks out the source code, configures Docker Buildx, and authenticates with Docker Hub using repository secrets. +4. **Build:** The runner executes the instructions within the Dockerfile to compile the container image locally in the CI environment. +5. **Tag & Push:** The built image is tagged with `latest` and the unique Git commit SHA, then pushed to the public Docker Hub registry. +6. **Pull:** The deployment server or local machine executes `docker pull` to download the newly pushed image from Docker Hub. +7. **Run:** The `docker run` command is executed to spin up the container, map ports, and bring the application live. + +![Task 6 - Local Run](task-6-local-run.png) diff --git a/2026/day-45/task-1-prepare.png b/2026/day-45/task-1-prepare.png new file mode 100644 index 0000000000..ce01ff99ff Binary files /dev/null and b/2026/day-45/task-1-prepare.png differ diff --git a/2026/day-45/task-2-build.png b/2026/day-45/task-2-build.png new file mode 100644 index 0000000000..b7bbd9fbda Binary files /dev/null and b/2026/day-45/task-2-build.png differ diff --git a/2026/day-45/task-3-docker-hub.png b/2026/day-45/task-3-docker-hub.png new file mode 100644 index 0000000000..7ff5823e18 Binary files /dev/null and b/2026/day-45/task-3-docker-hub.png differ diff --git a/2026/day-45/task-4-branch-skip.png b/2026/day-45/task-4-branch-skip.png new file mode 100644 index 0000000000..09d0c92d44 Binary files /dev/null and b/2026/day-45/task-4-branch-skip.png differ diff --git a/2026/day-45/task-5-status-badge.png b/2026/day-45/task-5-status-badge.png new file mode 100644 index 0000000000..c38a30e7de Binary files /dev/null and b/2026/day-45/task-5-status-badge.png differ diff --git a/2026/day-45/task-6-local-run.png b/2026/day-45/task-6-local-run.png new file mode 100644 index 0000000000..3f86fb91b0 Binary files /dev/null and b/2026/day-45/task-6-local-run.png differ diff --git a/2026/day-46/day-46-reusable-workflows.md b/2026/day-46/day-46-reusable-workflows.md new file mode 100644 index 0000000000..c53dd30598 --- /dev/null +++ b/2026/day-46/day-46-reusable-workflows.md @@ -0,0 +1,149 @@ +# Day 46: Reusable Workflows & Composite Actions + +## Task 1: Understand `workflow_call` +**1. What is a reusable workflow?** + - It is a standard GitHub Actions workflow designed to act like a function. Instead of being triggered directly by events like `push`, it is called by other workflows. + +**2. What is the `workflow_call` trigger?** + - It is the specific event trigger (`on: workflow_call`) that makes a workflow reusable, allowing it to define expected `inputs`, `secrets`, and `outputs`. + +**3. How is calling a reusable workflow different from using a regular action (`uses:`)?** + - A regular action is used *inside* a job's `steps`. A reusable workflow is an entire pipeline (with its own jobs and steps) that is called at the *job level*. + +**4. Where must a reusable workflow file live?** + - It must live inside the `.github/workflows/` directory. + +--- + +## Task 2: Create Your First Reusable Workflow +Created a central reusable pipeline (`.github/workflows/reusable-build.yml`) configured to accept `app_name`, `environment`, and a `docker_token` secret. + +**Reusable Workflow YAML:** +```yaml +name: Reusable Build Workflow +on: + workflow_call: + inputs: + app_name: + required: true + type: string + environment: + required: true + default: 'staging' + type: string + secrets: + docker_token: + required: true + outputs: + build_version: + value: ${{ jobs.build.outputs.version }} + +jobs: + build: + runs-on: ubuntu-latest + outputs: + version: ${{ steps.set_version.outputs.version_tag }} + steps: + - uses: actions/checkout@v4 + - run: echo "Building ${{ inputs.app_name }} for ${{ inputs.environment }}" + - run: | + if [ -n "${{ secrets.docker_token }}" ]; then + echo "Docker token is set: true" + else + echo "Docker token is set: false" + fi + - id: set_version + run: | + SHORT_SHA=$(git rev-parse --short HEAD) + VERSION="v1.0-$SHORT_SHA" + echo "version_tag=$VERSION" >> $GITHUB_OUTPUT +``` + +--- + +## Task 3: Create a Caller Workflow +Created a caller workflow (`.github/workflows/call-build.yml`) that triggers on a push to `main` and passes the required inputs to the reusable workflow. + +![Task 3 - Caller Run](task-3-caller-run.png) + +--- + +## Task 4: Add Outputs to the Reusable Workflow +Updated both workflows to pass dynamic data back to the caller. The reusable workflow generates a version string (e.g., `v1.0-76ee43a`), and a secondary job in the caller workflow consumes and prints it. + +**Caller Workflow YAML:** +```yaml +name: Call Reusable Workflow +on: + push: + branches: + - main + +jobs: + call-reusable-build: + uses: ./.github/workflows/reusable-build.yml + with: + app_name: "my-web-app" + environment: "production" + secrets: + docker_token: ${{ secrets.DOCKER_TOKEN }} + + print-output: + runs-on: ubuntu-latest + needs: call-reusable-build + steps: + - run: echo "The build version is: ${{ needs.call-reusable-build.outputs.build_version }}" +``` + +![Task 4 - Outputs](task-4-outputs.png) + +--- + +## Task 5: Create a Composite Action +Created a custom composite action at `.github/actions/setup-and-greet/action.yml` to bundle logic into reusable steps. Successfully passed inputs (`name`, `language`) and returned outputs while executing bash scripts. + +**Composite Action YAML:** +```yaml +name: 'Setup and Greet' +description: 'Greets the user in a specified language and prints runner info' +inputs: + name: + required: true + language: + required: false + default: 'en' +outputs: + greeted: + value: ${{ steps.greet.outputs.success }} +runs: + using: "composite" + steps: + - id: greet + shell: bash + run: | + if [ "${{ inputs.language }}" = "hi" ]; then + echo "Namaste, ${{ inputs.name }}!" + else + echo "Hello, ${{ inputs.name }}!" + fi + echo "success=true" >> $GITHUB_OUTPUT + - shell: bash + run: | + echo "Current Date: $(date)" + echo "Runner OS: $RUNNER_OS" +``` + +![Task 5 - Composite Action](task-5-composite-action.png) + +--- + +## Task 6: Reusable Workflow vs Composite Action + +| Feature | Reusable Workflow | Composite Action | +| :--- | :--- | :--- | +| **Triggered by** | `workflow_call` | `uses:` in a step | +| **Can contain jobs** | Yes (It defines its own jobs) | No (It runs *inside* a job) | +| **Can contain multiple steps** | Yes | Yes | +| **Lives where** | Strictly in `.github/workflows/` | Anywhere (usually `.github/actions/name/action.yml`) | +| **Can accept secrets directly** | Yes (Using `secrets:` block) | No (Must be passed as standard `inputs`) | +| **Best for** | Standardizing entire CI/CD pipelines (e.g., Build-Test-Deploy) | Bundling a sequence of repetitive steps | diff --git a/2026/day-46/task-3-caller-run.png b/2026/day-46/task-3-caller-run.png new file mode 100644 index 0000000000..0afc474ca6 Binary files /dev/null and b/2026/day-46/task-3-caller-run.png differ diff --git a/2026/day-46/task-4-outputs.png b/2026/day-46/task-4-outputs.png new file mode 100644 index 0000000000..7018c54eb6 Binary files /dev/null and b/2026/day-46/task-4-outputs.png differ diff --git a/2026/day-46/task-5-composite-action.png b/2026/day-46/task-5-composite-action.png new file mode 100644 index 0000000000..8e305552a5 Binary files /dev/null and b/2026/day-46/task-5-composite-action.png differ diff --git a/2026/day-47/day-47-advanced-triggers.md b/2026/day-47/day-47-advanced-triggers.md new file mode 100644 index 0000000000..e901257950 --- /dev/null +++ b/2026/day-47/day-47-advanced-triggers.md @@ -0,0 +1,57 @@ +# Day 47: Advanced Triggers - PR Events, Cron Schedules & Event-Driven Pipelines + +This document contains the execution details, answers to challenge questions, and screenshots for Day 47 of the 90DaysOfDevOps challenge. + +## Task 1: Pull Request Event Types +Created `.github/workflows/pr-lifecycle.yml` to trigger on specific PR activity types (opened, synchronize, reopened, closed). Added a conditional step that only runs when the PR is successfully merged. + +**Proof of Execution:** +![Task 1 PR Lifecycle](task-1-pr-lifecycle.png) + +## Task 2: PR Validation Workflow +Created `.github/workflows/pr-checks.yml` as a PR gate on the main branch. It verifies the branch name (feature/*, fix/*, docs/*), checks file sizes (fails if > 1MB), and warns if the PR description is empty. + +**Proof of Execution:** +![Task 2 PR Checks](task-2-pr-checks.png) + +## Task 3: Scheduled Workflows (Cron Deep Dive) +Created `.github/workflows/scheduled-tasks.yml` with a cron schedule and `workflow_dispatch` for manual testing. + +**Notes & Answers:** +* The cron expression for every weekday at 9 AM IST: `30 3 * * 1-5` +* The cron expression for the first day of every month at midnight: `0 0 1 * *` +* Why GitHub says scheduled workflows may be delayed or skipped on inactive repos: To conserve compute resources, GitHub automatically disables scheduled workflows if the repository has been inactive for 60 days. They must be re-enabled manually. + +**Proof of Execution:** +![Task 3 Cron Job](task-3-cron.png) + +## Task 4: Path & Branch Filters +Created `.github/workflows/smart-triggers.yml` to trigger only when files in `src/` or `app/` change. Created another workflow using `paths-ignore` to skip runs when only documentation (`*.md`) is changed. + +**Notes & Answers:** +* When would you use paths vs paths-ignore? + - Use `paths` when a workflow is strictly tied to a specific component (e.g., running backend tests only when backend code changes). Use `paths-ignore` for general CI workflows where you want to run checks on all code changes except for specific non-functional files like documentation, READMEs, or images. + +**Proof of Execution:** +![Task 4 Path Filters](task-4-filters.png) + +## Task 5: workflow_run β€” Chain Workflows Together +Created `tests.yml` and `deploy-after-tests.yml`. The deployment workflow uses the `workflow_run` trigger to start only after the test workflow completes successfully. + +**Notes & Answers:** +* Explanation of workflow_run vs workflow_call: + - `workflow_run` is event-driven. It automatically triggers a workflow based on the execution or completion of another workflow (like a chain reaction). + - `workflow_call` makes a workflow reusable. It does not trigger automatically; instead, another workflow explicitly calls it like a function passing inputs and secrets. + +**Proof of Execution:** +![Task 5 Workflow Chaining](task-5-workflow-run.png) + +## Task 6: repository_dispatch β€” External Event Triggers +Created `.github/workflows/external-trigger.yml` that listens for the `deploy-request` event type. Triggered it using the GitHub CLI with a custom JSON client payload. + +**Notes & Answers:** +* When would an external system trigger a pipeline? + - This is used when integrating external tools with GitHub Actions. For example, triggering a deployment via a custom Slack bot command, running an emergency rollback when a monitoring tool like Datadog detects a production failure, or triggering a build when content is updated in an external CMS. + +**Proof of Execution:** +![Task 6 Repository Dispatch](task-6-repository-dispatch.png) \ No newline at end of file diff --git a/2026/day-47/task-1-pr-lifecycle.png b/2026/day-47/task-1-pr-lifecycle.png new file mode 100644 index 0000000000..72a374f9ad Binary files /dev/null and b/2026/day-47/task-1-pr-lifecycle.png differ diff --git a/2026/day-47/task-2-pr-checks.png b/2026/day-47/task-2-pr-checks.png new file mode 100644 index 0000000000..d57f2bb233 Binary files /dev/null and b/2026/day-47/task-2-pr-checks.png differ diff --git a/2026/day-47/task-3-cron.png b/2026/day-47/task-3-cron.png new file mode 100644 index 0000000000..6da3813c97 Binary files /dev/null and b/2026/day-47/task-3-cron.png differ diff --git a/2026/day-47/task-4-filters.png b/2026/day-47/task-4-filters.png new file mode 100644 index 0000000000..5db481ee28 Binary files /dev/null and b/2026/day-47/task-4-filters.png differ diff --git a/2026/day-47/task-5-workflow-run.png b/2026/day-47/task-5-workflow-run.png new file mode 100644 index 0000000000..22c9db6946 Binary files /dev/null and b/2026/day-47/task-5-workflow-run.png differ diff --git a/2026/day-47/task-6-repository-dispatch.png b/2026/day-47/task-6-repository-dispatch.png new file mode 100644 index 0000000000..363612bddc Binary files /dev/null and b/2026/day-47/task-6-repository-dispatch.png differ diff --git a/2026/day-48/day-48-actions-project.md b/2026/day-48/day-48-actions-project.md new file mode 100644 index 0000000000..6f3c7f04d2 --- /dev/null +++ b/2026/day-48/day-48-actions-project.md @@ -0,0 +1,70 @@ +# Day 48 Actions Project: End-to-End CI/CD Pipeline + +This document covers the complete CI/CD pipeline built for the DevBoard backend. It includes automated testing, Docker containerization, security scanning, and scheduled health checks using GitHub Actions. + +## Pipeline Architecture Flow +1. PR Opened -> pr-pipeline -> Code Checkout -> Setup Environment -> Run Tests -> Output PR check status. +2. Push to Main -> main-pipeline -> Run Tests -> Docker Build & Push (tag: latest & short-sha) -> Trivy Vulnerability Scan -> Manual Deployment Approval -> Deploy step. +3. Every 12 Hours (Cron) -> health-check -> Pull Docker Image -> Setup custom Docker network -> Initialize Postgres Database -> Run Backend Container -> Curl /health endpoint -> Generate Action Summary -> Teardown network and containers. + +## Task 1: Set Up the Project Repo +Configured the github-actions-capstone repository containing the DevBoard Go/Node backend. Added a Dockerfile to containerize the application and set up basic health check endpoints for testing. + +**Proof of Execution:** +![Set Up the Project Repo](task1-repo-setup.png) + +## Task 2: Reusable Workflow - Build & Test +Created `.github/workflows/reusable-build-test.yml` using the `workflow_call` trigger. It handles the CI part by checking out the code, setting up the runtime environment, installing dependencies, and running tests. It does not contain any deployment logic. + +## Task 3: Reusable Workflow - Docker Build & Push +Created `.github/workflows/reusable-docker.yml` (triggered via `workflow_call`). It securely logs into Docker Hub using GitHub Secrets (`DOCKER_USERNAME`, `DOCKER_TOKEN`), builds the image, and pushes it to the registry with two tags: `latest` and a dynamically generated `sha-`. + +## Task 4: PR Pipeline +Created `.github/workflows/pr-pipeline.yml`. It triggers strictly on `pull_request` to the `main` branch. It calls the reusable build-test workflow to ensure code quality and runs a standalone job to print a summary comment. It explicitly avoids building or pushing Docker images to prevent registry clutter from untested code. + +**Proof of Execution:** +![PR Pipeline Run](task4-pr-checks.png) + +## Task 5: Main Branch Pipeline +Created `.github/workflows/main-pipeline.yml` for the core CD process. It triggers on push to main, running the test workflow, followed by the Docker build and push workflow. +For deployment, I set up a `production` environment in GitHub repository settings with protection rules. The final deploy job waits in a pending state until a reviewer manually approves the deployment. + +**Proof of Execution:** +![Main Pipeline Execution](task5-pipeline-graph.png) +![DockerHub Image](task5-dockerhub.png) + +**Docker Hub Image Link:** +https://hub.docker.com/r/rashidkhan6685/devboard-backend + +## Task 6: Scheduled Health Check +Created `.github/workflows/health-check.yml` running on a `0 */12 * * *` cron schedule. + +**Extra Work & Troubleshooting:** +During the initial run, the health check failed (exit code 1) because the DevBoard backend crashes immediately if it cannot connect to a PostgreSQL database on startup. A simple `docker run` was not enough. +To fix this inside the GitHub runner, I added steps to mimic the production environment: +1. Created a custom Docker network (`devboard-net`). +2. Spun up a `postgres:16-alpine` container, mapping the initialization volume (`-v "$PWD/init/postgres":/docker-entrypoint-initdb.d:ro`) to load required tables. +3. Spun up the backend container connected to the same network, passing the `POSTGRES_URL` environment variable. +4. Mapped the container port to host port 8081 and ran the curl command against `http://localhost:8081/health`. +5. Created the markdown summary and added an `if: always()` cleanup step to remove containers and the custom network. + +**Proof of Execution:** +![Scheduled Health Check](task6-healthcheck.png) + +## Task 7: Badges & Documentation +Status badges for `pr-pipeline`, `main-pipeline`, and `health-check` have been added to the root `README.md`. This architecture file was created to document the workflow files and execution proofs. + +## Brownie Points: Security Pipeline (DevSecOps) +Implemented a DevSecOps step in the main pipeline using the `aquasecurity/trivy-action`. +It runs immediately after the Docker image is built and pushed. It scans the image for vulnerabilities, is configured to fail the pipeline only if a `CRITICAL` severity CVE is found, and successfully outputs a scan report. The report is uploaded as an artifact (`trivy-security-report.zip`) which can be downloaded from the Action summary. + +**Proof of Security Scan Artifact:** +![Trivy Security Scan](task7-brownie-trivy.png) + +## What I would improve next +1. **Server Deployment:** Swap the dummy deploy echo statement with a real SSH action (like Appleboy SSH) to pull the image and run it on an AWS EC2 instance. +2. **Persistent Storage:** Implement Docker Named Volumes on the EC2 host so PostgreSQL data persists across multiple automated container restarts. +3. **Automated Rollback:** Add logic to revert to the previous Git SHA image if the post-deployment health check fails. + +## Workflow YAML Code +[Note: Insert the raw code of your YAML files here if required by the submission guidelines] \ No newline at end of file diff --git a/2026/day-48/task1-repo-setup.png b/2026/day-48/task1-repo-setup.png new file mode 100644 index 0000000000..5fb9700332 Binary files /dev/null and b/2026/day-48/task1-repo-setup.png differ diff --git a/2026/day-48/task4-pr-checks.png b/2026/day-48/task4-pr-checks.png new file mode 100644 index 0000000000..ff7561a737 Binary files /dev/null and b/2026/day-48/task4-pr-checks.png differ diff --git a/2026/day-48/task5-dockerhub.png b/2026/day-48/task5-dockerhub.png new file mode 100644 index 0000000000..47f9782948 Binary files /dev/null and b/2026/day-48/task5-dockerhub.png differ diff --git a/2026/day-48/task5-pipeline-graph.png b/2026/day-48/task5-pipeline-graph.png new file mode 100644 index 0000000000..3d335f20b7 Binary files /dev/null and b/2026/day-48/task5-pipeline-graph.png differ diff --git a/2026/day-48/task6-healthcheck.png b/2026/day-48/task6-healthcheck.png new file mode 100644 index 0000000000..dff9e4e868 Binary files /dev/null and b/2026/day-48/task6-healthcheck.png differ diff --git a/2026/day-48/task7-brownie-trivy.png b/2026/day-48/task7-brownie-trivy.png new file mode 100644 index 0000000000..241ecc5dc2 Binary files /dev/null and b/2026/day-48/task7-brownie-trivy.png differ diff --git a/2026/day-49/day-49-devsecops.md b/2026/day-49/day-49-devsecops.md new file mode 100644 index 0000000000..f239f66777 --- /dev/null +++ b/2026/day-49/day-49-devsecops.md @@ -0,0 +1,121 @@ +# Day 49: DevSecOps - Adding Security to CI/CD Pipeline + +## What is DevSecOps? + - DevSecOps is the practice of embedding automated security checks directly into the CI/CD pipeline, rather than treating security as a final, separate phase. By integrating tools like image scanners and secret detectors early in the process, we can proactively catch vulnerabilities, leaked credentials, and insecure dependencies in minutes before they are merged into production. + +--- + +## Task 1: Scan Your Docker Image for Vulnerabilities +Added the Aqua Security Trivy action to the main branch pipeline to scan the Docker image for known CVEs after the build step and before pushing it to Docker Hub. + +**My Notes:** +* **Base Image Used:** The Docker image uses `python:3.13` built on the `debian:13.7` base operating system. +* **CVEs Found:** Trivy detected a total of 54 HIGH vulnerabilities and 0 CRITICAL vulnerabilities. + * OS-level (50): Primarily found in the `util-linux` package (e.g., CVE-2026-76642). + * Python-pkg (4): Included vulnerabilities in `setuptools` and `urllib3`. + * Since the pipeline was configured to fail only on CRITICAL (`exit-code: '1'`, `severity: 'CRITICAL'`), the image passed the check successfully. + +> **Screenshot of Trivy Scan Output:** +![Trivy Scan](./task-1-trivy-scan.png) + +--- + +## Task 2: Enable GitHub's Built-in Secret Scanning +Enabled Secret Scanning and Push Protection in the repository settings under Code security and analysis to prevent credential leaks. + +**What I Learned:** +* **Secret Scanning vs. Push Protection:** + - Secret scanning is a reactive measure that scans the repository's history and alerts administrators if secrets are found. Push protection is a proactive measure that intercepts the `git push` command and outright blocks the commit from reaching the repository if a supported secret is detected. +* **Leaked AWS Key Handling:** + - If an AWS key is pushed, GitHub immediately notifies AWS via their Partner Program. AWS automatically attaches an `AWSCompromisedKeyQuarantineV2` policy to the IAM user/role to restrict high-risk operations (like spinning up EC2 instances) and notifies the account owner. + +> **Screenshot of Secret Scanning:** +![Secret Scanning](./task-2-secret-scanning.png) + +--- + +## Task 3: Scan Dependencies for Known Vulnerabilities +Added `actions/dependency-review-action@v4` to the PR workflow with `fail-on-severity: critical`. + +**What I Learned:** +* This action specifically targets `pull_request` events. It checks any new dependencies (like pip or npm packages) introduced in the PR against a vulnerability database. If a developer attempts to introduce a critically vulnerable package, the check fails and blocks the PR from merging into the main branch. + +> **Screenshot of Dependency Review:** +![Dependency Review](./task-3-dependency-review.png) + +--- + +## Task 4: Add Permissions to Your Workflows +Locked down workflows to enforce the Principle of Least Privilege by explicitly declaring permissions. + +```yaml +permissions: + contents: read +``` + +**My Notes:** +* **Why limit workflow permissions?** + - Workflows should only have the exact access they require. If a compromised third-party GitHub Action has `write` access, an attacker could exploit it to silently push malicious code, alter releases, or steal repository secrets. + +> **Screenshot of Workflow Permissions:** +> ![Permissions Check](./task-4-permissions.png) + +--- + +## Task 5: The Full Secure Pipeline (Updated Diagram) + +Here is the updated architecture of the DevSecOps pipeline incorporating both official tasks and custom security integrations: + +```text +PR opened + -> build & test + -> dependency vulnerability check [Security Step] + -> PR checks pass or fail + +Merge to main + -> devsecops_pipeline.yml (Master Orchestration) + -> code linting (Ruff) & testing + -> Docker build + -> Trivy image scan (fail on CRITICAL) [Security Step] + -> SonarCloud SAST Code Analysis [Security Step] + -> Docker push (only if scans pass) + -> Deploy to AWS EC2 (Self-Hosted Runner) + -> DAST (OWASP ZAP Baseline Scan) [Security Step] + +Always active + -> GitHub secret scanning [Security Step] + -> push protection for secrets [Security Step] +``` + +--- + +## Official Brownie Points (Implemented & Researched) + +1. **Pin Actions to Commit SHAs:** + - To protect against supply chain attacks where a tag (like `@v3`) is silently changed by the author, I pinned external actions to their exact 40-character commit SHAs. + *Example implemented in DAST:* `uses: zaproxy/action-baseline@7c4deb1000000000000000000000000000000000` +2. **Upload Scan Results to GitHub Security Tab:** + - By changing Trivy's format to SARIF, we can upload the scan output directly to GitHub's native Security tab using the CodeQL action: + ```yaml + - uses: aquasecurity/trivy-action@master + with: + image-ref: 'username/app:latest' + format: 'sarif' + output: 'trivy-results.sarif' + - uses: github/codeql-action/upload-sarif@v3 + with: + sarif_file: 'trivy-results.sarif' + ``` +3. **Learn About OIDC (Keyless Authentication):** + - OpenID Connect (OIDC) allows GitHub Actions to securely authenticate to cloud providers (AWS, GCP, Azure) using short-lived tokens generated via a trust relationship. This completely eliminates the need to store long-lived, hardcoded cloud credentials in GitHub Secrets, significantly reducing the risk of credential theft. + +--- + +## Extra Custom Brownie Points (Real-World Implementation) + +Beyond the standard requirements, I implemented a full-scale Master DevSecOps Pipeline (`devsecops_pipeline.yml`) utilizing `workflow_call` to orchestrate multiple advanced security stages: + +* **SonarCloud SAST Integration:** Integrated SonarQube scanning and resolved major Code Smells/Vulnerabilities. + * **Fixed Script Injection (Rule S7630):** Prevented shell script injection by securely mapping user-controlled variables (e.g., `github.head_ref`) to `env` blocks instead of injecting them directly into `run` bash blocks. +* **DAST via OWASP ZAP:** Added dynamic testing (`dast.yml`) to scan the live AWS EC2 endpoint for vulnerabilities automatically post-deployment. +* **Resolved Deployment Blockers:** Debugged and resolved complex EC2 runner issues, including Docker Socket (`docker.sock`) permission denials, Exit Code 127 (missing Ruff linter), and port binding conflicts. diff --git a/2026/day-49/task-1-trivy-scan.png b/2026/day-49/task-1-trivy-scan.png new file mode 100644 index 0000000000..8681f502b0 Binary files /dev/null and b/2026/day-49/task-1-trivy-scan.png differ diff --git a/2026/day-49/task-2-secret-scanning.png b/2026/day-49/task-2-secret-scanning.png new file mode 100644 index 0000000000..98dba5a528 Binary files /dev/null and b/2026/day-49/task-2-secret-scanning.png differ diff --git a/2026/day-49/task-3-dependency-review.png b/2026/day-49/task-3-dependency-review.png new file mode 100644 index 0000000000..c03b314cad Binary files /dev/null and b/2026/day-49/task-3-dependency-review.png differ diff --git a/2026/day-49/task-4-permissions.png b/2026/day-49/task-4-permissions.png new file mode 100644 index 0000000000..c1fd255d0b Binary files /dev/null and b/2026/day-49/task-4-permissions.png differ diff --git a/2026/day-50/day-50-k8s-setup.md b/2026/day-50/day-50-k8s-setup.md new file mode 100644 index 0000000000..f2e5bde8f6 --- /dev/null +++ b/2026/day-50/day-50-k8s-setup.md @@ -0,0 +1,72 @@ +# Day 50: Kubernetes Architecture and Cluster Setup + +## Task 1: The Kubernetes Story + +**Q1: Why was Kubernetes created? What problem does it solve that Docker alone cannot?** + - **Answer:** Kubernetes was created as an open-source container orchestration software for the management of containers at scale. While containerizing an application is highly versatile, supporting a distributed architecture with hundreds of identical containers behind a load balancer is complex and error-prone. Kubernetes solves problems that standalone container engines cannot by automatically monitoring active workloads and reconciling any differences between the cluster's desired state and its actual state. If a worker node machine fails entirely, Kubernetes automatically recreates those containers onto another healthy machine (self-healing), and it continuously redistributes containers to balance workloads. + +**Q2: Who created Kubernetes and what was it inspired by?** + - **Answer:** Kubernetes is an open-source container orchestration platform that was originally designed by Google. It was heavily inspired by "Borg," Google's highly secretive internal system that they used to manage massive, large-scale distributed production workloads for over a decade. + +**Q3: What does the name "Kubernetes" mean?** + - **Answer:** The name "Kubernetes" originates from the Greek language, meaning "helmsman" or "pilot" (the person who steers a ship). It is commonly abbreviated as "K8s" because there are exactly 8 letters between the "K" and the "s" in the word Kubernetes. + +--- + +## Task 2: Kubernetes Architecture + +**Kubernetes Architecture Diagram:** +```text +[ CONTROL PLANE (Master) ] + β”œβ”€β”€ kube-apiserver <-- Front-end/API gateway of the control plane + β”œβ”€β”€ etcd <-- Consistent and highly-available key-value backing store + β”œβ”€β”€ kube-scheduler <-- Watches for newly created Pods and assigns them to Nodes + └── kube-controller-manager <-- Runs controller processes (maintains desired state) + + ↕ (Network Communication via API) + +[ WORKER NODE(S) ] + β”œβ”€β”€ kubelet <-- An agent that ensures containers are running in a Pod + β”œβ”€β”€ kube-proxy <-- Maintains network rules to allow communication to Pods + └── Container Runtime <-- Software responsible for running containers (e.g., containerd/Docker) +``` + +![Kubernetes-Architechture-Diagram](k8s-Architecture-diagram.png) + +**Q1: What happens when you run `kubectl apply -f pod.yaml`?** + - **Answer:** `kubectl` submits the desired Pod state to the kube-apiserver via a REST API call. The API server authenticates the request and persists the initial configuration into the etcd backing store. The kube-scheduler continuously watches the API server; it notices the newly created Pod with no assigned node, evaluates resource requirements, and selects an optimal Worker Node. The API server updates etcd with this binding. Finally, the kubelet on the target Worker Node detects the assignment via the API server, instructs the Container Runtime to start the container, and reports the Pod's running status back to the API server. + +**Q2: What happens if the API server goes down?** + - **Answer:** The kube-apiserver is the front end of the Kubernetes control plane. If it goes down, the cluster becomes inaccessible for administrative commands (e.g., `kubectl` will timeout). However, already scheduled and running Pods on Worker Nodes will continue to run without interruption because the kubelet and Container Runtime manage local execution. The cluster simply cannot be updated or scaled until the API server is restored. + +**Q3: What happens if a worker node goes down?** + - **Answer:** The kubelet continuously sends health checks (heartbeats) to the control plane. If a worker node goes down and stops sending heartbeats, the Node controller (part of the kube-controller-manager) notices the timeout. It marks the node's status as `Unknown` or `NotReady`. The controller then evicts the Pods from the unavailable node and reschedules them onto other healthy, available worker nodes to maintain the desired cluster state. + +--- + +## Task 4: Local Cluster Setup + +**Which tool did you choose and why?** + - **Answer:** I chose **kind (Kubernetes in Docker)** to set up my local cluster. According to the official Kubernetes SIGs documentation, `kind` is a tool for running local Kubernetes clusters using Docker container "nodes". I selected it because it is significantly more lightweight and faster than provisioning full Virtual Machines (which alternative tools like minikube often require). Since my macOS development environment already has Docker Desktop configured, `kind` natively leverages this existing container runtime, minimizing resource overhead and providing a highly efficient cluster bootstrapping experience. + +--- + +## Task 5: Explore Your Cluster + +> **Screenshot of Nodes and Kube-System Pods:** +> ![Kubernetes Nodes and Pods](./task-5-k8s-explore.png) + +**What each `kube-system` pod does:** +* **kube-apiserver:** The component of the Kubernetes control plane that exposes the Kubernetes API. It acts as the front end for the Kubernetes control plane. +* **etcd:** A consistent and highly-available key-value store used as Kubernetes' backing store for all cluster data. +* **kube-scheduler:** The control plane component that watches for newly created Pods with no assigned node, and selects a node for them to run on. +* **kube-controller-manager:** The control plane component that runs controller processes, constantly watching the state of the cluster through the apiserver and making changes attempting to move the current state towards the desired state. +* **kube-proxy:** A network proxy that runs on each node in your cluster, implementing part of the Kubernetes Service concept by maintaining network rules on nodes. +* **coredns:** A flexible, extensible DNS server that serves as the Kubernetes cluster DNS, allowing pods to discover services by name. + +--- + +## Task 6: Cluster Lifecycle & Configuration + +**What is a kubeconfig? Where is it stored on your machine?** + - **Answer:** According to the official Kubernetes documentation, a `kubeconfig` file is a configuration file used by `kubectl` to organize information about clusters, users, namespaces, and authentication mechanisms. The `kubectl` command-line tool uses kubeconfig files to find the information it needs to choose a cluster and communicate with the API server of a cluster. By default, `kubectl` looks for a file named `config` located in the `$HOME/.kube` directory on the local machine (e.g., `~/.kube/config`). \ No newline at end of file diff --git a/2026/day-50/k8s-Architecture-diagram.png b/2026/day-50/k8s-Architecture-diagram.png new file mode 100644 index 0000000000..984e91a4b8 Binary files /dev/null and b/2026/day-50/k8s-Architecture-diagram.png differ diff --git a/2026/day-50/task-5-k8s-explore.png b/2026/day-50/task-5-k8s-explore.png new file mode 100644 index 0000000000..e0caa39291 Binary files /dev/null and b/2026/day-50/task-5-k8s-explore.png differ diff --git a/2026/day-51/day-51-pods.md b/2026/day-51/day-51-pods.md new file mode 100644 index 0000000000..510d5b2c3d --- /dev/null +++ b/2026/day-51/day-51-pods.md @@ -0,0 +1,98 @@ +# Day 51: Kubernetes Manifests and Your First Pods + +## Task 1: Create Your First Pod (Nginx) + +**The Four Required Fields of a Kubernetes Manifest:** +According to the official Kubernetes API documentation, every resource manifest requires four top-level fields: +1. **`apiVersion`**: Defines the versioned schema (e.g., `v1` for Pods). +2. **`kind`**: Represents the resource type you are creating (e.g., `Pod`). +3. **`metadata`**: Contains data that helps uniquely identify the object (e.g., `name`, `labels`, `namespace`). +4. **`spec`**: Describes the actual desired state of the object (e.g., containers, images, ports). + +**Manifest (`nginx-pod.yaml`):** +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: nginx-pod + labels: + app: nginx +spec: + containers: + - name: nginx + image: nginx:latest + ports: + - containerPort: 80 +``` + +**Proof of Execution:** +![Task 1 - Nginx Pod Running](./task-1-nginx-pod.png) + +--- + +## Task 2: Create a Custom Pod (BusyBox) + +**Manifest (`busybox-pod.yaml`):** +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: busybox-pod + labels: + app: busybox + environment: dev +spec: + containers: + - name: busybox + image: busybox:latest + command: ["sh", "-c", "echo Hello from BusyBox && sleep 3600"] +``` +*Note: A `command` field is explicitly required here. BusyBox does not run a continuous background daemon by default. Without a long-running process (like `sleep 3600`), the container would exit immediately upon startup, causing a `CrashLoopBackOff` state.* + +**Proof of Execution:** +![Task 2 - BusyBox Logs](./task-2-busybox-logs.png) + +--- + +## Task 3: Imperative vs Declarative + +* **Imperative Management (`kubectl run`):** + - Direct CLI commands to create/modify resources. Fast for one-off tasks and generating YAML templates via `--dry-run`, but hard to track. +* **Declarative Management (`kubectl apply -f`):** + - Writing YAML configuration files specifying the *desired state*. Best for production, version control (GitOps), and team collaboration. + +--- + +## Task 4: Validate Before Applying + +**What error does Kubernetes give when the image field is missing?** + - When applying a broken manifest (missing the `image` field) and validating it via the API server (`--dry-run=server`), Kubernetes strictly catches the error before creation: +`The Pod "broken-pod" is invalid: spec.containers[0].image: Required value` + +--- + +## Task 5: Pod Labels and Filtering + +**My Third Pod Manifest (`third-pod.yaml`):** +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: labeled-pod + labels: + app: frontend + environment: staging + team: devops +spec: + containers: + - name: my-container + image: nginx:alpine +``` +*Filtering pods based on labels ensures you can easily group, manage, or delete specific resources without affecting the rest of the cluster.* + +--- + +## Task 6: Clean Up + +**What happens when you delete a standalone Pod?** + - When you delete a standalone Pod, it is terminated and gone forever. There is no controller (like a ReplicaSet or Deployment) actively monitoring its state to recreate it. This is why bare Pods are rarely used directly in production environments. \ No newline at end of file diff --git a/2026/day-51/task-1-nginx-pod.png b/2026/day-51/task-1-nginx-pod.png new file mode 100644 index 0000000000..e0e0672119 Binary files /dev/null and b/2026/day-51/task-1-nginx-pod.png differ diff --git a/2026/day-51/task-2-busybox-logs.png b/2026/day-51/task-2-busybox-logs.png new file mode 100644 index 0000000000..272dff13e5 Binary files /dev/null and b/2026/day-51/task-2-busybox-logs.png differ diff --git a/2026/day-52/day-52-namespaces-deployments.md b/2026/day-52/day-52-namespaces-deployments.md new file mode 100644 index 0000000000..eb53cf182a --- /dev/null +++ b/2026/day-52/day-52-namespaces-deployments.md @@ -0,0 +1,61 @@ +# Day 52: Kubernetes Namespaces and Deployments + +## 1. Namespaces: What and Why? +**What:** Namespaces act as virtual clusters inside a single physical Kubernetes cluster. +**Why:** They provide logical isolation for resources. For example, creating `dev`, `staging`, and `prod` namespaces prevents naming collisions (e.g., having an `nginx-pod` in both dev and prod) and allows for strict access control and resource quotas across different teams. + +--- + +## 2. My First Deployment Manifest +**Manifest (`nginx-deployment.yaml`):** +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: nginx-deployment + namespace: dev + labels: + app: nginx +spec: + replicas: 3 + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.24 + ports: + - containerPort: 80 +``` +**Explanation of Sections:** +* **`kind: Deployment`**: Specifies we are creating a deployment controller, not a bare pod. +* **`replicas: 3`**: Tells the controller to maintain exactly 3 running pod instances at all times. +* **`selector.matchLabels`**: The mechanism the deployment uses to identify which pods it owns and manages. +* **`template`**: The blueprint used to create the actual pods when scaling up or replacing crashed pods. + +**Proof of Execution (Deployment & Pods across namespaces):** +![Task 3 - Deployment](./task-3-deployment.png) + +--- + +## 3. Self-Healing: Deployment vs. Standalone Pod +* **Standalone Pod:** When deleted or crashed, it is gone forever. No controller is watching to revive it. +* **Deployment Pod:** When I deleted `nginx-deployment-7f5f95d8d-5c66f`, the Deployment controller immediately detected that current replicas (2) did not match desired replicas (3). It instantly spun up a new replacement pod with a completely different hash name (`kzpn4`). This is Kubernetes self-healing in action. + +--- + +## 4. Scaling (Imperative vs Declarative) +Scaling adjusts the number of pod replicas to handle load changes. +* **Imperative Scaling:** Executed directly via CLI: `kubectl scale deployment nginx-deployment --replicas=5 -n dev`. Kubernetes immediately provisions 2 extra pods. +* **Declarative Scaling:** Editing the `replicas: 5` field in `nginx-deployment.yaml` and running `kubectl apply -f nginx-deployment.yaml`. + +--- + +## 5. Rolling Updates and Rollbacks +* **Rolling Updates:** Updates application versions with **zero downtime**. When I updated to `nginx:1.25`, Kubernetes created a new replica set, spun up new pods one by one, and only terminated the old pods once the new ones were healthy and ready to receive traffic. +* **Rollbacks:** If a deployment fails or contains bugs, we can instantly revert to the previous stable state using `kubectl rollout undo deployment/nginx-deployment -n dev`. My verification `describe` command confirmed the image reverted perfectly back to `nginx:1.24`. \ No newline at end of file diff --git a/2026/day-52/task-3-deployment.png b/2026/day-52/task-3-deployment.png new file mode 100644 index 0000000000..ba8ab98620 Binary files /dev/null and b/2026/day-52/task-3-deployment.png differ diff --git a/2026/day-53/day-53-services.md b/2026/day-53/day-53-services.md new file mode 100644 index 0000000000..9390c9f1eb --- /dev/null +++ b/2026/day-53/day-53-services.md @@ -0,0 +1,121 @@ +# Day 53: Kubernetes Services + +## Why Services? +Every Pod gets its own IP address. But there are two problems: +1. Pod IPs are not stable β€” when a Pod restarts or gets replaced, it gets a new IP. +2. A Deployment runs multiple Pods β€” connecting to a specific IP is unreliable for clients. + +A Service solves both problems by providing a stable IP and DNS name that never changes, along with load balancing across all Pods that match its selector. + +--- + +## Challenge Tasks & Verification + +### Task 1: Deploy the Application +Created a Deployment manifest `app-deployment.yaml` with 3 replicas of Nginx. +**Verify:** Are all 3 pods running? Note down their IP addresses. +> **Answer:** Yes, all 3 pods are running successfully. Based on the `kubectl get pods -o wide` output, they were assigned the following ephemeral IPs: +> * `web-app-56b5ddf4c5-gppdj`: `10.244.2.2` +> * `web-app-56b5ddf4c5-s5rlx`: `10.244.2.3` +> * `web-app-56b5ddf4c5-t59wk`: `10.244.1.3` + +### Task 2: ClusterIP Service (Internal Access) +Created `clusterip-service.yml` to expose the pods internally. + +```yml +apiVersion: v1 +kind: Service +metadata: + name: web-app-clusterip +spec: + type: ClusterIP + selector: + app: web-app + ports: + - port: 80 + targetPort: 80 +``` +**Verify:** Does the Service respond? Try running the wget command multiple times. +> **Answer:** Yes, the service responds. When executing `wget -qO- http://web-app-clusterip` from inside the temporary `test-client` pod, it successfully returned the Nginx welcome HTML. The Service load-balanced the request to one of the 3 underlying pods. + +### Task 3: Discover Services with DNS +Tested Kubernetes built-in DNS using a temporary `dns-test` pod. +**Verify:** What IP does nslookup return? Does it match the CLUSTER-IP from kubectl get services? +> **Answer:** Running `nslookup web-app-clusterip` returned the IP address `10.96.248.219`. Yes, this perfectly matches the CLUSTER-IP assigned to `web-app-clusterip` in the `kubectl get services` output. + +### Task 4: NodePort Service (External Access via Node) +Created `nodeport-service.yml` to expose the application on port `30080` across all cluster nodes. + +```yml +apiVersion: v1 +kind: Service +metadata: + name: web-app-nodeport +spec: + type: NodePort + selector: + app: web-app + ports: + - port: 80 + targetPort: 80 + nodePort: 30080 +``` +**Verify:** Can you see the Nginx welcome page from your browser or terminal using the NodePort? +> **Answer:** Yes. Due to Mac/Kind Docker networking limitations, direct Node IP curl timed out. However, using the standard workaround `kubectl port-forward service/web-app-nodeport 30080:80`, I was able to successfully view the Nginx page via `localhost:30080`. + +### Task 5: LoadBalancer Service (Cloud External Access) +Created `loadbalancer-service.yml` for external cloud load balancing. + +```yml +apiVersion: v1 +kind: Service +metadata: + name: web-app-loadbalancer +spec: + type: LoadBalancer + selector: + app: web-app + ports: + - port: 80 + targetPort: 80 +``` +**Verify:** What does the EXTERNAL-IP column show? Why is it `` on a local cluster? +> **Answer:** The `EXTERNAL-IP` column shows ``. This is the expected official Kubernetes behavior for local environments (like Kind or Minikube) because there is no external cloud provider (like AWS, GCP, or Azure) attached to the cluster to physically provision a public Load Balancer. + +### Task 6: Understand the Service Types Side by Side +Used `kubectl describe service web-app-loadbalancer` to inspect the underlying architecture. +**Verify:** Does the LoadBalancer service also have a ClusterIP and NodePort assigned? +> **Answer:** Yes. The describe command confirmed that the LoadBalancer inherently builds on the other types. It had a ClusterIP (`10.96.71.154`) and automatically assigned a NodePort (`31754/TCP`). + +### Task 7: Clean Up +**Verify:** Is everything cleaned up? +> **Answer:** Yes. After running the respective `kubectl delete -f` commands, `kubectl get pods` and `kubectl get services` returned empty results (except for the default `kubernetes` service). + +--- + +## Documentation + +* **What problem Services solve and how they relate to Pods and Deployments:** + Services decouple the frontend/clients from the backend Pods. While Deployments manage the lifecycle and scaling of Pod replicas, Services provide a static, permanent network identity (IP and DNS) and load-balance incoming traffic to the healthy, dynamically changing IPs of those Pods. + +* **Your three Service manifests with an explanation of each type:** + * **ClusterIP:** Exposes the service on a cluster-internal IP. Reachable only within the cluster. + * **NodePort:** Exposes the service on each Node's IP at a static port (30000-32767). + * **LoadBalancer:** Exposes the service externally using a cloud provider's load balancer. + +* **The difference between ClusterIP, NodePort, and LoadBalancer:** + | Type | Accessible From | Use Case | + | :--- | :--- | :--- | + | **ClusterIP** | Inside the cluster only | Internal communication (e.g., Backend to DB). | + | **NodePort** | Outside via `:` | Direct node access, local testing. | + | **LoadBalancer** | Outside via Cloud LB | Production traffic in cloud environments. | + +* **How Kubernetes DNS works for service discovery:** + Kubernetes (CoreDNS) automatically creates DNS records for Services. Instead of hardcoding IPs, pods can communicate with services using the FQDN: `..svc.cluster.local`. + +* **What Endpoints are and how to inspect them:** + Endpoints are the exact, active IP addresses of the individual Pods that the Service routes traffic to. They are dynamically updated by the Service's `selector`. + *How to inspect:* `kubectl describe service `. My LoadBalancer output showed `Endpoints: 10.244.2.3:80,10.244.2.2:80,10.244.1.3:80`, proving it tracked my 3 Pods perfectly. + +* **Screenshot of your services and the test output:** + ![Task 6 - Services Side by Side](./task-6-services-side-by-side.png) \ No newline at end of file diff --git a/2026/day-53/task-6-services-side-by-side.png b/2026/day-53/task-6-services-side-by-side.png new file mode 100644 index 0000000000..b38570997f Binary files /dev/null and b/2026/day-53/task-6-services-side-by-side.png differ diff --git a/2026/day-54/day-54-configmaps-secrets.md b/2026/day-54/day-54-configmaps-secrets.md new file mode 100644 index 0000000000..a31f99aaaa --- /dev/null +++ b/2026/day-54/day-54-configmaps-secrets.md @@ -0,0 +1,77 @@ +# Day 54: Kubernetes ConfigMaps and Secrets + +## Challenge Tasks & Verification + +### Task 1: Create a ConfigMap from Literals +* **Verification Question:** Can you see all three key-value pairs? +* **Official Answer & Output:** Yes. According to the Kubernetes official documentation, ConfigMaps store data as plain text to decouple configuration from image content. When I ran `kubectl get configmap app-config -o yaml`, it explicitly showed the values in plain text without any encryption: + ```yaml + data: + APP_DEBUG: "false" + APP_ENV: production + APP_PORT: "8080" + ``` + +### Task 2: Create a ConfigMap from a File +* **Verification Question:** Does `kubectl get configmap nginx-config -o yaml` show the file contents? +* **Official Answer & Output:** Yes. Creating a ConfigMap `--from-file` maps the filename as the key and the file's contents as the value. My output clearly showed `default.conf` as the key holding my Nginx server block: + ```yaml + data: + default.conf: |- + server { + listen 80; + location /health { + return 200 'healthy\n'; + } + } + ``` +* **Task Proof (Screenshot):** + ![Task 2 - ConfigMap File](./task-2-configmap-file.png) + +### Task 3: Use ConfigMaps in a Pod +* **Verification Question:** Does the `/health` endpoint respond? +* **Official Answer & Output:** Yes. The file was successfully mounted into the Nginx container at `/etc/nginx/conf.d`. When testing internal connectivity using `kubectl exec nginx-test-pod -- curl -s http://localhost/health`, the server responded exactly as configured: + ```text + healthy + ``` +* **Task Proof (Screenshot):** + ![Task 3 - ConfigMap in Pod](./task-3-configmap-pod.png) + +### Task 4: Create a Secret +* **Verification Question:** Can you decode the password back to plaintext? +* **Official Answer & Output:** Yes. The official Kubernetes documentation explicitly warns that Secrets are merely Base64 encoded, not encrypted. By passing the encoded string through `base64 --decode` in my terminal, I easily converted `czNjdXJlUEBzc3cwcmQ=` back to its original plaintext form: `s3cureP@ssw0rd`. + +### Task 5: Use Secrets in a Pod +* **Verification Question:** Are the mounted file values plaintext or base64? +* **Official Answer & Output:** They are automatically decoded into plaintext by Kubernetes when mounted or injected. My terminal execution proved this: + * **Environment Variable Injection:** `kubectl exec secret-test-pod -- env | grep DB_USER` returned `DB_USER_ENV=admin`. + * **Volume Mount Injection:** `kubectl exec secret-test-pod -- cat /etc/db-credentials/DB_PASSWORD` returned `s3curep@ssw0rd`. +* **Task Proof (Screenshot):** + ![Task 5 - Secret in Pod](./task-5-secret-pod.png) + +### Task 6: Update a ConfigMap and Observe Propagation +* **Verification Question:** Did the volume-mounted value change without a pod restart? +* **Official Answer & Output:** Yes. As per official Kubernetes documentation: *"When a ConfigMap currently consumed in a volume is updated, projected keys are eventually updated as well."* The kubelet syncs these changes automatically (typically within 1 minute), updating the mounted files without requiring a pod restart. +* **Task Proof (Screenshot):** + ![Task 6 - ConfigMap Update Propagation](./task-6-configmap-update.png) + +### Task 7: Clean Up +All created Pods, ConfigMaps, and Secrets were successfully deleted using `kubectl delete` to keep the cluster environment clean. + +--- + +## Documentation + +* **What ConfigMaps and Secrets are and when to use each:** + * **ConfigMaps:** Used to store non-confidential data in key-value pairs (e.g., URLs, environment modes, configuration files). They decouple configuration artifacts from image content to keep containerized applications portable. + * **Secrets:** Intended to hold a small amount of sensitive data such as passwords, OAuth tokens, or SSH keys. Putting this information in a Secret is safer than putting it verbatim in a Pod definition or in a container image, as it allows tighter RBAC control and stores data in memory (tmpfs) on nodes. + +* **The difference between environment variables and volume mounts:** + * **Environment Variables (`envFrom` / `valueFrom`):** Good for simple, short string values (like port numbers). However, they are injected only at Pod startup and **do not update** dynamically if the ConfigMap/Secret changes. + * **Volume Mounts (`volumeMounts`):** Ideal for larger files (like `nginx.conf`). They are mounted as actual files inside the container and **automatically update** (propagate) when the underlying ConfigMap or Secret changes in the cluster. + +* **Why base64 is encoding, not encryption:** + Base64 is a data-encoding scheme designed to translate binary data into a text format. It uses no cryptographic keys and can be effortlessly reversed (decoded) by anyone using standard command-line tools. Kubernetes Secrets use Base64 to safely transmit data across APIs, but it provides zero cryptographic security. To truly secure Secrets, Encryption at Rest must be enabled on the etcd cluster. + +* **How ConfigMap updates propagate to volumes but not env vars:** + When a ConfigMap is mounted as a volume, the local `kubelet` component periodically checks whether the mounted ConfigMap is fresh. If a change is detected, `kubelet` automatically updates the file in the container. In contrast, environment variables are tied to the container's main process (`PID 1`) at the exact moment of creation and cannot be modified dynamically without restarting the entire Pod. \ No newline at end of file diff --git a/2026/day-54/task-2-configmap-file.png b/2026/day-54/task-2-configmap-file.png new file mode 100644 index 0000000000..c457ddc00b Binary files /dev/null and b/2026/day-54/task-2-configmap-file.png differ diff --git a/2026/day-54/task-3-configmap-pod.png b/2026/day-54/task-3-configmap-pod.png new file mode 100644 index 0000000000..317e673d5c Binary files /dev/null and b/2026/day-54/task-3-configmap-pod.png differ diff --git a/2026/day-54/task-5-secret-pod.png b/2026/day-54/task-5-secret-pod.png new file mode 100644 index 0000000000..9180b40b59 Binary files /dev/null and b/2026/day-54/task-5-secret-pod.png differ diff --git a/2026/day-54/task-6-configmap-update.png b/2026/day-54/task-6-configmap-update.png new file mode 100644 index 0000000000..31e80d8679 Binary files /dev/null and b/2026/day-54/task-6-configmap-update.png differ diff --git a/2026/day-55/day-55-persistent-volumes.md b/2026/day-55/day-55-persistent-volumes.md new file mode 100644 index 0000000000..8f9eb270be --- /dev/null +++ b/2026/day-55/day-55-persistent-volumes.md @@ -0,0 +1,91 @@ +# Day 55: Persistent Volumes (PV) and Persistent Volume Claims (PVC) + +## Challenge Tasks & Verification + +### Task 1: See the Problem β€” Data Lost on Pod Deletion +* **Verification Question:** Is the timestamp the same or different after recreation? +* **Official Answer & Output:** The timestamp is different. According to the official Kubernetes documentation, when a Pod ceases to exist, Kubernetes destroys ephemeral volumes (like `emptyDir`). The data is strictly tied to the Pod's lifecycle. When the Pod was deleted and recreated, a completely new timestamp was generated, proving data loss. + ```text + # Output from the recreated pod: + Wed Oct 7 22:51:17 UTC 2026 + ``` + +### Task 2: Create a PersistentVolume (Static Provisioning) +* **Verification Question:** What is the STATUS of the PV? +* **Official Answer & Output:** The status is `Available`. The official Kubernetes documentation states that a PV is "Available" when it is a free resource that has not yet been bound to a claim. + ```text + NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS + pv-volume 1Gi RWO Retain Available + ``` +* **Task Proof (Screenshot):** + ![Task 2 - PV Available](./task-2-pv-available.png) + +### Task 3: Create a PersistentVolumeClaim +* **Verification Question:** What does the VOLUME column in `kubectl get pvc` show? +* **Official Answer & Output:** It shows the name of the bound PV (`pv-volume`). The Kubernetes control plane watches for new PVCs and finds a matching PV (by capacity and access modes). If found, it binds them together, changing the status to `Bound`. + ```text + NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS + pvc-claim Bound pv-volume 1Gi RWO + ``` +* **Task Proof (Screenshot):** + ![Task 3 - PVC Bound](./task-3-pvc-bound.png) + +### Task 4: Use the PVC in a Pod β€” Data That Survives +* **Verification Question:** Does the file contain data from both the first and second Pod? +* **Official Answer & Output:** Yes. As per Kubernetes documentation, once a PVC is mounted as a volume in a Pod, the data written to it persists independently of the Pod's lifecycle. After deleting and recreating the Pod, both timestamps were preserved. + ```text + Pod ran at Wed Oct 7 23:27:12 UTC 2026 + Pod ran at Wed Oct 7 23:28:16 UTC 2026 + ``` +* **Task Proof (Screenshot):** + ![Task 4 - PVC Pod](./task-4-pvc-pod.png) + +### Task 5: StorageClasses and Dynamic Provisioning +* **Verification Question:** What is the default StorageClass in your cluster? +* **Official Answer & Output:** The default StorageClass is `standard`, backed by the `rancher.io/local-path` provisioner. Crucially, its `VOLUMEBINDINGMODE` is set to `WaitForFirstConsumer`, which means Kubernetes delays the binding and provisioning of a PersistentVolume until a Pod using the PersistentVolumeClaim is actually created. + ```text + NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE + standard (default) rancher.io/local-path Delete WaitForFirstConsumer + ``` + +### Task 6: Dynamic Provisioning +* **Verification Question:** How many PVs exist now? Which was manual, which was dynamic? +* **Official Answer & Output:** Two PVs exist. The one named `pv-volume` was created manually (Static Provisioning). The second one with the UUID name (`pvc-4071f4bf...`) was created dynamically by the `standard` StorageClass provisioner without manual administrative intervention. + ```text + NAME CAPACITY RECLAIM POLICY STATUS CLAIM STORAGECLASS + pv-volume 1Gi Retain Bound default/pvc-claim + pvc-4071f4bf-50db-4925-bded-65495e45948c 500Mi Delete Bound default/dynamic-pvc-claim standard + ``` +* **Task Proof (Screenshot):** + ![Task 6 - Dynamic PV](./task-6-dynamic-pv.png) + +### Task 7: Clean Up +* **Verification Question:** Which PV was auto-deleted and which was retained? Why? +* **Official Answer & Output:** The dynamically provisioned PV was auto-deleted because its Reclaim Policy was `Delete` (inherited from the StorageClass). The manual `pv-volume` was retained and moved to the `Released` state because its Reclaim Policy was explicitly set to `Retain`. The Kubernetes documentation specifies that `Retain` allows for manual reclamation of the resource, while `Delete` removes both the PV object and the associated storage asset in the external infrastructure. + +--- + +## Documentation: Deep Dive into Kubernetes Storage + +* **Why containers need persistent storage:** + By default, the file system of a container is ephemeral. When a container crashes, the kubelet restarts it, but the data is lost (the container starts with a clean state). Similarly, when a Pod is evicted or deleted, any data stored in an ephemeral volume (like `emptyDir`) is permanently wiped. Persistent storage is required to decouple data availability from the Pod's lifecycle, ensuring stateful applications (like databases) do not lose data across restarts. + +* **What PVs and PVCs are and how they relate:** + * **PersistentVolume (PV):** A cluster-scoped storage resource provisioned by an administrator or dynamically by a StorageClass. It has a lifecycle independent of any individual Pod. + * **PersistentVolumeClaim (PVC):** A namespace-scoped request for storage by a user/developer. + * **Relationship:** PVCs consume PV resources in the same way Pods consume Node resources (CPU/Memory). The Kubernetes control plane acts as a matchmaker, securely binding a PVC to a qualifying PV based on requested capacity and access modes. + +* **Static vs. Dynamic Provisioning:** + * **Static Provisioning:** A cluster administrator manually creates several PVs in advance. Developers then create PVCs to consume them. It doesn't scale well for large teams. + * **Dynamic Provisioning:** Eliminates manual intervention. When a PVC requests a specific `StorageClass`, the cluster automatically calls the cloud or storage provider's API (e.g., AWS EBS, GCP PD) to create the storage asset and its corresponding PV on the fly. + +* **Access Modes:** + Dictate how a volume can be mounted on a host. + 1. **ReadWriteOnce (RWO):** Mounted as read-write by a single node. + 2. **ReadOnlyMany (ROX):** Mounted as read-only by multiple nodes simultaneously. + 3. **ReadWriteMany (RWX):** Mounted as read-write by multiple nodes simultaneously (typically requires a network file system like NFS or EFS). + +* **Reclaim Policies:** + Tell the cluster what to do with the PV after the user deletes the PVC. + 1. **Retain:** The PV is not deleted. Its status changes to `Released`. The data remains intact, but another PVC cannot claim it until an admin manually cleans it up. + 2. **Delete:** Both the PV object in Kubernetes and the actual storage asset in the backend infrastructure (e.g., AWS EBS volume) are automatically destroyed. \ No newline at end of file diff --git a/2026/day-55/task-2-pv-available.png b/2026/day-55/task-2-pv-available.png new file mode 100644 index 0000000000..65a163df96 Binary files /dev/null and b/2026/day-55/task-2-pv-available.png differ diff --git a/2026/day-55/task-3-pvc-bound.png b/2026/day-55/task-3-pvc-bound.png new file mode 100644 index 0000000000..74ac078c85 Binary files /dev/null and b/2026/day-55/task-3-pvc-bound.png differ diff --git a/2026/day-55/task-4-pvc-pod.png b/2026/day-55/task-4-pvc-pod.png new file mode 100644 index 0000000000..cf9c9fa01d Binary files /dev/null and b/2026/day-55/task-4-pvc-pod.png differ diff --git a/2026/day-55/task-6-dynamic-pv.png b/2026/day-55/task-6-dynamic-pv.png new file mode 100644 index 0000000000..357980ed70 Binary files /dev/null and b/2026/day-55/task-6-dynamic-pv.png differ diff --git a/2026/day-56/day-56-statefulsets.md b/2026/day-56/day-56-statefulsets.md new file mode 100644 index 0000000000..0435a4a08e --- /dev/null +++ b/2026/day-56/day-56-statefulsets.md @@ -0,0 +1,99 @@ +# Day 56: Kubernetes StatefulSets + +## Challenge Tasks & Verification + +### Task 1: Understand the Problem +* **Verification Question:** Why would random pod names be a problem for a database cluster? +* **Official Answer:** According to Kubernetes documentation, databases (stateful applications) require stable network identities to maintain cluster topology (e.g., knowing exactly who the master and replica nodes are). If a pod is deleted and comes back with a completely new random name and IP, the cluster peers lose connection, breaking the database replication. Deployments are designed for stateless apps where pods are interchangeable; databases need StatefulSets. +* **Real Output Observation:** When the deployment pod `jwbnr` was deleted, it was replaced by a completely new random identity `6sgq9`: + ```text + test-deploy-7bfdbfb9b-jwbnr 1/1 Running 0 24s + # After deletion: + test-deploy-7bfdbfb9b-6sgq9 1/1 Running 0 21s + ``` + +### Task 2: Create a Headless Service +* **Verification Question:** What does the CLUSTER-IP column show? +* **Official Answer & Output:** It shows `None`. As per official documentation, a Headless Service does not allocate a single IP for load balancing. Instead, it allows DNS queries to directly return the IPs of the individual pods backing the service. + ```text + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + nginx-headless ClusterIP None 80/TCP 22s + ``` + +### Task 3: Create a StatefulSet +* **Verification Question:** What are the exact pod names and PVC names? +* **Official Answer & Output:** + * **Pod Names:** They are created sequentially with stable ordinals: `web-0`, `web-1`, `web-2`. + * **PVC Names:** They are generated using the `volumeClaimTemplates` in the format `-`. + ```text + NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS + web-data-web-0 Bound pvc-2e69bdc6-7be9-4191-9b02-940d7b5fdeeb 100Mi RWO standard + web-data-web-1 Bound pvc-4c00959b-4bd3-445d-a0a6-760eced10434 100Mi RWO standard + web-data-web-2 Bound pvc-81126fab-f58d-410a-9868-1b913ab80c59 100Mi RWO standard + ``` +* **Task Proof (Screenshot):** + ![Task 3 - StatefulSet Pods and PVCs](./task-3-statefulset-pods-pvc.png) + +### Task 4: Stable Network Identity +* **Verification Question:** Does the nslookup IP match the pod IP? +* **Official Answer & Output:** Yes. The Headless Service creates a stable DNS record for each pod in the format `...svc.cluster.local`. `nslookup` perfectly resolved `web-0` to its exact IP (`10.244.0.10`). + ```text + / # nslookup web-0.nginx-headless.default.svc.cluster.local + Address: 10.244.0.10 + + / # nslookup web-1.nginx-headless.default.svc.cluster.local + Address: 10.244.0.12 + ``` +* **Task Proof (Screenshot):** + ![Task 4 - Stable DNS](./task-4-stable-dns.png) + +### Task 5: Stable Storage β€” Data Survives Pod Deletion +* **Verification Question:** Is the data identical after pod recreation? +* **Official Answer & Output:** Yes. Kubernetes documentation states that a StatefulSet maintains a sticky identity for its Pods. When a Pod is rescheduled, it is automatically re-attached to the same exact PersistentVolume. The written data successfully survived the pod deletion. + ```text + # Output from the recreated web-0 pod: + Data from web-0 + ``` +* **Task Proof (Screenshot):** + ![Task 5 - Stable Storage](./task-5-stable-storage.png) + +### Task 6: Ordered Scaling +* **Verification Question:** After scaling down, how many PVCs exist? +* **Official Answer & Output:** 5 PVCs still existed. The official documentation confirms that when a StatefulSet is scaled down, its associated PersistentVolumeClaims are intentionally **not deleted**. This safety feature ensures that data is preserved in case of an accidental scale-down. + ```text + NAME STATUS VOLUME CAPACITY + web-data-web-0 Bound pvc-2e69bdc6-7be9-4191-9b02-940d7b5fdeeb 100Mi + web-data-web-1 Bound pvc-4c00959b-4bd3-445d-a0a6-760eced10434 100Mi + web-data-web-2 Bound pvc-81126fab-f58d-410a-9868-1b913ab80c59 100Mi + web-data-web-3 Bound pvc-19a897ca-8ce5-4ddf-b499-8b384b4ebddf 100Mi + web-data-web-4 Bound pvc-28d3d193-95cc-420e-85c2-c26d4342421a 100Mi + ``` +* **Task Proof (Screenshot):** + ![Task 6 - Scale Down PVCs](./task-6-scaledown-pvcs.png) + +### Task 7: Clean Up +* **Verification Question:** Were PVCs auto-deleted with the StatefulSet? +* **Official Answer:** No. Deleting a StatefulSet (or a Pod within it) does not delete the volumes associated with the StatefulSet. PVCs must be deleted manually by the administrator. + +--- + +## Documentation: Deep Dive into StatefulSets + +### What StatefulSets are and when to use them vs Deployments +A **StatefulSet** is the Kubernetes workload API object used to manage stateful applications. +* **Use Deployments for:** Stateless applications (like web servers or API endpoints) where pods are completely interchangeable and do not require stable local storage or stable network identities. +* **Use StatefulSets for:** Stateful applications (like MySQL, PostgreSQL, Kafka, or Elasticsearch) where pods require strict ordering, stable network identifiers, and persistent, per-pod storage. + +### Deployment vs StatefulSet Comparison Table +| Feature | Deployment | StatefulSet | +| :--- | :--- | :--- | +| **Pod names** | Random (e.g., `app-xyz-abc`) | Stable, ordered (`app-0`, `app-1`) | +| **Startup order** | All at once (Parallel) | Ordered sequentially (`pod-0`, then `pod-1`) | +| **Storage** | Shared PVC across all replicas | Each pod dynamically gets its own PVC | +| **Network identity** | No stable individual hostname | Stable DNS per pod | +| **Scaling Down** | Random termination | Terminated in reverse order (`web-2`, then `web-1`) | + +### Core Mechanisms Explained +* **Headless Services:** A Service configured with `clusterIP: None`. It disables standard Kubernetes load balancing and instead allows DNS to directly return the IP addresses of the individual StatefulSet pods. +* **Stable DNS:** K8s automatically creates a stable DNS record for every pod in a StatefulSet. The format strictly follows `...svc.cluster.local`, allowing pods to reliably discover each other. +* **volumeClaimTemplates:** A template embedded in the StatefulSet manifest. Instead of an admin manually creating a PVC for every single pod, Kubernetes uses this template to dynamically auto-generate a unique PVC and bind it to each newly created pod in the StatefulSet. \ No newline at end of file diff --git a/2026/day-56/task-4-stable-dns.png b/2026/day-56/task-4-stable-dns.png new file mode 100644 index 0000000000..b30d20965b Binary files /dev/null and b/2026/day-56/task-4-stable-dns.png differ diff --git a/2026/day-56/task-5-stable-storage.png b/2026/day-56/task-5-stable-storage.png new file mode 100644 index 0000000000..2bdb219fbf Binary files /dev/null and b/2026/day-56/task-5-stable-storage.png differ diff --git a/2026/day-56/task-6-scaledown-pvcs.png b/2026/day-56/task-6-scaledown-pvcs.png new file mode 100644 index 0000000000..40aca3543d Binary files /dev/null and b/2026/day-56/task-6-scaledown-pvcs.png differ diff --git a/2026/day-57/day-57-resources-probes.md b/2026/day-57/day-57-resources-probes.md new file mode 100644 index 0000000000..b46c8ec94b --- /dev/null +++ b/2026/day-57/day-57-resources-probes.md @@ -0,0 +1,103 @@ +# Day 57: Resource Requests, Limits, and Probes + +## Challenge Tasks + +### Task 1: Resource Requests and Limits +**Verify: What QoS class does your Pod have?** +The QoS class is Burstable because the memory and CPU requests are lower than the configured limits. + +**Terminal Output:** +```text + Limits: + cpu: 250m + memory: 256Mi + Requests: + cpu: 100m + memory: 128Mi +QoS Class: Burstable +``` +![Task 1 Proof](./task-1-qos-burstable.png) + +### Task 2: OOMKilled β€” Exceeding Memory Limits +**Verify: What exit code does an OOMKilled container have?** +The exit code is 137. This occurs because the container requested 200M of memory, exceeding its hard limit of 100Mi. The system sends a SIGKILL (9), resulting in an exit code of 137 (128 + 9). + +**Terminal Output:** +```text + State: Terminated + Reason: OOMKilled + Exit Code: 137 + Limits: + memory: 100Mi +``` +![Task 2 Proof](./task-2-oomkilled.png) + +### Task 3: Pending Pod β€” Requesting Too Much +**Verify: What event message does the scheduler produce?** +The scheduler produces a "FailedScheduling" warning, explicitly stating "Insufficient memory". + +**Terminal Output:** +```text +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Warning FailedScheduling 54s default-scheduler 0/1 nodes are available: 1 Insufficient memory. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling. +``` +![Task 3 Proof](./task-3-failed-scheduling.png) + +### Task 4: Liveness Probe +**Verify: How many times has the container restarted?** +The container restarted 4 times. Once the liveness probe repeatedly failed to locate the `/tmp/healthy` file, the kubelet killed and restarted the container to attempt recovery. + +**Terminal Output:** +```text + Warning Unhealthy 32s (x12 over 4m27s) kubelet spec.containers{liveness-container}: Liveness probe failed: cat: can'topen '/tmp/healthy': No such file or directory + Normal Killing 32s (x4 over 4m17s) kubelet spec.containers{liveness-container}: Container liveness-container failed liveness probe, will be restarted +``` + +### Task 5: Readiness Probe +**Verify: When readiness failed, was the container restarted?** +No, the container was not restarted. The restart count remained at 0 and the status remained Running. However, the READY state transitioned to 0/1, and the IP was removed from the Service endpoints to stop traffic routing. + +**Terminal Output:** +```text +NAME ENDPOINTS AGE +readiness-svc 2m34s + +NAME READY STATUS RESTARTS AGE +readiness-pod 0/1 Running 0 3m38s +``` +![Task 5 Proof](./task-5-readiness-fail.png) + +### Task 6: Startup Probe +**Verify: What would happen if failureThreshold were 2 instead of 12?** +If the threshold were 2, the probe budget would only be 10 seconds (2 attempts * 5 seconds period). Because the application requires 20 seconds to start, the startup probe would fail prematurely, and the kubelet would kill the container before initialization completes, causing a CrashLoopBackOff. + +**Terminal Output:** +```text +NAME READY STATUS RESTARTS AGE +startup-pod 0/1 Running 0 21s +startup-pod 0/1 Running 0 25s +startup-pod 1/1 Running 0 25s +``` +![Task 6 Proof](./task-6-startup-success.png) + +### Task 7: Clean Up +All test pods (resource-pod, oom-pod, greedy-pod, liveness-pod, readiness-pod, startup-pod) and the readiness service were successfully deleted from the cluster. + +--- + +## Documentation + +**Requests vs limits (scheduling vs enforcement)** +* **Requests:** Act as a guaranteed minimum for scheduling. The kube-scheduler uses this value to determine if a node has enough unallocated resources to host the pod. +* **Limits:** Act as a hard cap enforced at runtime. The kubelet uses this value to ensure the container never consumes more resources than permitted on the node. + +**What happens when CPU or memory limits are exceeded** +* **CPU Limit Exceeded:** CPU is a compressible resource. The container is throttled (slowed down) but is not killed. +* **Memory Limit Exceeded:** Memory is an incompressible resource. The Linux kernel invokes the OOM (Out of Memory) killer and terminates the container immediately, resulting in an OOMKilled status and an exit code of 137. + +**Liveness vs readiness vs startup probes** +* **Liveness Probe:** Checks if the application is running and healthy. If it fails, the container is restarted. Used to break out of deadlocks. +* **Readiness Probe:** Checks if the application is ready to process user traffic. If it fails, the pod's IP is removed from the Service endpoints. The container is not restarted. +* **Startup Probe:** Protects slow-starting applications. It disables liveness and readiness probes until the application successfully starts, granting it a dedicated time budget for initialization. \ No newline at end of file diff --git a/2026/day-57/task-1-qos-burstable.png b/2026/day-57/task-1-qos-burstable.png new file mode 100644 index 0000000000..05d1e13ff7 Binary files /dev/null and b/2026/day-57/task-1-qos-burstable.png differ diff --git a/2026/day-57/task-2-oomkilled.png b/2026/day-57/task-2-oomkilled.png new file mode 100644 index 0000000000..9000631439 Binary files /dev/null and b/2026/day-57/task-2-oomkilled.png differ diff --git a/2026/day-57/task-3-failed-scheduling.png b/2026/day-57/task-3-failed-scheduling.png new file mode 100644 index 0000000000..588bfb5893 Binary files /dev/null and b/2026/day-57/task-3-failed-scheduling.png differ diff --git a/2026/day-57/task-5-readiness-fail.png b/2026/day-57/task-5-readiness-fail.png new file mode 100644 index 0000000000..e1f85564ad Binary files /dev/null and b/2026/day-57/task-5-readiness-fail.png differ diff --git a/2026/day-57/task-6-startup-success.png b/2026/day-57/task-6-startup-success.png new file mode 100644 index 0000000000..7fea83400f Binary files /dev/null and b/2026/day-57/task-6-startup-success.png differ diff --git a/2026/day-58/day-58-metrics-hpa.md b/2026/day-58/day-58-metrics-hpa.md new file mode 100644 index 0000000000..1de0441a44 --- /dev/null +++ b/2026/day-58/day-58-metrics-hpa.md @@ -0,0 +1,106 @@ +# Day 58: Metrics Server and Horizontal Pod Autoscaler (HPA) + +## Challenge Tasks + +### Task 1: Install the Metrics Server +**Verify: What is the current CPU and memory usage of your node?** +The current usage of the `devops-cluster-control-plane` node is 164m (2%) CPU and 771Mi (19%) Memory. + +**Terminal Output:** +```text +NAME CPU(cores) CPU(%) MEMORY(bytes) MEMORY(%) +devops-cluster-control-plane 164m 2% 771Mi 19% +``` + +### Task 2: Explore kubectl top +**Verify: Which pod is using the most CPU right now?** +The `kube-apiserver-devops-cluster-control-plane` pod in the kube-system namespace is using the most CPU at 37m. + +**Terminal Output:** +```text +NAMESPACE NAME CPU(cores) MEMORY(bytes) +kube-system kube-apiserver-devops-cluster-control-plane 37m 238Mi +kube-system etcd-devops-cluster-control-plane 20m 60Mi +kube-system kube-controller-manager-devops-cluster-control-plane 14m 57Mi +``` + +### Task 3: Create a Deployment with CPU Requests +**Verify: What is the current CPU usage of the Pod?** +The current CPU usage of the `php-apache` pod is 1m. + +**Terminal Output:** +```text +NAME CPU(cores) MEMORY(bytes) +php-apache-59d7b9cd64-lpqg4 1m 28Mi +``` +![Task 3 Proof](./task-3-php-apache-top.png) + +### Task 4: Create an HPA (Imperative) +**Verify: What does the TARGETS column show?** +The TARGETS column shows `cpu: 0%/50%`, indicating that the current utilization is 0% against a target threshold of 50%. + +**Terminal Output:** +```text +NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE +php-apache Deployment/php-apache cpu: 0%/50% 1 10 1 71s +``` + +### Task 5: Generate Load and Watch Autoscaling +**Verify: How many replicas did HPA scale to under load?** +The HPA successfully scaled the deployment to the maximum limit of 10 replicas as the CPU load spiked to 492%. + +**Terminal Output:** +```text +NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE +php-apache Deployment/php-apache cpu: 0%/50% 1 10 1 8m48s +php-apache Deployment/php-apache cpu: 395%/50% 1 10 1 9m1s +php-apache Deployment/php-apache cpu: 492%/50% 1 10 4 9m16s +php-apache Deployment/php-apache cpu: 128%/50% 1 10 8 9m31s +php-apache Deployment/php-apache cpu: 59%/50% 1 10 10 9m46s +php-apache Deployment/php-apache cpu: 49%/50% 1 10 10 10m +``` +![Task 5 Proof](./task-5-hpa-scaling.png) + +### Task 6: Create an HPA from YAML (Declarative) +**Verify: What does the behavior section control?** +The behavior section controls the rate of scaling (scale-up and scale-down speed). `scaleUp.stabilizationWindowSeconds: 0` ensures immediate scale-up under load, while `scaleDown.stabilizationWindowSeconds: 300` enforces a 5-minute delay before scaling down to prevent thrashing. + +**Terminal Output:** +```text +Metrics: ( current / target ) + resource cpu on pods (as a percentage of request): 0% (1m) / 50% +Min replicas: 1 +Max replicas: 10 +Behavior: + Scale Up: + Stabilization Window: 0 seconds + Select Policy: Max + Policies: + - Type: Pods Value: 4 Period: 15 seconds + - Type: Percent Value: 100 Period: 15 seconds + Scale Down: + Stabilization Window: 300 seconds + Select Policy: Max + Policies: + - Type: Percent Value: 100 Period: 15 seconds +``` +![Task 6 Proof](./task-6-hpa-v2-behavior.png) + +### Task 7: Clean Up +The HPA, Service, Deployment, and load-generator pod were successfully deleted. The Metrics Server was left running for future resource monitoring. + +--- + +## Documentation + +**What the Metrics Server is and why HPA needs it** +The Metrics Server is a cluster-wide aggregator of resource usage data. It continuously collects CPU and memory metrics from the kubelets on each node. The Horizontal Pod Autoscaler (HPA) is entirely dependent on the Metrics Server because it requires real-time utilization metrics to determine if a pod has crossed its scaling threshold. + +**How HPA calculates desired replicas** +HPA calculates the desired number of replicas using this standard formula: +`desiredReplicas = ceil(currentReplicas * (currentUsage / targetUsage))` +It compares the actual metric (currentUsage) against the configured threshold (targetUsage, derived from the container's resource requests) and scales proportionally. + +**The difference between autoscaling/v1 and v2** +* **autoscaling/v1:** An older API version that only supports CPU utilization as a metric for autoscaling. +* **autoscaling/v2:** The modern API version that supports CPU, memory, and custom/external metrics. Crucially, it introduces the `behavior` block, allowing fine-grained control over scale-up and scale-down speeds (stabilization windows). \ No newline at end of file diff --git a/2026/day-58/task-3-php-apache-top.png b/2026/day-58/task-3-php-apache-top.png new file mode 100644 index 0000000000..5a20cccca5 Binary files /dev/null and b/2026/day-58/task-3-php-apache-top.png differ diff --git a/2026/day-58/task-5-hpa-scaling.png b/2026/day-58/task-5-hpa-scaling.png new file mode 100644 index 0000000000..34c12a377e Binary files /dev/null and b/2026/day-58/task-5-hpa-scaling.png differ diff --git a/2026/day-58/task-6-hpa-v2-behavior.png b/2026/day-58/task-6-hpa-v2-behavior.png new file mode 100644 index 0000000000..8f59b5de71 Binary files /dev/null and b/2026/day-58/task-6-hpa-v2-behavior.png differ diff --git a/2026/day-59/day-59-helm.md b/2026/day-59/day-59-helm.md new file mode 100644 index 0000000000..5ae264661b --- /dev/null +++ b/2026/day-59/day-59-helm.md @@ -0,0 +1,123 @@ +# Day 59: Helm β€” Kubernetes Package Manager + +## Challenge Tasks + +### Task 1: Install Helm +**Verify: What version of Helm is installed?** +The installed version of Helm is `v4.3.0`. + +**Terminal Output:** +```text +version.BuildInfo{Version:"v4.3.0", GitCommit:"bec5b06ed841fe5269972d864d5177944fd5970f", GitTreeState:"clean", GoVersion:"go1.27.1", KubeClientVersion:"v1.37"} +``` +![Task 1 Proof](./task-1-helm-version.png) + +### Task 2: Add a Repository and Search +**Verify: How many charts does Bitnami have?** +The Bitnami repository hosts an extensive list of production-ready charts (over 100+ applications), including popular tools like Airflow, Nginx, Redis, Kafka, and PostgreSQL. + +**Terminal Output:** +```text +NAME CHART VERSION APP VERSION DESCRIPTION +bitnami/nginx 25.2.1 1.31.6 NGINX Open Source is a web server that can be a... +bitnami/nginx-ingress-controller 12.0.7 1.13.1 NGINX Ingress Controller is an Ingress controll... +``` +![Task 2 Proof](./task-2-helm-search.png) + +### Task 3: Install a Chart +**Verify: How many Pods are running? What Service type was created?** +After installing the default Bitnami Nginx chart, **1 Pod** is running, and a Service of type **LoadBalancer** was automatically created. + +**Terminal Output:** +```text +NAME READY STATUS RESTARTS AGE +pod/my-nginx-5ff6687869-k7687 1/1 Running 0 69s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/my-nginx LoadBalancer 10.96.127.251 80:31985/TCP,443:31229/TCP 69s +``` +![Task 3 Proof](./task-3-helm-install.png) + +### Task 4: Customize with Values +**Verify: Does the values file release have the correct replicas and service type?** +Yes. By providing a `custom-values.yaml` file, we successfully overrode the defaults. The release was deployed with **3 Pods** and the Service type was changed to **NodePort**. + +**Terminal Output:** +```text +USER-SUPPLIED VALUES: +replicaCount: 3 +service: + type: NodePort + +NAME READY STATUS RESTARTS AGE +pod/custom-nginx-6d6d5988b-2zg7l 1/1 Running 0 83s +pod/custom-nginx-6d6d5988b-82dfg 1/1 Running 0 83s +pod/custom-nginx-6d6d5988b-8j7wr 1/1 Running 0 83s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/custom-nginx NodePort 10.96.178.145 80:32470/TCP,443:32195/TCP 83s +``` +![Task 4 Proof](./task-4-helm-values.png) + +### Task 5: Upgrade and Rollback +**Verify: How many revisions after the rollback?** +There are **3 Revisions** in the history. Helm does not delete or overwrite previous revisions; instead, a rollback creates a brand new revision (Revision 3) mapping back to the state of Revision 1, preserving full auditability. + +**Terminal Output:** +```text +REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION +1 Wed Oct 14 04:38:10 2026 superseded nginx-25.2.1 1.31.6 Install complete +2 Wed Oct 14 04:48:32 2026 superseded nginx-25.2.1 1.31.6 Upgrade complete +3 Wed Oct 14 04:51:04 2026 deployed nginx-25.2.1 1.31.6 Rollback to 1 +``` +![Task 5 Proof](./task-5-helm-history.png) + +### Task 6: Create Your Own Chart +**Verify: After installing, 3 replicas? After upgrading, 5?** +Yes. Initially, the custom-built chart was deployed with 3 replicas (as modified in the `values.yaml`). Upon running `helm upgrade --set replicaCount=5`, Helm instantly spun up 2 additional replicas, resulting in 5 running pods. + +**Terminal Output:** +```text +NAME READY STATUS RESTARTS AGE +my-release-my-app-66b89dff67-5c5hs 1/1 Running 0 13s +my-release-my-app-66b89dff67-f2hfs 1/1 Running 0 3m20s +my-release-my-app-66b89dff67-mk7v6 1/1 Running 0 3m20s +my-release-my-app-66b89dff67-svtss 1/1 Running 0 3m20s +my-release-my-app-66b89dff67-xfbk2 1/1 Running 0 13s +``` +![Task 6 Proof](./task-6-custom-chart-upgrade.png) + +### Task 7: Clean Up +All test releases (`custom-nginx` and `my-release`), custom files, and the scaffolded chart directory were successfully uninstalled and removed from the cluster. + +--- + +## Documentation + +**What Helm is and the three core concepts** +Helm is the package manager for Kubernetes (analogous to `apt` for Ubuntu or `npm` for Node.js). It simplifies deploying complex applications by bundling multiple Kubernetes resource definitions into a single, reusable package. +1. **Chart:** A bundle of Kubernetes manifest templates and default configurations. +2. **Release:** A running instance of a Chart deployed in a Kubernetes cluster. +3. **Repository:** A hosted location where Charts can be stored, searched, and shared (e.g., Bitnami). + +**How to install, customize, upgrade, and rollback** +* **Install:** `helm install ` deploys the application with default values. +* **Customize:** Use `-f values.yaml` to pass a custom configuration file or `--set key=value` to override specific defaults during installation. +* **Upgrade:** `helm upgrade ` applies new configurations or updates the application version seamlessly. +* **Rollback:** `helm rollback ` reverts the application to a previous stable state, generating a new revision entry in the history. + +**The structure of a Helm chart and how Go templating works** +Running `helm create` generates a standard structure: +* `Chart.yaml`: Contains metadata about the chart (name, version, description). +* `values.yaml`: Contains the default configuration values. +* `templates/`: Contains Kubernetes manifest templates (e.g., `deployment.yaml`, `service.yaml`). +Helm uses **Go template syntax** (e.g., `{{ .Values.replicaCount }}`) inside the `templates/` folder. During deployment, the Helm engine reads the `values.yaml` file and injects those values into the Go templates to generate standard Kubernetes YAML manifests, which are then applied to the cluster. + +**My `custom-values.yaml` with explanations** +```yaml +replicaCount: 3 +service: + type: NodePort +``` +* `replicaCount: 3`: Overrides the default of 1 replica, instructing Kubernetes to maintain 3 running pods for high availability. +* `service.type: NodePort`: Changes the default Service type (LoadBalancer) to NodePort, allowing external access to the application via a specific port on the Kubernetes Node IPs. \ No newline at end of file diff --git a/2026/day-59/task-1-helm-version.png b/2026/day-59/task-1-helm-version.png new file mode 100644 index 0000000000..c6afa97887 Binary files /dev/null and b/2026/day-59/task-1-helm-version.png differ diff --git a/2026/day-59/task-2-helm-search.png b/2026/day-59/task-2-helm-search.png new file mode 100644 index 0000000000..c0174bd76b Binary files /dev/null and b/2026/day-59/task-2-helm-search.png differ diff --git a/2026/day-59/task-3-helm-install.png b/2026/day-59/task-3-helm-install.png new file mode 100644 index 0000000000..3616a3ca4f Binary files /dev/null and b/2026/day-59/task-3-helm-install.png differ diff --git a/2026/day-59/task-4-helm-values.png b/2026/day-59/task-4-helm-values.png new file mode 100644 index 0000000000..3ea6696f92 Binary files /dev/null and b/2026/day-59/task-4-helm-values.png differ diff --git a/2026/day-59/task-5-helm-history.png b/2026/day-59/task-5-helm-history.png new file mode 100644 index 0000000000..55e4aee904 Binary files /dev/null and b/2026/day-59/task-5-helm-history.png differ diff --git a/2026/day-59/task-6-custom-chart-upgrade.png b/2026/day-59/task-6-custom-chart-upgrade.png new file mode 100644 index 0000000000..46a81220ec Binary files /dev/null and b/2026/day-59/task-6-custom-chart-upgrade.png differ diff --git a/2026/day-60/day-60-capstone.md b/2026/day-60/day-60-capstone.md new file mode 100644 index 0000000000..94e88258a7 --- /dev/null +++ b/2026/day-60/day-60-capstone.md @@ -0,0 +1,90 @@ +# Day 60 - Capstone: Deploy WordPress + MySQL on Kubernetes + +## Task Overview +Ten days of Kubernetes β€” clusters, Pods, Deployments, Services, ConfigMaps, Secrets, storage, StatefulSets, resource management, autoscaling, and Helm. Today we put it all together. Deploy a real WordPress + MySQL application using every major concept learned. + +## Expected Output +- A complete WordPress + MySQL stack in a capstone namespace +- Self-healing and data persistence verified +- A markdown file: day-60-capstone.md +- Screenshot of the running WordPress site and kubectl get all -n capstone + +--- + +## Challenge Tasks + +### Task 1: Create the Namespace (Day 52) +Created an isolated namespace for the project and set it as the default context to streamline subsequent commands. +```bash +kubectl create namespace capstone +kubectl config set-context --current --namespace=capstone +``` +**Verify:** The context was successfully modified and `capstone` is active. + +### Task 2: Deploy MySQL (Days 54-56) +Deployed the database backend using a StatefulSet for network identity, a Headless Service for internal DNS resolution, and a PersistentVolumeClaim for storage. Passwords were securely stored in a Kubernetes Secret using `stringData`. +**Verify:** Accessed the `mysql-0` pod via `kubectl exec` and confirmed the existence of the `wordpress` database. +![Task 2 - Deploy MySQL](./task-2-mysql-verify.png) + +### Task 3: Deploy WordPress (Days 52, 54, 57) +Deployed the WordPress frontend using a Deployment with 2 replicas. Database connection details were injected via a ConfigMap, and credentials via `secretKeyRef`. Configured resource limits and `tcpSocket` Liveness/Readiness probes to monitor port 80 with an initial delay. +**Verify:** Both WordPress pods successfully reached the `1/1 Running` state. +![Task 3 - Deploy WordPress](./task-3-wordpress-verify.png) + +### Task 4: Expose WordPress (Day 53) +Created a NodePort Service on port 30080 to route external traffic to the WordPress pods. Used port-forwarding to access the application locally. +**Verify:** Successfully accessed the WordPress setup wizard via `localhost:8080`, completed the installation, and published a test blog post. +![Task 4 - Expose WordPress](./task-4-wordpress-browser.png) + +### Task 5: Test Self-Healing and Persistence +Executed intentional deletion of all WordPress pods and the MySQL pod to observe cluster behavior. +**Verify:** The ReplicaSet and StatefulSet controllers immediately recreated the terminated pods. Upon refreshing the browser, the blog post was still present, confirming successful data persistence via the PVC. +![Task 5 - Test Self-Healing](./task-5-persistence.png) + +### Task 6: Set Up HPA (Day 58) +Configured a HorizontalPodAutoscaler targeting the WordPress Deployment to maintain 50% CPU utilization, with a minimum of 2 and maximum of 10 replicas. +**Verify:** The HPA successfully registered the targets (`0%/50%`) and the `kubectl get all -n capstone` output displayed the complete, running architecture. +![Task 6 - Set Up HPA](./task-6-get-all.png) + +### Task 7: (Bonus) Compare with Helm (Day 59) +Installed the Bitnami WordPress chart in a separate `helm-test` namespace to observe automated resource provisioning. +**Verify:** Helm provisioned an entire stack (including MariaDB, Deployments, Services, and Secrets) automatically. While Helm provides convenience and speed, the manual manifest approach provides granular control over individual K8s components. +![Task 7 - Helm Comparison](./task-7-helm-compare.png) + +### Task 8: Clean Up and Reflect +Deleted the isolated namespace, which automatically cascaded the deletion to all associated resources (Pods, Services, PVCs, Deployments). +**Verify:** Executing `kubectl get all -n capstone` returned "No resources found", confirming complete environment cleanup. + +--- + +## Documentation + +### Architecture of your deployment +The deployment architecture is isolated within the `capstone` namespace and consists of two primary tiers: +1. **Database Tier:** Powered by MySQL running as a `StatefulSet` (1 replica) to maintain stable network identity. Data persistence is handled by a `PersistentVolumeClaim` (1Gi, ReadWriteOnce) mounted at `/var/lib/mysql`. Internal network routing is managed by a Headless Service (`clusterIP: None`). Authentication credentials are provided via a `Secret`. +2. **Application Tier:** Powered by WordPress running as a `Deployment` (2 replicas for High Availability). It receives database host configurations via a `ConfigMap` and credentials via specific `secretKeyRef` injections. External traffic is routed through a `NodePort` Service (Port 30080). Pod lifecycle is monitored by `tcpSocket` probes, and automatic scaling is governed by a `HorizontalPodAutoscaler` targeting 50% CPU utilization. + +### Results of self-healing and persistence tests +Terminating the pods resulted in immediate intervention by the Kubernetes control plane. The `ReplicaSet` recreated the stateless WordPress pods, while the `StatefulSet` recreated the MySQL pod maintaining the exact `mysql-0` identifier. The `PersistentVolumeClaim` successfully retained the underlying storage block. Consequently, upon application recovery, the WordPress database and all associated posts remained completely intact with zero data loss. + +### Concept Mapping Table + +| Kubernetes Concept | Day Learned | Application in Capstone Project | +| :--- | :---: | :--- | +| **Namespaces** | Day 52 | Provided logical isolation and resource grouping (`capstone`). | +| **Deployments** | Day 52 | Managed the stateless frontend WordPress replicas. | +| **Services (NodePort)** | Day 53 | Exposed the WordPress application externally on port 30080. | +| **ConfigMaps** | Day 54 | Decoupled configuration data (e.g., `WORDPRESS_DB_HOST`). | +| **Secrets** | Day 54 | Encrypted and injected database passwords securely. | +| **PVCs** | Day 55 | Provisioned persistent storage independent of pod lifecycle. | +| **StatefulSets** | Day 56 | Managed stateful MySQL pod identity and volume attachment. | +| **Headless Service** | Day 56 | Enabled direct DNS resolution for the StatefulSet pods. | +| **Resource Limits** | Day 57 | Constrained CPU/Memory allocation to prevent node exhaustion. | +| **Probes** | Day 57 | Monitored container health to dictate traffic routing readiness. | +| **HPA** | Day 58 | Automated scaling based on CPU utilization metrics. | +| **Helm** | Day 59 | Demonstrated automated, packaged application deployment. | + +### Reflection +* **What was hardest:** Configuring the K8s Probes. During initial boot, WordPress returns an HTTP 302 Redirect to its installation script. Kubernetes HTTP GET probes classify HTTP 302 responses as failures, which triggers a continuous `CrashLoopBackOff` state. Resolving this required migrating from HTTP probes to `tcpSocket` probes on port 80. +* **What clicked:** Kubernetes Service Discovery and internal DNS mechanisms. The frontend deployment successfully resolved the database backend using only the Fully Qualified Domain Name (`mysql-0.mysql.capstone.svc.cluster.local`), entirely abstracting the underlying, ephemeral pod IP addresses. +* **What I would add for production:** For a production-grade environment, the NodePort service should be replaced with an Ingress Controller (e.g., NGINX) for robust Layer 7 HTTP/HTTPS routing. Additionally, I would implement `cert-manager` for automated TLS certificate provisioning and integrate a volume backup solution like Velero for disaster recovery. \ No newline at end of file diff --git a/2026/day-60/task-2-mysql-verify.png b/2026/day-60/task-2-mysql-verify.png new file mode 100644 index 0000000000..e4dfda4232 Binary files /dev/null and b/2026/day-60/task-2-mysql-verify.png differ diff --git a/2026/day-60/task-3-wordpress-verify.png b/2026/day-60/task-3-wordpress-verify.png new file mode 100644 index 0000000000..9ff62afe17 Binary files /dev/null and b/2026/day-60/task-3-wordpress-verify.png differ diff --git a/2026/day-60/task-4-wordpress-browser.png b/2026/day-60/task-4-wordpress-browser.png new file mode 100644 index 0000000000..e14d116f68 Binary files /dev/null and b/2026/day-60/task-4-wordpress-browser.png differ diff --git a/2026/day-60/task-5-persistence.png b/2026/day-60/task-5-persistence.png new file mode 100644 index 0000000000..1aa1de50bc Binary files /dev/null and b/2026/day-60/task-5-persistence.png differ diff --git a/2026/day-60/task-6-get-all.png b/2026/day-60/task-6-get-all.png new file mode 100644 index 0000000000..a77b6fc6f8 Binary files /dev/null and b/2026/day-60/task-6-get-all.png differ diff --git a/2026/day-60/task-7-helm-compare.png b/2026/day-60/task-7-helm-compare.png new file mode 100644 index 0000000000..44d713508a Binary files /dev/null and b/2026/day-60/task-7-helm-compare.png differ diff --git a/2026/day-61/day-61-terraform-intro.md b/2026/day-61/day-61-terraform-intro.md new file mode 100644 index 0000000000..899faa8c63 --- /dev/null +++ b/2026/day-61/day-61-terraform-intro.md @@ -0,0 +1,84 @@ +# Day 61: Introduction to Terraform and Your First AWS Infrastructure + +## Challenge Tasks + +### Task 1: Understand Infrastructure as Code +**1. What is Infrastructure as Code (IaC)? Why does it matter in DevOps?** +Infrastructure as Code (IaC) is the process of managing and provisioning computing infrastructure through machine-readable definition files, rather than interactive configuration tools or physical hardware setup. In DevOps, it matters because it enables automation, infrastructure version control, consistency across environments, and faster deployment cycles. + +**2. What problems does IaC solve compared to manually creating resources in the AWS console?** +Manual creation (ClickOps) leads to human errors, configuration drift (inconsistencies between staging and production), and lack of auditability. IaC solves this by providing a single source of truth, enabling automated testing, and allowing teams to reliably recreate environments in minutes. + +**3. How is Terraform different from AWS CloudFormation, Ansible, and Pulumi?** +- **CloudFormation:** Strictly tied to AWS, whereas Terraform is cloud-agnostic. +- **Ansible:** Primarily a configuration management tool (configuring OS/software inside a server), whereas Terraform is an infrastructure provisioning tool (creating the server itself). +- **Pulumi:** Uses general-purpose programming languages (Python, Go), whereas Terraform uses its own declarative HashiCorp Configuration Language (HCL). + +**4. What does it mean that Terraform is "declarative" and "cloud-agnostic"?** +- **Declarative:** You define the desired end-state of your infrastructure, and Terraform determines the logic and steps to achieve it. +- **Cloud-Agnostic:** Using a unified workflow, Terraform can manage resources across multiple cloud providers (AWS, Azure, GCP) via different provider plugins. + +### Task 2: Install Terraform and Configure AWS +- **Terraform Installation:** Installed latest Terraform via Homebrew on macOS (`v1.15.x`). +- **AWS CLI:** Installed AWS CLI v2 via package installer and configured IAM User credentials (`terraform-admin`) for the `ap-south-1` region. +- Verified connection using `aws sts get-caller-identity`. + +### Task 3: Your First Terraform Config -- Create an S3 Bucket +Created `main.tf` and defined the AWS provider and an `aws_s3_bucket` resource. +- **Bucket Name:** `terraweek-rashid-khan-2026-unique-bucket` +- **Commands Executed:** `terraform init`, `terraform plan`, `terraform apply` + +**Q: What did terraform init download? What does the .terraform/ directory contain?** +**Answer:** The `terraform init` command downloaded the AWS provider plugin (specifically `hashicorp/aws v5.100.0`) required to interact with AWS APIs. The `.terraform/` directory contains these downloaded provider binaries and modules, acting as a local cache for the project. + +### Task 4: Add an EC2 Instance +Added an `aws_instance` block to `main.tf`. +- **AMI:** `ami-0f5ee92e2d63afc18` (Amazon Linux 2) +- **Instance Type:** `t2.micro` +- **Tag:** `Name = "TerraWeek-Day1"` + +**Q: How does Terraform know the S3 bucket already exists and only the EC2 instance needs to be created?** +**Answer:** Terraform tracks the real-world infrastructure in its state file (`terraform.tfstate`). During `terraform plan`, it compared the local code against this state file and the actual AWS environment. It recognized the S3 bucket was already present in the state and unmodified in the code, so it only planned the creation of the newly added EC2 instance. + +### Task 5: Understand the State File +Inspected the state file using CLI commands: `terraform show`, `terraform state list`, and `terraform state show`. + +**1. What information does the state file store about each resource?** +It stores a complete map of the resources deployed, including metadata, unique cloud identifiers (like the EC2 Instance ID `i-0f47a1c3aa7951a19`), private/public IP addresses, and resource dependencies. + +**2. Why should you never manually edit the state file?** +The state file is a strictly formatted JSON file. Manually editing it can corrupt the JSON structure or break the synchronization between Terraform and AWS, causing Terraform to lose track of resources, create duplicates, or fail execution. + +**3. Why should the state file not be committed to Git?** +The state file stores infrastructure data in plain text, which can include highly sensitive information like database passwords, API tokens, and private keys. Committing this to a Git repository exposes your cloud environment to severe security risks. + +### Task 6: Modify, Plan, and Destroy +Modified the EC2 instance tag from `TerraWeek-Day1` to `TerraWeek-Modified`. + +**Q: What do the ~, +, and - symbols mean in terraform plan?** +- `+` means Create (resource will be added). +- `-` means Destroy (resource will be removed). +- `~` means Update in-place (resource properties will be modified without recreation). +- `-/+` means Replace (resource will be destroyed and recreated). + +**Q: Is this an in-place update or a destroy-and-recreate?** +**Answer:** Because only a tag was changed, it was an **in-place update** (`~`). + +Finally, executed `terraform destroy` to tear down all resources safely. + +--- + +## Documentation Summary + +- **IaC Explanation:** Infrastructure as Code (IaC) replaces manual configuration with machine-readable definition files. It allows developers to version-control infrastructure, automate deployments, and eliminate configuration drift, providing a consistent, auditable, and scalable cloud environment. +- **Execution Screenshots:** + - [Insert Screenshot: `terraform apply` creating S3 and EC2] + - [Insert Screenshot: S3 and EC2 visible in AWS Console] +- **Command Reference:** + - `terraform init`: Initializes the directory and downloads providers. + - `terraform plan`: Shows the execution plan of changes to be made. + - `terraform apply`: Applies the changes to reach the desired state. + - `terraform destroy`: Deletes all managed infrastructure. + - `terraform show`: Displays a human-readable view of the current state. + - `terraform state list`: Lists all resources currently tracked by Terraform. +- **State File Importance:** The state file maps the configuration to the real world. It optimizes performance by preventing unnecessary API calls and tracks dependencies, but must be secured remotely (not in Git) to protect sensitive data. diff --git a/2026/day-62/day-62-providers-resources.md b/2026/day-62/day-62-providers-resources.md new file mode 100644 index 0000000000..f2724a9d89 --- /dev/null +++ b/2026/day-62/day-62-providers-resources.md @@ -0,0 +1,152 @@ +# Day 62: Terraform Providers, Resources and Dependencies + +## Challenge Tasks + +### Task 1: Explore the AWS Provider +- **Terraform Block & Provider Setup:** Created `providers.tf` specifying the `hashicorp/aws` provider. +- **Version Installed:** After running `terraform init`, version `v5.100.0` of the AWS provider was successfully installed. +- **The Lock File (`.terraform.lock.hcl`):** This file records the exact provider versions and their cryptographic hashes (`h1:` and `zh:`). It ensures that execution environments are strictly synchronized across different machines and team members. + +**Q: What does `~> 5.0` mean? How is it different from `>= 5.0` and `= 5.0.0`?** +- **`~> 5.0` (Pessimistic Constraint Operator):** Allows only minor and patch updates within the 5.x release branch (e.g., `5.100.0`), but explicitly prevents upgrading to a major release like `6.0` which could introduce breaking changes. +- **`>= 5.0`:** Allows any version greater than or equal to 5.0, including major releases like `6.0` or `7.0`, which is highly risky for production infrastructure. +- **`= 5.0.0`:** Locks the provider to one exact, strict version. This prevents the adoption of necessary security patches or non-breaking bug fixes. + +### Task 2: Build a VPC from Scratch +Created the core networking stack in `main.tf` using dynamic variables rather than hardcoded strings. + +- **VPC:** `10.0.0.0/16` (ID: `vpc-04951540add890021`) +- **Public Subnet:** `10.0.1.0/24` (ID: `subnet-089cf6673598b4152`) +- **Components attached:** Internet Gateway (`igw-0b09cf6c372be1e22`), Route Table (`rtb-0812810c428136fb7`), and Route Table Association (`rtbassoc-0d1399c62de99a108`). +- **Result:** `Apply complete! Resources: 5 added, 0 changed, 0 destroyed.` + +### Task 3: Understand Implicit Dependencies + +**Q: How does Terraform know to create the VPC before the subnet?** +Terraform evaluates configuration files and builds a local dependency graph dynamically. Because the `aws_subnet` resource explicitly references the ID of the VPC (`vpc_id = aws_vpc.main.id`), Terraform intelligently maps this reference as an "implicit dependency" and guarantees the VPC is fully provisioned before initializing the subnet creation. + +**Q: What would happen if you tried to create the subnet before the VPC existed?** +The operation would fail. The AWS API requires a valid VPC ID as a required parameter to instantiate a subnet. Terraform’s internal logic prevents this by respecting the dependency graph order. + +**Implicit Dependencies found in config:** +1. `aws_subnet.public` depends on `aws_vpc.main` +2. `aws_internet_gateway.igw` depends on `aws_vpc.main` +3. `aws_route_table.public_rt` depends on `aws_vpc.main` and `aws_internet_gateway.igw` +4. `aws_route_table_association.public_assoc` depends on `aws_subnet.public` and `aws_route_table.public_rt` + +### Task 4: Add a Security Group and EC2 Instance +Deployed a Security Group to allow SSH (22) and HTTP (80) inbound traffic, and provisioned a `t2.micro` EC2 instance attached to the public subnet. + +- **EC2 Instance ID:** `i-0bf570c580aefca8a` +- **Output Verified:** `server_public_ip = "13.201.36.6"` +- **Result:** `Apply complete! Resources: 2 added, 0 changed, 0 destroyed.` + +### Task 5: Explicit Dependencies with depends_on +Added an `aws_s3_bucket` (`terraweek-logs-vpc-04951540add890021`) and forced it to wait for the EC2 instance using the `depends_on = [aws_instance.web]` argument. + +**Q: When would you use `depends_on` in real projects? Give two examples.** +1. **Application Initialization:** An EC2 instance might run a user-data startup script that downloads configuration files directly from an S3 bucket. The bucket (and its objects) must be fully provisioned before the EC2 instance boots. +2. **Database Integration:** A stateless web server deployment should only trigger after the backend relational database (RDS) is completely initialized, available, and accepting connections. + +### Task 6: Lifecycle Rules and Destroy +Modified the EC2 instance AMI to force a resource replacement and added a lifecycle block. + +**Q: What are the three lifecycle arguments and when would you use each?** +1. **`create_before_destroy = true`:** Modifies Terraform's default (destroy-then-create) behavior. It provisions the replacement resource first, updates dependencies, and then destroys the old one. Crucial for **Zero-Downtime Deployments**. +2. **`prevent_destroy = true`:** Rejects any execution plan that attempts to destroy the resource. Used strictly for critical infrastructure (e.g., Production Databases) to prevent accidental data loss. +3. **`ignore_changes = [...]`:** Instructs Terraform to ignore differences between the state file and the actual cloud environment for specific attributes (e.g., `tags`). Used when attributes are managed by external automation or mutated manually in the console. + +**Teardown Verified:** +Successfully executed `terraform destroy`. Verified that destruction followed the reverse dependency order (S3 -> EC2 -> Route Tables -> Subnet -> VPC). +- [Insert task-06-destroy.png here] - Terminal output showing `Destroy complete!`. + +--- + +## The Complete `main.tf` + +```hcl +locals { + common_tags = { + Project = "TerraWeek" + Environment = var.environment + } +} + +resource "aws_vpc" "main" { + cidr_block = var.vpc_cidr + enable_dns_support = true + enable_dns_hostnames = true + tags = merge(local.common_tags, { Name = "TerraWeek-VPC-${var.environment}" }) +} + +resource "aws_subnet" "public" { + vpc_id = aws_vpc.main.id + cidr_block = var.public_subnet_cidr + map_public_ip_on_launch = var.enable_public_ip + tags = merge(local.common_tags, { Name = "TerraWeek-Public-Subnet" }) +} + +resource "aws_internet_gateway" "igw" { + vpc_id = aws_vpc.main.id + tags = merge(local.common_tags, { Name = "TerraWeek-IGW" }) +} + +resource "aws_route_table" "public_rt" { + vpc_id = aws_vpc.main.id + route { + cidr_block = "0.0.0.0/0" + gateway_id = aws_internet_gateway.igw.id + } + tags = merge(local.common_tags, { Name = "TerraWeek-Public-RT" }) +} + +resource "aws_route_table_association" "public_assoc" { + subnet_id = aws_subnet.public.id + route_table_id = aws_route_table.public_rt.id +} + +resource "aws_security_group" "web_sg" { + name = "TerraWeek-SG" + description = "Allow SSH and HTTP inbound traffic" + vpc_id = aws_vpc.main.id + + ingress { + from_port = 22 + to_port = 22 + protocol = "tcp" + cidr_blocks = ["0.0.0.0/0"] + } + ingress { + from_port = 80 + to_port = 80 + protocol = "tcp" + cidr_blocks = ["0.0.0.0/0"] + } + egress { + from_port = 0 + to_port = 0 + protocol = "-1" + cidr_blocks = ["0.0.0.0/0"] + } + tags = merge(local.common_tags, { Name = "TerraWeek-SG" }) +} + +resource "aws_instance" "web" { + ami = "ami-0287a05f0ef0e9d9a" + instance_type = "t2.micro" + subnet_id = aws_subnet.public.id + vpc_security_group_ids = [aws_security_group.web_sg.id] + associate_public_ip_address = true + tags = merge(local.common_tags, { Name = "TerraWeek-Server" }) + + lifecycle { + create_before_destroy = true + } +} + +resource "aws_s3_bucket" "app_logs" { + bucket = "terraweek-logs-${aws_vpc.main.id}" + depends_on = [aws_instance.web] + tags = merge(local.common_tags, { Name = "TerraWeek-Logs-Bucket" }) +} +``` diff --git a/2026/day-63/day-63-variables-outputs.md b/2026/day-63/day-63-variables-outputs.md new file mode 100644 index 0000000000..8eb321e3bf --- /dev/null +++ b/2026/day-63/day-63-variables-outputs.md @@ -0,0 +1,123 @@ +# Day 63: Variables, Outputs, Data Sources and Expressions + +## Task 1: Extract Variables + +Refactored the static infrastructure configuration into a dynamic, parameterized model by extracting all hardcoded values into a `variables.tf` file. + +**Q: What are the five variable types in Terraform?** +According to the official HashiCorp documentation, the primary variable types are: +1. **string**: A sequence of Unicode characters representing text (e.g., "ap-south-1"). +2. **number**: A numeric value, which can be an integer or fractional value. +3. **bool**: A boolean value, either `true` or `false`. +4. **list**: A sequence of values of the same type, identified by consecutive whole numbers starting with zero. +5. **map**: A collection of values where each is identified by a string label (key). + +## Task 2: Variable Files and Precedence + +Created explicit environment variable files to separate development logic from production logic. + +**`terraform.tfvars` (Default)** +```hcl +project_name = "terraweek" +environment = "dev" +instance_type = "t2.micro" +``` + +**`prod.tfvars`** +```hcl +project_name = "terraweek" +environment = "prod" +instance_type = "t3.small" +vpc_cidr = "10.1.0.0/16" +subnet_cidr = "10.1.1.0/24" +``` + +**Q: Write the variable precedence order from lowest to highest priority.** +According to official Terraform documentation, Terraform loads variables in the following order, with later sources taking precedence over earlier ones: +1. Environment variables (e.g., `TF_VAR_environment`) +2. The `terraform.tfvars` file, if present. +3. The `terraform.tfvars.json` file, if present. +4. Any `*.auto.tfvars` or `*.auto.tfvars.json` files, processed in lexical order of their filenames. +5. Any `-var` and `-var-file` options on the command line, in the order they are provided. + +**Screenshot:** +- `task-02-prod-plan.png` (Verification of prod.tfvars taking precedence). +![Task-2](./task-02-prod-plan.png) + +## Task 3: Add Outputs + +Created an `outputs.tf` file to extract operational data after deployment. + +```hcl +output "instance_public_ip" { + description = "The public IP of the EC2 instance" + value = aws_instance.web.public_ip +} +# Additional outputs truncated for brevity +``` + +**Verify: Does `terraform output instance_public_ip` return the correct IP?** +Yes. Running `terraform output instance_public_ip` returned `"43.205.115.100"`, which exactly matched the public IP assigned to the EC2 instance upon successful provisioning. + +**Screenshots:** +- `task-03-apply-outputs.png` (Terminal output showing all 6 outputs post-apply). +- `task-03-state-list.png` (Verification of state and JSON outputs). +![Task-3](./task-03-state-list.png) + +## Task 4: Use Data Sources + +Replaced hardcoded AMI IDs and Availability Zones with dynamic data blocks. + +**Q: What is the difference between a resource and a data source?** +According to HashiCorp documentation: +- **Resource**: Describes one or more infrastructure objects. Resources are the primary constructs in Terraform, used to create, update, and delete infrastructure. Terraform manages their complete lifecycle. +- **Data Source**: Allows data to be fetched or computed for use elsewhere in Terraform configuration. Data sources are read-only; they do not create or manage infrastructure, but rather query existing infrastructure or APIs to retrieve information (e.g., fetching the latest AMI ID dynamically). + +**Screenshot:** +- `task-04-data-sources.png` (Verification of EC2 replacement with dynamic AMI). +![Task-04](./task-04-data-sources.png) + +## Task 5: Use Locals for Dynamic Values + +Implemented a `locals` block to enforce consistent tagging and naming conventions across all resources. + +```hcl +locals { + name_prefix = "${var.project_name}-${var.environment}" + common_tags = { + Project = var.project_name + Environment = var.environment + ManagedBy = "Terraform" + } +} +``` +All resource tags were updated using the `merge()` function to combine `local.common_tags` with resource-specific attributes. + +## Task 6: Built-in Functions and Conditional Expressions + +Implemented a conditional expression to dynamically select the EC2 instance type based on the active environment: +```hcl +instance_type = var.environment == "prod" ? "t3.small" : "t2.micro" +``` + +**Q: Pick five functions you find most useful and explain what each does.** +1. **`merge()`**: Takes one or more maps and returns a single map containing a merged set of elements. Highly useful for standardizing resource tags. +2. **`upper()`**: Converts all lowercase letters in a given string to uppercase. Useful for standardizing naming conventions. +3. **`join()`**: Produces a string by concatenating together all elements of a given list of strings with a specified delimiter. +4. **`cidrsubnet()`**: Calculates a subnet address within a given IP network address prefix. Crucial for automated VPC network segmentation without manual IP math. +5. **`lookup()`**: Retrieves the value of a single element from a map, given its key. If the key does not exist, it returns the provided default value. + +**Screenshots:** +- `task-05-conditional.png` (Plan output showing dynamic instance type change). +- `task-05-destroy.png` (Successful reverse-order teardown of the infrastructure). +![Task-05a](./task-05-conditional.png) +![Task-05b](./task-05-destroy.png) +--- + +## Final Documentation Requirements + +**Difference between Variable, Local, Output, and Data:** +- **Input Variables (`variable`)**: Serve as parameters for a Terraform module, allowing users to customize behavior without editing the source code. +- **Local Values (`locals`)**: Assign a name to an expression or value, allowing it to be used multiple times within a module without repeating the calculation. +- **Output Values (`output`)**: Return values from a Terraform module to the CLI or to other modules, exposing data about the provisioned infrastructure. +- **Data Sources (`data`)**: Query external APIs or existing infrastructure to fetch read-only data that Terraform does not manage, returning values for use in the configuration. diff --git a/2026/day-63/task-02-prod-plan.png b/2026/day-63/task-02-prod-plan.png new file mode 100644 index 0000000000..b042ee0a7c Binary files /dev/null and b/2026/day-63/task-02-prod-plan.png differ diff --git a/2026/day-63/task-03-state-list.png b/2026/day-63/task-03-state-list.png new file mode 100644 index 0000000000..5c502348f3 Binary files /dev/null and b/2026/day-63/task-03-state-list.png differ diff --git a/2026/day-63/task-04-data-sources.png b/2026/day-63/task-04-data-sources.png new file mode 100644 index 0000000000..e702eb92ce Binary files /dev/null and b/2026/day-63/task-04-data-sources.png differ diff --git a/2026/day-63/task-05-conditional.png b/2026/day-63/task-05-conditional.png new file mode 100644 index 0000000000..95affcda66 Binary files /dev/null and b/2026/day-63/task-05-conditional.png differ diff --git a/2026/day-63/task-05-destroy.png b/2026/day-63/task-05-destroy.png new file mode 100644 index 0000000000..4b55abc197 Binary files /dev/null and b/2026/day-63/task-05-destroy.png differ diff --git a/2026/day-64/day-64-state-management.md b/2026/day-64/day-64-state-management.md new file mode 100644 index 0000000000..0129cd0b3a --- /dev/null +++ b/2026/day-64/day-64-state-management.md @@ -0,0 +1,121 @@ +# Day 64: Terraform State Management and Remote Backends + +## Task 1: Inspect Your Current State + +Applied the existing AWS networking and compute configuration and inspected the state file. + +**Answers to Task 1 Questions:** +1. **How many resources does Terraform track?** + Based on the execution of `terraform state list`, Terraform is actively tracking 11 resources (including data sources, VPC components, EC2, and S3). +2. **What attributes does the state store for an EC2 instance?** + According to the execution of `terraform state show aws_instance.web`, the state stores every computed and defined attribute of the instance. This includes attributes not explicitly written in the configuration, such as `arn`, `primary_network_interface_id`, `private_dns`, `cpu_options`, `root_block_device` configurations, and AWS-assigned IDs. +3. **What does the serial number represent?** + According to official HashiCorp documentation, the `serial` attribute is a version number for the state file. It increments on every state modification. Terraform uses this integer to ensure it does not accidentally overwrite a newer state file with an older one. + +**Screenshots:** +- ![task-01-state-list.png](./task-01-state-list.png) (List of 11 tracked resources) +- ![task-01-state-show.png](./task-01-state-show.png) (Deep dive into the EC2 instance attributes) + +## Task 2: Set Up S3 Remote Backend + +Migrated the local `terraform.tfstate` to a remote AWS S3 backend. Utilizing Terraform 1.11+ best practices, native S3 state locking (`use_lockfile = true`) was implemented, bypassing the legacy DynamoDB requirement. + +**Screenshots:** +- ![task-03-bootstrap-bucket.png](./task-03-bootstrap-bucket.png) (Creation of the secure S3 state bucket) +- ![task-04-state-migrated.png](./task-04-state-migrated.png) (Successful migration confirmation in terminal) + +## Task 3: Test State Locking + +Simulated a concurrent execution environment by running `terraform apply` in Terminal 1 (paused) and executing `terraform plan` in Terminal 2. + +**Documentation:** +- **What is the error message?** + The terminal threw the following error: + `Error acquiring the state lock` + `api error PreconditionFailed: At least one of the pre-conditions you specified did not hold` + `Operation: OperationTypeApply` +- **Why is locking critical for team environments?** + According to Terraform documentation, state locking prevents others from acquiring the lock and potentially corrupting the state. In team environments or CI/CD pipelines, if multiple processes attempt to write to the state file concurrently, it can lead to state corruption, resource conflicts, and catastrophic infrastructure data loss. + +**Screenshot:** +- ![task-03-lock-error.png](./task-03-lock-error.png) (Terminal output showing the lock acquisition failure) + +## Task 4: Import an Existing Resource + +Manually provisioned an S3 bucket via AWS CLI (`terraweek-import-rashid-2026`) and adopted it into Terraform management using the modern `import {}` block and the `-generate-config-out` parameter. + +**Documentation:** +- **What is the difference between terraform import and creating a resource from scratch?** + According to HashiCorp documentation, standard resource creation involves Terraform calling cloud provider APIs to provision net-new infrastructure. The `import` function, conversely, maps an existing, already-provisioned cloud resource to a Terraform configuration block, bringing it under Terraform's management without recreating it. + +**Screenshots:** +- ![task-04-import-plan.png](./task-04-import-plan.png) (Execution of generate-config-out) +- ![task-04-import-list.png](./task-04-import-list.png) (Verification of imported bucket in state list) + +## Task 5: State Surgery -- mv and rm + +Executed non-destructive state surgery commands. +1. Renamed the imported bucket within the state using `terraform state mv aws_s3_bucket.imported aws_s3_bucket.logs_bucket` and aligned the `.tf` files. +2. Removed the bucket from state tracking using `terraform state rm aws_s3_bucket.logs_bucket`. + +**Documentation:** +- **When would you use state mv in a real project?** + According to official documentation, `state mv` is used when renaming a resource or moving it into a child module. It updates the resource's logical address in the state file without triggering a destructive recreation of the actual cloud infrastructure. +- **When would you use state rm?** + `state rm` is used to remove a resource binding from the Terraform state. The underlying physical cloud resource is not destroyed. This is critical when transitioning the management of a resource to another tool, another Terraform workspace, or manual management. + +**Screenshot:** +- ![task-05-state-surgery.png](./task-05-state-surgery.png) (Execution of mv, sed, and rm commands) + +## Task 6: Simulate and Fix State Drift + +Induced state drift by utilizing the AWS CLI to manually modify the `Name` tag of the EC2 instance to `ManuallyChanged`. Executing `terraform plan` successfully detected the configuration divergence. Executing `terraform apply` forced the cloud environment to reconcile back to the defined Infrastructure as Code state. + +**Documentation:** +- **How do teams prevent state drift in production?** + State drift is prevented by strictly enforcing the Principle of Least Privilege. Direct "write" access to production environments via the AWS Management Console or AWS CLI is revoked for human users. All infrastructure modifications are mandated to pass through version control (Git) and are exclusively deployed via automated CI/CD pipelines (e.g., GitHub Actions, Jenkins). + +**Screenshot:** +- ![task-06-state-drift.png](./task-06-state-drift.png) (Plan output detecting the tag divergence) + +--- + +## Documentation Section + +### Local State vs Remote State Setup + +```text +[ LOCAL STATE ARCHITECTURE ] +Developer Workstation ++-------------------+ +-------------------+ +| main.tf | | AWS Cloud | +| terraform.tfstate|<----->| (Resources) | ++-------------------+ +-------------------+ +* High risk of data loss. No team collaboration. + +[ REMOTE STATE ARCHITECTURE (2026 Standard) ] +Developer 1 Developer 2 CI/CD Pipeline ++---------+ +---------+ +---------+ +| main.tf | | main.tf | | main.tf | ++----+----+ +----+----+ +----+----+ + | | | + +----------------------+----------------------+ + v ++-----------------------------------------------------------+ +| AWS S3 Backend Bucket | +| - dev/terraform.tfstate (Encrypted & Versioned) | +| - dev/.tflock (Native State Locking) | ++-----------------------------------------------------------+ + | + v + +---------------+ + | AWS Cloud | + +---------------+ +``` + +### Official Command Definitions +- **state mv**: Moves a resource to a different address in the state file. +- **state rm**: Removes an item from the Terraform state, completely decoupling it from Terraform management without destroying the real infrastructure. +- **import**: Maps an existing cloud infrastructure resource to a Terraform configuration block. +- **force-unlock**: Manually removes a lock on the state for the current configuration. This must only be used if a lock has become orphaned (stale) and you are absolutely certain no other applies are running. +- **refresh**: Reads the current settings from all managed remote objects and updates the Terraform state to match. (Note: Primarily achieved now via `terraform apply -refresh-only`). diff --git a/2026/day-64/task-01-state-list.png b/2026/day-64/task-01-state-list.png new file mode 100644 index 0000000000..227e6b687f Binary files /dev/null and b/2026/day-64/task-01-state-list.png differ diff --git a/2026/day-64/task-01-state-show.png b/2026/day-64/task-01-state-show.png new file mode 100644 index 0000000000..60542eb680 Binary files /dev/null and b/2026/day-64/task-01-state-show.png differ diff --git a/2026/day-64/task-03-bootstrap-bucket.png b/2026/day-64/task-03-bootstrap-bucket.png new file mode 100644 index 0000000000..c3b9dadd91 Binary files /dev/null and b/2026/day-64/task-03-bootstrap-bucket.png differ diff --git a/2026/day-64/task-03-lock-error.png b/2026/day-64/task-03-lock-error.png new file mode 100644 index 0000000000..0367d38fb7 Binary files /dev/null and b/2026/day-64/task-03-lock-error.png differ diff --git a/2026/day-64/task-04-import-list.png b/2026/day-64/task-04-import-list.png new file mode 100644 index 0000000000..8200a20ef3 Binary files /dev/null and b/2026/day-64/task-04-import-list.png differ diff --git a/2026/day-64/task-04-import-plan.png b/2026/day-64/task-04-import-plan.png new file mode 100644 index 0000000000..c2157691de Binary files /dev/null and b/2026/day-64/task-04-import-plan.png differ diff --git a/2026/day-64/task-04-state-migrated.png b/2026/day-64/task-04-state-migrated.png new file mode 100644 index 0000000000..b89b254a3c Binary files /dev/null and b/2026/day-64/task-04-state-migrated.png differ diff --git a/2026/day-64/task-05-state-surgery.png b/2026/day-64/task-05-state-surgery.png new file mode 100644 index 0000000000..03448cf1cd Binary files /dev/null and b/2026/day-64/task-05-state-surgery.png differ diff --git a/2026/day-64/task-06-state-drift.png b/2026/day-64/task-06-state-drift.png new file mode 100644 index 0000000000..664ecb9f6a Binary files /dev/null and b/2026/day-64/task-06-state-drift.png differ diff --git a/2026/day-65/day-65-modules.md b/2026/day-65/day-65-modules.md new file mode 100644 index 0000000000..10ea489700 --- /dev/null +++ b/2026/day-65/day-65-modules.md @@ -0,0 +1,97 @@ +# Day 65: Terraform Modules - Build Reusable Infrastructure + +## Task 1: Custom Module Directory Structure +A Terraform module is a package of `.tf` files. The root module calls child modules. +```text +terraform-modules-project/ +β”œβ”€β”€ main.tf # Root module -- calls child modules +β”œβ”€β”€ variables.tf # Root variables +β”œβ”€β”€ outputs.tf # Root outputs +β”œβ”€β”€ providers.tf # Provider config +└── modules/ + β”œβ”€β”€ ec2-instance/ # Child Module (EC2) + β”‚ β”œβ”€β”€ main.tf + β”‚ β”œβ”€β”€ variables.tf + β”‚ └── outputs.tf + └── security-group/ # Child Module (SG) + β”œβ”€β”€ main.tf + β”œβ”€β”€ variables.tf + └── outputs.tf +``` +*(Reference: [task-01-module-structure.png](screenshots/task-01-module-structure.png))* + +## Task 2: Custom EC2 Module Code + +**`variables.tf` (Inputs)** +```hcl +variable "instance_name" { type = string } +variable "environment" { type = string, default = "dev" } +variable "ami_id" { type = string } +variable "instance_type" { type = string, default = "t2.micro" } +variable "subnet_id" { type = string } +variable "security_group_ids" { type = list(string) } +variable "tags" { type = map(string), default = {} } +``` + +**`main.tf` (Resource definition)** +```hcl +resource "aws_instance" "this" { + ami = var.ami_id + instance_type = var.instance_type + subnet_id = var.subnet_id + vpc_security_group_ids = var.security_group_ids + tags = merge( + { Name = "${var.environment}-${var.instance_name}" }, + var.tags, + ) +} +``` +*(Reference: [task-02-ec2-module.png](screenshots/task-02-ec2-module.png))* + +## Task 3: Custom Security Group Module (Dynamic Blocks) + +**`main.tf` (Dynamic Ingress Block)** +```hcl +resource "aws_security_group" "this" { + name = var.sg_name + vpc_id = var.vpc_id + + dynamic "ingress" { + for_each = var.ingress_ports + content { + from_port = ingress.value + to_port = ingress.value + protocol = "tcp" + cidr_blocks = ["0.0.0.0/0"] + } + } +} +``` +*(Reference: [task-03-sg-module.png](screenshots/task-03-sg-module.png))* + +## Task 4: Root main.tf (Calling Modules with for_each) +```hcl +# Custom EC2 Module (Deployed twice using for_each) +module "servers" { + source = "./modules/ec2-instance" + for_each = toset(["web-server", "api-server"]) + instance_name = each.key + ami_id = data.aws_ami.amazon_linux.id + subnet_id = module.vpc.public_subnets[0] + security_group_ids = [module.web_sg.sg_id] +} +``` +*(Reference: [task-04-apply-modules.png](screenshots/task-04-apply-modules.png))* + +## Task 5: Hand-written VPC vs Registry Module Comparison +When building the VPC manually in Day 62, it took ~6 resources and 50+ lines of code to create a basic network. +By switching to the official `terraform-aws-modules/vpc/aws` registry module, a single block of code generated **17 networking resources** (Route tables, ACLs, associations, IGW, etc.) in the background flawlessly. The registry module was downloaded into `.terraform/modules/vpc`. + +*(Reference: [task-05-registry-module.png](screenshots/task-05-registry-module.png))* + +## 6. Five Module Best Practices +1. **Always pin versions for registry modules:** Use pessimistic constraints (e.g., `~> 5.0`) or Git tags to prevent sudden breaking changes when publishers update their code. +2. **Keep modules focused:** A module should do one thing well (e.g., just an EC2 instance, or just an RDS cluster). Do not build "mega-modules" that do everything. +3. **Use variables for everything, hardcode nothing:** Pass dynamic IDs (like VPC or Subnet IDs) from the root module so the child module can be reused across different AWS accounts and regions. +4. **Always define outputs:** Expose resource IDs and IPs in `outputs.tf` so that other modules (or the root module) can consume them (Modular Composition). +5. **Add a README.md:** Always document your custom module so other engineers in your team know what inputs are required and what outputs are returned. diff --git a/2026/day-65/screenshots/task-01-module-structure.png b/2026/day-65/screenshots/task-01-module-structure.png new file mode 100644 index 0000000000..da3d9e242b Binary files /dev/null and b/2026/day-65/screenshots/task-01-module-structure.png differ diff --git a/2026/day-65/screenshots/task-02-ec2-module.png b/2026/day-65/screenshots/task-02-ec2-module.png new file mode 100644 index 0000000000..d204ab4087 Binary files /dev/null and b/2026/day-65/screenshots/task-02-ec2-module.png differ diff --git a/2026/day-65/screenshots/task-03-sg-module.png b/2026/day-65/screenshots/task-03-sg-module.png new file mode 100644 index 0000000000..18d677179f Binary files /dev/null and b/2026/day-65/screenshots/task-03-sg-module.png differ diff --git a/2026/day-65/screenshots/task-04-apply-modules.png b/2026/day-65/screenshots/task-04-apply-modules.png new file mode 100644 index 0000000000..68434f25fe Binary files /dev/null and b/2026/day-65/screenshots/task-04-apply-modules.png differ diff --git a/2026/day-65/screenshots/task-05-registry-module.png b/2026/day-65/screenshots/task-05-registry-module.png new file mode 100644 index 0000000000..1cd68b6309 Binary files /dev/null and b/2026/day-65/screenshots/task-05-registry-module.png differ