Day-7 | Live AWS Project using SHELL SCRIPTING for DevOps | AWS DevOps project| #devops #aws #2023

Video Title:

Day-7 | Live AWS Project using Shell Scripting for DevOps | AWS DevOps project| #devops #aws #2023

---

Summary of Technological Concepts, Product Features, and Analysis

1. Context & Purpose:
   • The video is part of a full DevOps course (Day 7) focusing on a real-time AWS project using Shell Scripting.
   • The project simulates a common DevOps task: tracking AWS resource usage to maintain cost-effectiveness and manageability on cloud infrastructure.
2. Why Move to Cloud?
   • Two primary reasons:
     1. Reduce maintenance overhead (no need to manage physical servers).
     2. Cost-effectiveness via pay-as-you-go pricing.
   • Organizations need to track resource usage to avoid paying for unused resources (e.g., unused EC2 instances or unattached EBS volumes).
3. Project Objective:
   • Create a shell script to generate daily reports on AWS resource usage for key resources: EC2 instances, S3 buckets, Lambda functions, and IAM users.
   • The report helps managers monitor usage and control costs.
4. Implementation Approach:
   • Use AWS CLI commands within a Bash shell script to query AWS resources.
   • The script outputs resource details and can be scheduled to run automatically using a Cron job (Linux scheduler), ensuring daily reports without manual intervention.
   • The example organization is “example.com,” and the demo uses a real AWS account.
5. Key AWS CLI Commands Used:
   • aws s3 ls — List all S3 buckets.
   • aws ec2 describe-instances — List EC2 instances.
   • aws lambda list-functions — List Lambda functions.
   • aws iam list-users — List IAM users.
6. Shell Scripting Best Practices:
   • Use shebang (#!/bin/bash) for bash scripting to avoid compatibility issues with other shells like dash.
   • Add comments at the top of the script for author info, date, version, and purpose.
   • Add comments before commands for clarity and maintainability.
   • Use echo statements to print descriptive messages for better readability of output.
   • Use chmod to set executable permissions on the script.
7. Output Handling and Debugging:
   • Redirect output to files for easier reading (| more, or output redirection).
   • Use set -x to enable debug mode, which prints each command before execution for troubleshooting.
8. Improving Output with JSON Parsing:
   • AWS CLI outputs JSON data, which can be verbose.
   • Use jq, a JSON parser tool, to extract specific fields (e.g., instance IDs) to simplify and clarify the output.
   • Example:

     aws ec2 describe-instances | jq '.Reservations[].Instances[].InstanceId'

   • This approach filters out unnecessary details and presents concise information.
9. Scheduling with Cron job:
   • Automate script execution daily at a fixed time (e.g., 6 PM) using cron.
   • Cron jobs run scripts automatically without manual login or intervention, ensuring timely reports.
10. Further Improvements & Assignments:
    • The instructor encourages viewers to enhance the script by integrating it with cron and improving modularity (e.g., using shell functions).
    • Future videos will cover more advanced Shell Scripting projects.

---

Key Takeaways / Guide Steps for the Project

1. Setup:
   • Install and configure AWS CLI (aws configure with access key, secret key, region, and output format).
   • Ensure bash shell environment and jq installed.
2. Script Development:
   • Start with a bash script with proper shebang.
   • Write commands to list AWS resources using AWS CLI.
   • Add echo statements for clarity.
   • Use jq to parse JSON output and simplify reports.
3. Execution & Debugging:
   • Make script executable (chmod +x script.sh).
   • Run script and verify output.
   • Use set -x for debugging if needed.
4. Automation:
   • Create a Cron job to schedule the script daily at a specific time.
5. Output Management:
   • Redirect script output to a file for reporting purposes.

---

Main Speaker / Source

• Abhishek — The instructor and channel owner guiding through the AWS DevOps project with Shell Scripting.

This video serves as a practical tutorial for DevOps engineers or AWS administrators to automate AWS resource usage tracking using Shell Scripting and AWS CLI, emphasizing cost control and automation best practices.