AWS Inventory

This is a brand new version, completely rebuilt from scratch (again) in nov. 2024. It may be quicker and a bit simpler to maintain, but the inventory may have sometimes less details than the previous one.

Overview

Did your never forget to stop an EC2 instance? Or to delete some unusued AWS resource? I'm sure you did. And only remembering it when getting the bill. I know, AWS Config may help you, but did you ever try? It's prohibitive! This python script lists all the main resources owned within your AWS account. This inventory may be uncomplete, but it should help you to find what I call main resources that are, in my mind, resources that should affect billing and/or security.

Intended for personal and non-professional use (even if I added some professional features like logging), and for only one account. For more information on multiple accounts, read the wiki. This script provides a comprehensive inventory of AWS services used within a single account across all available regions. It leverages multithreading to perform inventory operations concurrently, ensuring efficient and timely data collection.

This project is donateware.

Main features

• Multithreading: Concurrently performs inventory operations to speed up data collection.
• Service Coverage: Supports a wide range of AWS services, including EC2, S3, RDS, Lambda, and more.
• Extendability: All ressources and details inventory services are in JSON files. No need to write code to add new ressources!
• Policy Management: Reads and merges IAM policy files to determine the necessary permissions for inventory operations directly from JSON files used in the project.
• Logging: Detailed logging of operations and errors for troubleshooting and auditing purposes. Ensures the existence of log and output directories, and creates a timestamped log file.
• Inventory Processes:
  • Retrieves all AWS regions and tests connectivity.
  • Creates a service structure based on IAM policy files.
  • Compiles and logs results, handles errors, and updates progress.
• Output: Generates JSON files with the inventory results, including metadata if specified.

The script functionally processes AWS service inventory with detailed logging, multi-threading, and customizable options via command-line arguments.

Project Structure and key Components

The new_inventory_api.py file is a Python script designed for inventorying AWS services for a specified account across all available regions. It uses multi-threading to perform inventory operations concurrently. The key components are:

• Global Variables: Manages inventory results, progress counters, and threading configurations.
• Modules Imported: threading, boto3, json, os, sys, re, datetime, time, argparse, multiprocessing, concurrent.futures, tqdm, glob, utils (custom utility functions).
• Classes:
  • InventoryThread: A thread class for performing inventory tasks.
• Functions:
  • Utility Functions: write_log, transform_function_name, json_serial, is_empty```.
  • *Inventory Management Functions: get_all_regions, test_region_connectivity, create_services_structure, inventory_handling, list_used_services.
• The main function initiates the script execution, parses arguments, and manages the inventory process.

Getting Started

Prerequisites

• Python 3.11+
• AWS CLI configured with appropriate credentials (see wiki)
• Install required Python packages:

  pip install -r requirements.txt

Usage

Please see wiki.

How to contribute?

Testing

TESTERS WANTED! If you test this code, please send me feedback: I can't test every configuration (especially when there are a lot of items in inventories), so either if it works or not, let me know what is fine and what needs to be corrected (use issues).

Writing some stuff

Contributions are welcome! Please fork the repository and submit a pull request.

License

This project is licensed under the terms of the GNU General Public License v3.0. See the LICENSE file for details.