Adding An AWS Service

Adding a new AWS API service to awless is an important process that needs to be crafted patiently.

This process consists basically in reviewing the cumbersome/exhaustive params/attributes for a given AWS service and adapting them intelligently to awless commands/models.

It means that the awless CLI will be augmented. For instance adding ELB Classic Load Balancer, the following CLI commands/models would come to life:

1. Listing your new entity with relevant and enrich properties: awless list classicloadbalancer ...
2. CRUD on your new entity: awless create classicloadbalancer name=... vpc=... (also awless delete/attach/detach/update classicloadbalancer)
3. Show more properties for your new entity: awless show my-classic-loadb
4. Your new entity is transparently synced and now part of the local graph model. Relations from your new entity to already existing entity in the model have been built as well.

What kind of AWS service to add? Usually, infrastructure centric AWS services (EC2, ELB, S3, IAM, etc.) add the most value to awless since they complement and enrich the existing awless model of your infrastructure. Other AWS service that stands more on their own and do not enrich the infrastructure local graph can be added, but they have less relations with existing entities, so they will focus more on the listing. Also do not forget that each AWS service added to awless add more data to sync and grows the graph to manage.

Before doing a pull request to add a new service make sure you have read the other Contributing sections on Testing, Code generation, Code navigation, etc.

---

Now the method to help you understand this process is to take as reference this commit that added the support of Classic ELB. It is a very good real life example to see how to integrate a new service.

The following files from the commit starting by gen_ are generated code (check out documentation on code generation in awless)

• acceptance/aws/gen_factory.go
• acceptance/aws/gen_mocks.go
• aws/doc/gen_paramsdoc.go
• aws/fetch/gen_fetchers.go
• aws/services/gen_mocks_test.go
• aws/services/gen_services.go
• aws/spec/gen_cmds_defs.go
• aws/spec/gen_inits.go
• aws/spec/gen_runs.go
• cloud/rdf/gen_rdf.go
• cloud/properties/gen_properties.go

This commit should be self documenting but it can be a bit hard to see what is going on, so let us point to some important files in this commit:

• acceptance/aws/classicloadbalancer_test.go: main Test Driven file that you start from when you design a new service. That is where you can start thinking of how the commands will behave and look like. This is behaviour driven development.
• aws/conv/convert.go (and aws/conv/convert_test.go): mostly transform functions that allow to convert AWS complicated data structure to more friendly structure and format, usually for display.
• aws/conv/model.go: give the model for a classicloadbalancer, properties and transform functions for each.
• aws/doc/clidoc.go: manually added documentation
• aws/doc/paramsdoc.go: manually added documentation
• aws/fetch/config.go: manually add a new API when needed, in this case elb
• aws/services/relations.go: defines the graph relations between your entity and others
• aws/services/services_test.go: fetching/listing classicloadbalancer integration tests. A bit too cumbersome right now, they should be refactored and adopt the spec acceptance model.
• aws/spec/classicloadbalancer.go: classicloadbalancer spec containing all the manual driver implementation and logic
• aws/spec/setters.go (and aws/spec/setters_test.go): contains complex type converters to convert from CLI params to AWS data struct
• cloud/cloud.go: entity high level and global strings
• console/defaults.go: all that has to do on how entities will be displayed in listing
• gen/aws/fetchers_definitions.go: manually augment AWS definitions to feed and help code generation (make generate)
• gen/aws/generators/paramsdoc.go: manually added documentation for params
• gen/aws/mock_definitions.go: manually added to have some integration test mocks on listing entities
• gen/aws/properties_definitions.go: corresponding new RDF vocabulary needed for your entity. Usually what you need will be already available in the vocabulary.
• graph/resourcetest/resourcetest.go: builder pattern for the entity to quickly build entity with wanted properties using simple DSL (all this should be refactored in another proper package)
• template/internal/ast/entities.go: internally we define to the parser engine that we accept a new entity classicloadbalancer from the command line
• template/revert.go (and template/revert_test.go): all the logic for reverting a classic load balancer. This massive revert method is unit/integration tested, but it is still horrible. Revert logic should be captured in each entity from aws/spec. That is what the aws/spec was intended for actually. It will be much more robust, simple and scalable.