Adhoc Provider - Custom Resource / Data Source

[skorfmann]

Feature Spec

With Wing Custom Resources, it's now simple to interface with bespoke APIs and extend the Terraform inflight behaviour with custom code snippets.

bring resource

class GithubIssue impl resource.ICustomResource {
  create(title: str) {
    // ...
  }
  update(id: num, title: str) {
    // ...
  }
  read(id: num) {
    // ...
  }
  destroy(id: num) {
    // ...
  }
}

or for data fetching / manipulation something like this (inspired by this)

interface CustomDockerImageArgs {
  repositoryUrl: str;
  username: str;
  password: str;
}

class CustomDockerImage impl resource.ICustomDataSource {    
  sha256Digest: str?;  
  reposoitryUrl: str;
  username: str;
  password: str;

  init(config: CustomDockerImageArgs) {
    this.repositoryUrl = config.repositoryUrl;
    this.username = config.username;
    this.password = config.password;
  }

  onRead(): {    
    // For complex cases, this should be JS code. So maybe external?
    const drc = require('docker-registry-client')
    return new Promise((resolve, reject) => {
      var rar = drc.parseRepoAndRef(this.repositoryUrl);
      var client = drc.createClientV2({
        repo: rar,
        insecure: false,
        username: this.username,
        password: this.password,
        maxSchemaVersion: 2
      });
      var tagOrDigest = rar.tag || rar.digest;
      client.getManifest({ref: tagOrDigest}, function (err:any, _manifest:any, _res:any, manifestStr:any) {
        client.close();
        if (err) {
          reject(err)
        }
        this.sha256Digest = drc.digestFromManifestStr(manifestStr);
        resolve(this);
      });
    });
  }
}

Both, resources and data sources, are build ad-hoc as a Terraform provider when comping this with the Wing CLI transparent to to the user.

In a Wing application, this could be used like:

let image = new CustomDockerImage(...)

which would render down to a Terraform HCL (JSON) data source, something like (pseudo HCL code)

provider "wing-adhoc" {}

data "wing-adhoc_custom-docker-image" {
  repository_url = "https://abc"
  username = "hello"
  password = "world"
}

output "sha56" {
  value = data.wing-adhoc_custom-docker-image.sha56
}

Use Cases

• build ad hoc resources to interface with APIs not covered by a Terraform provider
• build ad hoc data sources, to fetch and / or manipulate data as needed
• perform custom computations at Terraform run time
• build test cases and bake them into Terraform
• enable running any JS code at Terraform run time

Implementation Notes

• I proposed the same for the CDK for Terraform project a while ago
• There's an incomplete and unmaintained nodejs port of the Go Plugin Framework https://github.com/lukekaalim/terraform-plugin-node-SDK/
• There is something similar in Pulumi with Dynamic Providers
• I believe Deno would be a great fit as the runtime enabling this, primarily due to the dependency management mechanics of url imports, the possibility to entirely sandbox a Deno process and creating binaries is a direct feature of Deno.
• This could be an open source project on its own and quite useful beyond Wing
• There is a Javascript Terraform provider for data wrangling https://github.com/lewispeckover/terraform-provider-javascript (based on https://github.com/dop251/goja).

There are two components to make this work at a high level:

1. Build a solid implementation of the Provider Plugin Framework in Javascript or find a way to utilize parts the Go implementation under the hood.
2. Build abstractions in Wing which can use this with minimal overhead

Component

Other

Community Notes

• Please vote by adding a 👍 reaction to the issue to help us prioritize.
• If you are interested to work on this issue, please leave a comment.

[github-actions]

Hi,

This issue hasn't seen activity in 60 days. Therefore, we are marking this issue as stale for now. It will be closed after 7 days.
Feel free to re-open this issue when there's an update or relevant information to be added.
Thanks!

[skorfmann]

Here's a use case for fly.io - while there's a Terraform provider, the readme states:

This project is not currently maintained, and is not a recommended method of deployment to Fly.io.

we want to do create a project via Terraform, right now this is shelling out via null_resource and local-exec. While this works for creation, it's pretty much a dead-end for destroying - see https://developer.hashicorp.com/terraform/language/resources/provisioners/syntax#destroy-time-provisioners

   let resource = new nullProvider.resource.Resource(triggers: { "changed": util.nanoid() }) as "create"
    resource.addOverride("provisioner.local-exec.environment", {"FLY_APP_NAME": appName});
    resource.addOverride("provisioner.local-exec.command", "
flyctl status -a \$FLY_APP_NAME || flyctl launch --copy-config --no-deploy --name \$FLY_APP_NAME -o ${props.org} -r iad -y
    ");

  // FIXME: We'll have to find another way to do this. Can't reference other resources here. Also, there are drawbacks
    // see https://developer.hashicorp.com/terraform/language/resources/provisioners/syntax#destroy-time-provisioners
  
   let destroy =  new nullProvider.resource.Resource() as "destroy";
    destroy.addOverride("provisioner.local-exec.when", "destroy");
    destroy.addOverride("provisioner.local-exec.environment", {"FLY_APP_NAME": appName});
    destroy.addOverride("provisioner.local-exec.command", "flyctl status -a \$FLY_APP_NAME && flyctl apps destroy \$FLY_APP_NAME -y");

so, if it was possible to do something like:

bring http;

struct FlyProjectProps {
  name: str;
  org: str;
  region: str;
}

class FlyProject extends CustomResource {
  var name: str;
  var region: str;
  var bucket: str;

  init(props: FlyProjectProps) {
    this.name = props.name;
    this.region = props.region;
    this.bucket = props.bucket;
  }

  // get invoked during terraform apply as normal Terraform resource
  pub create(): void {
    // Add code here to create the project via http
    http.post("https://api.fly.io/v2/projects", ...)
  }

  pub read(): void {
    // Add code here to read the resource
    http.get("https://api.fly.io/v2/projects", ...)
  }

  pub update(): void {
    // Add code here to update the resource
    http.put("https://api.fly.io/v2/projects", ...)
  }

  pub delete(): void {
    // Add code here to delete the resource
    http.delete("https://api.fly.io/v2/projects", ...)
  }
}

then this could become

bring fly

new fly.Project(
  name: "foo",
  region: "iad",
  org: "bar"
);

[github-actions]

Hi,

This issue hasn't seen activity in 90 days. Therefore, we are marking this issue as stale for now. It will be closed after 7 days.
Feel free to re-open this issue when there's an update or relevant information to be added.
Thanks!

[github-actions]

Hi,

This issue hasn't seen activity in 90 days. Therefore, we are marking this issue as stale for now. It will be closed after 7 days.
Feel free to re-open this issue when there's an update or relevant information to be added.
Thanks!