Add DataPipeline Resource Support

[teraken0509]

Fixes #1538

Changes proposed in this pull request:

• Add DataPipeline Resource
  • Support Pipeline Objects
    • default
    • copy activity
    • Resources
      • ec2 resource
    • Databases
      • Rds Database
    • Data nodes
      • s3 data node
      • sql data node
    • schedule

Output from acceptance testing:

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSDataPipeline_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -parallel 20 -run=TestAccAWSDataPipeline_ -timeout 120m
=== RUN   TestAccAWSDataPipeline_basic
=== PAUSE TestAccAWSDataPipeline_basic
=== RUN   TestAccAWSDataPipeline_copyActivity
=== PAUSE TestAccAWSDataPipeline_copyActivity
=== RUN   TestAccAWSDataPipeline_disappears
=== PAUSE TestAccAWSDataPipeline_disappears
=== CONT  TestAccAWSDataPipeline_basic
=== CONT  TestAccAWSDataPipeline_disappears
=== CONT  TestAccAWSDataPipeline_copyActivity
--- PASS: TestAccAWSDataPipeline_disappears (31.63s)
--- PASS: TestAccAWSDataPipeline_basic (36.01s)
--- PASS: TestAccAWSDataPipeline_copyActivity (37.50s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	37.557s

TODO

• Add support serveral resources
• Add documentation

[scraly]

Do you know when this feature will be released?
Thanks! :-)

[teraken0509]

Rerun test after rebase master

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSDataPipeline_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -parallel 20 -run=TestAccAWSDataPipeline_ -timeout 120m
=== RUN   TestAccAWSDataPipeline_basic
=== PAUSE TestAccAWSDataPipeline_basic
=== RUN   TestAccAWSDataPipeline_copyActivity
=== PAUSE TestAccAWSDataPipeline_copyActivity
=== RUN   TestAccAWSDataPipeline_disappears
=== PAUSE TestAccAWSDataPipeline_disappears
=== CONT  TestAccAWSDataPipeline_basic
=== CONT  TestAccAWSDataPipeline_disappears
=== CONT  TestAccAWSDataPipeline_copyActivity
--- PASS: TestAccAWSDataPipeline_disappears (35.32s)
--- PASS: TestAccAWSDataPipeline_copyActivity (41.50s)
--- PASS: TestAccAWSDataPipeline_basic (41.64s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	41.694s

[noldar]

Nice work! Hope it will be merged soon!

[Szasza]

Thank you for the implementation, it is much appreciated.

@bflad do you happen to know when this will be merged?

Thank you in advance for the response.

[Szasza]

@kterada0509 apparently this now has a merge conflict.

[teraken0509]

@kterada0509 apparently this now has a merge conflict.

@Szasza Thanks. I rebased master.

[vauchok]

Thanks, @kterada0509
hope, it'll be included in the next terraform-aws-provider release. @bflad

[teraken0509]

Rebase master & fix to corresponding to sdk update.

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSDataPipeline_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -parallel 20 -run=TestAccAWSDataPipeline_ -timeout 120m
=== RUN   TestAccAWSDataPipeline_basic
=== PAUSE TestAccAWSDataPipeline_basic
=== RUN   TestAccAWSDataPipeline_copyActivity
=== PAUSE TestAccAWSDataPipeline_copyActivity
=== RUN   TestAccAWSDataPipeline_disappears
=== PAUSE TestAccAWSDataPipeline_disappears
=== CONT  TestAccAWSDataPipeline_basic
=== CONT  TestAccAWSDataPipeline_disappears
=== CONT  TestAccAWSDataPipeline_copyActivity
--- PASS: TestAccAWSDataPipeline_disappears (44.21s)
--- PASS: TestAccAWSDataPipeline_basic (50.54s)
--- PASS: TestAccAWSDataPipeline_copyActivity (74.77s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	74.838s

[Szasza]

@bflad do you happen to have any information on when will this get merged?

Thank you in advance for your insight, it is much appreciated.

[kyprifog]

Ditto, when will this be released? Lack of Terraform support for Data pipelines derailed an initiative I tried to push recently.

[scraly]

Any news about the release date? :-)
Thx

[Szasza]

@bflad now that TF 0.12 is released, could this PR get some attention please?

Thank you in advance for your time.

[teraken0509]

This pull request seems to be difficult to merge.
I split pr to #9267 .

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!