Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate Storage Gateway resources to AWS SDK V2 #38554

Merged
merged 42 commits into from
Jul 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
9d06385
feat: regenerate client to v2
DanielRieske Jul 23, 2024
4bba24e
feat: migrate r/cache to sdkv2
DanielRieske Jul 23, 2024
bd171da
feat: migrate errors and find package to use sdkv2
DanielRieske Jul 23, 2024
394b8e9
Merge branch 'main' into f/migrate-storagegateway-sdkv2
DanielRieske Jul 26, 2024
68394b5
feat: migrate r/working_storage to SDKv2
DanielRieske Jul 26, 2024
ef9d002
feat: migrate r/cached_iscsi_volume to SDKv2
DanielRieske Jul 26, 2024
2773371
feat: migrate r/file_system_association to SDKv2
DanielRieske Jul 26, 2024
921fbf6
feat: migrate r/gateway to SDKv2
DanielRieske Jul 26, 2024
90183a0
feat: migrate r/nfs_file_share to SDKv2
DanielRieske Jul 26, 2024
443784c
feat: migrate r/smb_file_share to SDKv2
DanielRieske Jul 26, 2024
d1c250d
feat: migrate r/stored_iscsi_volume to SDKv2
DanielRieske Jul 26, 2024
4cd2069
feat: migrate r/upload_buffer to SDKv2
DanielRieske Jul 26, 2024
2f03d58
feat: migrate sweepers to SDKv2
DanielRieske Jul 26, 2024
bbff4e1
feat: migrate r/tape_pool to SDKv2
DanielRieske Jul 26, 2024
77f4b74
feat: migrate d/local_disk to SDKv2
DanielRieske Jul 26, 2024
07b9973
chore: reduce visibility
DanielRieske Jul 26, 2024
93b918f
feat: migrate `find` functions
DanielRieske Jul 26, 2024
2fe6a86
feat: migrate `status` functions to SDKv2
DanielRieske Jul 26, 2024
60ba820
feat: migrate `wait` functions to SDKv2
DanielRieske Jul 26, 2024
3755e47
chore: fix exports
DanielRieske Jul 26, 2024
5b18c41
fix: nil exception
DanielRieske Jul 26, 2024
c65609a
fix: fix `TestAccStorageGatewayTapePool_disappears`
DanielRieske Jul 26, 2024
b62214d
fix: spelling mistake in error message
DanielRieske Jul 26, 2024
0070e3f
fix: fix `TestAccStorageGatewayGateway_GatewayType_cached`
DanielRieske Jul 27, 2024
e8165f6
Merge branch 'main' into HEAD
ewbankkit Jul 27, 2024
703b349
Run 'go mod tidy'.
ewbankkit Jul 27, 2024
29fa298
Add 'findCacheByTwoPartKey'.
ewbankkit Jul 27, 2024
9077e79
Add 'findCachediSCSIVolumeByARN'.
ewbankkit Jul 28, 2024
6a680d5
r/aws_storagegateway_file_system_association: Cosmetics.
ewbankkit Jul 28, 2024
450041e
r/aws_storagegateway_gateway: Cosmetics.
ewbankkit Jul 29, 2024
4963ecb
d/aws_storagegateway_local_disk: Cosmetics.
ewbankkit Jul 29, 2024
ea20d02
r/aws_storagegateway_nfs_file_share: Cosmetics.
ewbankkit Jul 29, 2024
e2341a5
r/aws_storagegateway_smb_file_share: Cosmetics.
ewbankkit Jul 29, 2024
6d5d1a7
Add 'findStorediSCSIVolumeByARN'.
ewbankkit Jul 29, 2024
f3278ac
Add 'findTapePoolByARN'.
ewbankkit Jul 29, 2024
44c3acf
r/aws_storagegateway_upload_buffer: Cosmetics.
ewbankkit Jul 30, 2024
c524616
r/aws_storagegateway_working_storage: Add 'findWorkingStorageDiskIDBy…
ewbankkit Jul 31, 2024
2864aec
storagegateway: Tweak 'operationErrorCode'.
ewbankkit Jul 31, 2024
998c0d6
Merge branch 'main' into HEAD
ewbankkit Jul 31, 2024
4ab7daa
r/aws_storagegateway_file_system_association: Add configurable timeouts.
ewbankkit Jul 31, 2024
dafc14b
Fix semgrep 'dgryski.semgrep-go.errnilcheck.err-nil-check'.
ewbankkit Jul 31, 2024
ce32cd7
Fix 'TestAccStorageGatewayLocalDiskDataSource_diskNode' and 'TestAccS…
ewbankkit Jul 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .changelog/38554.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:enhancement
resource/aws_storagegateway_file_system_association: Add configurable timeouts
```
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,7 @@ require (
github.com/aws/aws-sdk-go-v2/service/ssmsap v1.15.3
github.com/aws/aws-sdk-go-v2/service/sso v1.22.4
github.com/aws/aws-sdk-go-v2/service/ssoadmin v1.27.4
github.com/aws/aws-sdk-go-v2/service/storagegateway v1.31.3
github.com/aws/aws-sdk-go-v2/service/sts v1.30.3
github.com/aws/aws-sdk-go-v2/service/swf v1.25.3
github.com/aws/aws-sdk-go-v2/service/synthetics v1.26.3
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,8 @@ github.com/aws/aws-sdk-go-v2/service/ssoadmin v1.27.4 h1:oXiKn9jcx+8yLLuwm8TO6qh
github.com/aws/aws-sdk-go-v2/service/ssoadmin v1.27.4/go.mod h1:EyoPT+dUT5zqspxSub9KHDWOZyIP30bPgIavBvGGVz0=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.26.4 h1:yiwVzJW2ZxZTurVbYWA7QOrAaCYQR72t0wrSBfoesUE=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.26.4/go.mod h1:0oxfLkpz3rQ/CHlx5hB7H69YUpFiI1tql6Q6Ne+1bCw=
github.com/aws/aws-sdk-go-v2/service/storagegateway v1.31.3 h1:0hdxWCS8mM4qsZI3GldBkXeee4X25aC9wZGQhpbt6w8=
github.com/aws/aws-sdk-go-v2/service/storagegateway v1.31.3/go.mod h1:N2tZQtDCR/Ls4o1pH6neRhhlkhKNE6SoruLn6nTpnzU=
github.com/aws/aws-sdk-go-v2/service/sts v1.30.3 h1:ZsDKRLXGWHk8WdtyYMoGNO7bTudrvuKpDKgMVRlepGE=
github.com/aws/aws-sdk-go-v2/service/sts v1.30.3/go.mod h1:zwySh8fpFyXp9yOr/KVzxOl8SRqgf/IDw5aUt9UKFcQ=
github.com/aws/aws-sdk-go-v2/service/swf v1.25.3 h1:7zYsHA9ORjiCHYzTJf0g+gwo3mPpn2XbMlWQreiXWdM=
Expand Down
6 changes: 3 additions & 3 deletions internal/conns/awsclient_gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

93 changes: 56 additions & 37 deletions internal/service/storagegateway/cache.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,16 @@ import (
"log"
"strings"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/arn"
"github.com/aws/aws-sdk-go/service/storagegateway"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/aws/arn"
"github.com/aws/aws-sdk-go-v2/service/storagegateway"
awstypes "github.com/aws/aws-sdk-go-v2/service/storagegateway/types"
"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-provider-aws/internal/conns"
"github.com/hashicorp/terraform-provider-aws/internal/errs/sdkdiag"
tfslices "github.com/hashicorp/terraform-provider-aws/internal/slices"
"github.com/hashicorp/terraform-provider-aws/internal/tfresource"
"github.com/hashicorp/terraform-provider-aws/internal/verify"
)
Expand Down Expand Up @@ -49,17 +52,17 @@ func resourceCache() *schema.Resource {

func resourceCacheCreate(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {
var diags diag.Diagnostics
conn := meta.(*conns.AWSClient).StorageGatewayConn(ctx)
conn := meta.(*conns.AWSClient).StorageGatewayClient(ctx)

diskID := d.Get("disk_id").(string)
gatewayARN := d.Get("gateway_arn").(string)
id := cacheCreateResourceID(gatewayARN, diskID)
inputAC := &storagegateway.AddCacheInput{
DiskIds: aws.StringSlice([]string{diskID}),
DiskIds: []string{diskID},
GatewayARN: aws.String(gatewayARN),
}

_, err := conn.AddCacheWithContext(ctx, inputAC)
_, err := conn.AddCache(ctx, inputAC)

if err != nil {
return sdkdiag.AppendErrorf(diags, "creating Storage Gateway Cache (%s): %s", id, err)
Expand All @@ -76,16 +79,16 @@ func resourceCacheCreate(ctx context.Context, d *schema.ResourceData, meta inter
inputLLD := &storagegateway.ListLocalDisksInput{
GatewayARN: aws.String(gatewayARN),
}
disk, err := findLocalDisk(ctx, conn, inputLLD, func(v *storagegateway.Disk) bool {
return aws.StringValue(v.DiskId) == diskID || aws.StringValue(v.DiskNode) == diskID || aws.StringValue(v.DiskPath) == diskID
disk, err := findLocalDisk(ctx, conn, inputLLD, func(v awstypes.Disk) bool {
return aws.ToString(v.DiskId) == diskID || aws.ToString(v.DiskNode) == diskID || aws.ToString(v.DiskPath) == diskID
})

switch {
case tfresource.NotFound(err):
case err != nil:
return sdkdiag.AppendErrorf(diags, "reading Storage Gateway local disk: %s", err)
default:
id = cacheCreateResourceID(gatewayARN, aws.StringValue(disk.DiskId))
id = cacheCreateResourceID(gatewayARN, aws.ToString(disk.DiskId))
d.SetId(id)
}

Expand All @@ -94,45 +97,23 @@ func resourceCacheCreate(ctx context.Context, d *schema.ResourceData, meta inter

func resourceCacheRead(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {
var diags diag.Diagnostics
conn := meta.(*conns.AWSClient).StorageGatewayConn(ctx)
conn := meta.(*conns.AWSClient).StorageGatewayClient(ctx)

gatewayARN, diskID, err := cacheParseResourceID(d.Id())
if err != nil {
return sdkdiag.AppendFromErr(diags, err)
}

input := &storagegateway.DescribeCacheInput{
GatewayARN: aws.String(gatewayARN),
}

output, err := conn.DescribeCacheWithContext(ctx, input)
if err != nil {
if IsErrGatewayNotFound(err) {
log.Printf("[WARN] Storage Gateway Cache (%s) not found, removing from state", d.Id())
d.SetId("")
return diags
}
return sdkdiag.AppendErrorf(diags, "reading Storage Gateway Cache (%s): %s", d.Id(), err)
}
err = findCacheByTwoPartKey(ctx, conn, gatewayARN, diskID)

if output == nil || len(output.DiskIds) == 0 {
if !d.IsNewResource() && tfresource.NotFound(err) {
log.Printf("[WARN] Storage Gateway Cache (%s) not found, removing from state", d.Id())
d.SetId("")
return diags
}

found := false
for _, existingDiskID := range output.DiskIds {
if aws.StringValue(existingDiskID) == diskID {
found = true
break
}
}

if !found {
log.Printf("[WARN] Storage Gateway Cache (%s) not found, removing from state", d.Id())
d.SetId("")
return diags
if err != nil {
return sdkdiag.AppendErrorf(diags, "reading Storage Gateway Cache (%s): %s", d.Id(), err)
}

d.Set("disk_id", diskID)
Expand All @@ -158,7 +139,7 @@ func cacheParseResourceID(id string) (string, string, error) {
return "", "", idFormatErr
}
// gatewayARNAndDisk.Resource = gateway/sgw-12345678:pci-0000:03:00.0-scsi-0:0:0:0
resourceParts := strings.SplitN(gatewayARNAndDisk.Resource, ":", 2)
resourceParts := strings.SplitN(gatewayARNAndDisk.Resource, cacheResourceIDSeparator, 2)
if len(resourceParts) != 2 {
return "", "", idFormatErr
}
Expand All @@ -172,3 +153,41 @@ func cacheParseResourceID(id string) (string, string, error) {
}
return gatewayARN.String(), resourceParts[1], nil
}

func findCacheByTwoPartKey(ctx context.Context, conn *storagegateway.Client, gatewayARN, diskID string) error {
input := &storagegateway.DescribeCacheInput{
GatewayARN: aws.String(gatewayARN),
}
output, err := findCache(ctx, conn, input)

if err != nil {
return err
}

_, err = tfresource.AssertSingleValueResult(tfslices.Filter(output.DiskIds, func(v string) bool {
return v == diskID
}))

return err
}

func findCache(ctx context.Context, conn *storagegateway.Client, input *storagegateway.DescribeCacheInput) (*storagegateway.DescribeCacheOutput, error) {
output, err := conn.DescribeCache(ctx, input)

if isGatewayNotFoundErr(err) {
return nil, &retry.NotFoundError{
LastError: err,
LastRequest: input,
}
}

if err != nil {
return nil, err
}

if output == nil {
return nil, tfresource.NewEmptyResultError(input)
}

return output, err
}
55 changes: 15 additions & 40 deletions internal/service/storagegateway/cache_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ import (
"fmt"
"testing"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/storagegateway"
sdkacctest "github.com/hashicorp/terraform-plugin-testing/helper/acctest"
"github.com/hashicorp/terraform-plugin-testing/helper/resource"
"github.com/hashicorp/terraform-plugin-testing/terraform"
Expand All @@ -19,7 +17,7 @@ import (
"github.com/hashicorp/terraform-provider-aws/names"
)

func TestDecodeCacheID(t *testing.T) {
func TestCacheParseResourceID(t *testing.T) {
t.Parallel()

var testCases = []struct {
Expand Down Expand Up @@ -87,9 +85,8 @@ func TestAccStorageGatewayCache_fileGateway(t *testing.T) {
PreCheck: func() { acctest.PreCheck(ctx, t) },
ErrorCheck: acctest.ErrorCheck(t, names.StorageGatewayServiceID),
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories,
// Storage Gateway API does not support removing caches,
// but we want to ensure other resources are removed.
CheckDestroy: testAccCheckGatewayDestroy(ctx),
// Storage Gateway API does not support removing caches.
CheckDestroy: acctest.CheckDestroyNoop,
Steps: []resource.TestStep{
{
Config: testAccCacheConfig_fileGateway(rName),
Expand Down Expand Up @@ -118,9 +115,7 @@ func TestAccStorageGatewayCache_tapeAndVolumeGateway(t *testing.T) {
PreCheck: func() { acctest.PreCheck(ctx, t) },
ErrorCheck: acctest.ErrorCheck(t, names.StorageGatewayServiceID),
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories,
// Storage Gateway API does not support removing caches,
// but we want to ensure other resources are removed.
CheckDestroy: testAccCheckGatewayDestroy(ctx),
CheckDestroy: acctest.CheckDestroyNoop,
Steps: []resource.TestStep{
{
Config: testAccCacheConfig_tapeAndVolumeGateway(rName),
Expand All @@ -139,53 +134,33 @@ func TestAccStorageGatewayCache_tapeAndVolumeGateway(t *testing.T) {
})
}

func testAccCheckCacheExists(ctx context.Context, resourceName string) resource.TestCheckFunc {
func testAccCheckCacheExists(ctx context.Context, n string) resource.TestCheckFunc {
return func(s *terraform.State) error {
rs, ok := s.RootModule().Resources[resourceName]
rs, ok := s.RootModule().Resources[n]
if !ok {
return fmt.Errorf("Not found: %s", resourceName)
return fmt.Errorf("Not found: %s", n)
}

conn := acctest.Provider.Meta().(*conns.AWSClient).StorageGatewayConn(ctx)
conn := acctest.Provider.Meta().(*conns.AWSClient).StorageGatewayClient(ctx)

gatewayARN, diskID, err := tfstoragegateway.CacheParseResourceID(rs.Primary.ID)
if err != nil {
return err
}

input := &storagegateway.DescribeCacheInput{
GatewayARN: aws.String(gatewayARN),
}

output, err := conn.DescribeCacheWithContext(ctx, input)

if err != nil {
return fmt.Errorf("error reading Storage Gateway cache: %s", err)
}

if output == nil || len(output.DiskIds) == 0 {
return fmt.Errorf("Storage Gateway cache %q not found", rs.Primary.ID)
}

for _, existingDiskID := range output.DiskIds {
if aws.StringValue(existingDiskID) == diskID {
return nil
}
}

return fmt.Errorf("Storage Gateway cache %q not found", rs.Primary.ID)
return tfstoragegateway.FindCacheByTwoPartKey(ctx, conn, gatewayARN, diskID)
}
}

func testAccCacheConfig_fileGateway(rName string) string {
return testAccGatewayConfig_typeFileS3(rName) + fmt.Sprintf(`
return acctest.ConfigCompose(testAccGatewayConfig_typeFileS3(rName), fmt.Sprintf(`
resource "aws_ebs_volume" "test" {
availability_zone = aws_instance.test.availability_zone
size = "10"
type = "gp2"

tags = {
Name = %q
Name = %[1]q
}
}

Expand Down Expand Up @@ -215,18 +190,18 @@ resource "aws_storagegateway_cache" "test" {
disk_id = data.aws_storagegateway_local_disk.test.id
gateway_arn = aws_storagegateway_gateway.test.arn
}
`, rName)
`, rName))
}

func testAccCacheConfig_tapeAndVolumeGateway(rName string) string {
return testAccGatewayConfig_typeCached(rName) + fmt.Sprintf(`
return acctest.ConfigCompose(testAccGatewayConfig_typeCached(rName), fmt.Sprintf(`
resource "aws_ebs_volume" "test" {
availability_zone = aws_instance.test.availability_zone
size = "10"
type = "gp2"

tags = {
Name = %q
Name = %[1]q
}
}

Expand Down Expand Up @@ -256,5 +231,5 @@ resource "aws_storagegateway_cache" "test" {
disk_id = data.aws_storagegateway_local_disk.test.id
gateway_arn = aws_storagegateway_gateway.test.arn
}
`, rName)
`, rName))
}
Loading
Loading