Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removing credentials from log #1241

Merged
merged 3 commits into from
Apr 2, 2021

Conversation

PravinRanjan10
Copy link
Collaborator

This PR is about removing Access and Security crednetials details from log. WHile creating/deleting or doing any operation on bucket, Access and Security key of users were visible in logs. So, need to remove due to security violance issue.

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:

/kind new feature
/kind bug fix
/kind cleanup
/kind revert change
/kind design
/kind documentation
/kind enhancement

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Test Report Added?:

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:
/kind TESTED
/kind NOT-TESTED

Test Report:
I tried to collect log of:
Create Bucket
Delete Bucket
List Bucket
Upload Object


c: {S3Domain:[s3.test.com s3-internal.test.com] Region:cn-bj-1 Plugins:map[dummy_iam:{Path:/etc/yig/plugins/dummy_iam_plugin.so Enable:true Args:map[url:s3.test.com]} not_exist:{Path:not_exist_so Enable:false Args:map[]}] LogPath:/var/log/multi-cloud/s3.log AccessLogPath:/var/log/multi-cloud/access.log AccessLogFormat:{combined} PanicLogPath:/var/log/multi-cloud/panic.log PidFile:/var/run/multi-cloud/s3.pid BindApiAddress:0.0.0.0:8080 BindAdminAddress:0.0.0.0:9000 SSLKeyPath: SSLCertPath: ZookeeperAddress:hbase:2181 InstanceId: ConcurrentRequestLimit:0 HbaseZnodeParent: HbaseTimeout:0s DebugMode:true AdminKey:secret GcThread:0 LcThread:0 LcDebug:true LogLevel:20 CephConfigPattern: ReservedOrigins:s3.test.com,s3-internal.test.com MetaStore:tidb TidbInfo:root:@tcp(tidb:4000)/s3 KeepAlive:true RedisAddress:redis:6379 RedisConnectionNumber:10 RedisPassword: MetaCacheType:0 EnableDataCache:false RedisMode:0 RedisNodes:[] RedisSentinelMasterName:master RedisConnectTimeout:1 RedisReadTimeout:1 RedisWriteTimeout:1 RedisKeepAlive:60 RedisPoolMaxIdle:3 RedisPoolIdleTimeout:30 CacheCircuitCheckInterval:3 CacheCircuitCloseSleepWindow:1 CacheCircuitCloseRequiredCount:3 CacheCircuitOpenThreshold:1 DownLoadBufPoolSize:0 KMS:{Type: Endpoint: Id: Secret: Version:0 Keyname:} MsgBus:{Enabled:false Type:0 Topic: RequestTimeoutMs:0 MessageTimeoutMs:0 SendMaxRetries:0 Server:map[]}}
[2021-04-01T21:56:03+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/cmd/main.go] [main.main():62] [PID:11] YIG conf: {S3Domain:[s3.test.com s3-internal.test.com] Region:cn-bj-1 Plugins:map[dummy_iam:{Path:/etc/yig/plugins/dummy_iam_plugin.so Enable:true Args:map[url:s3.test.com]} not_exist:{Path:not_exist_so Enable:false Args:map[]}] LogPath:/var/log/multi-cloud/s3.log AccessLogPath:/var/log/multi-cloud/access.log AccessLogFormat:{combined} PanicLogPath:/var/log/multi-cloud/panic.log PidFile:/var/run/multi-cloud/s3.pid BindApiAddress:0.0.0.0:8080 BindAdminAddress:0.0.0.0:9000 SSLKeyPath: SSLCertPath: ZookeeperAddress:hbase:2181 InstanceId:7ZA654SGU1P580FN ConcurrentRequestLimit:10000 HbaseZnodeParent:/hbase HbaseTimeout:30s DebugMode:true AdminKey:secret GcThread:1 LcThread:1 LcDebug:true LogLevel:20 CephConfigPattern: ReservedOrigins:s3.test.com,s3-internal.test.com MetaStore:tidb TidbInfo:root:@tcp(tidb:4000)/s3 KeepAlive:true RedisAddress:redis:6379 RedisConnectionNumber:10 RedisPassword: MetaCacheType:0 EnableDataCache:false RedisMode:0 RedisNodes:[] RedisSentinelMasterName:master RedisConnectTimeout:1 RedisReadTimeout:1 RedisWriteTimeout:1 RedisKeepAlive:60 RedisPoolMaxIdle:3 RedisPoolIdleTimeout:30 CacheCircuitCheckInterval:3 CacheCircuitCloseSleepWindow:1 CacheCircuitCloseRequiredCount:3 CacheCircuitOpenThreshold:1 DownLoadBufPoolSize:524288 KMS:{Type: Endpoint: Id: Secret: Version:0 Keyname:} MsgBus:{Enabled:false Type:0 Topic: RequestTimeoutMs:3000 MessageTimeoutMs:5000 SendMaxRetries:2 Server:map[]}}

[2021-04-01T21:56:03+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/cmd/main.go] [main.main():63] [PID:11] YIG instance ID:%!(EXTRA string=7ZA654SGU1P580FN)
Can't find database driver tidb!
[2021-04-01T21:56:03+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/service.go] [service.initStorageClass():311] [PID:11] USE_DEFAULT_STORAGE_CLASS:set=1, val=1, err=.

[2021-04-01T21:56:03+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/service.go] [service.loadDefaultStorageClass():256] [PID:11] Supported storage classes:[{STANDARD 1 {} [] 0} {STANDARD_IA 99 {} [] 0} {GLACIER 999 {} [] 0}]

[2021-04-01T21:56:03+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/service.go] [service.loadDefaultStorageClass():269] [PID:11] Int2ExtTierMap:map[OpenSDS:0xc0000112b8 alibaba-oss:0xc000011318 aws-s3:0xc0000112a8 azure-blob:0xc0000112c8 ceph-s3:0xc0000112f8 fusionstorage-object:0xc000011308 gcp-s3:0xc0000112e8 hw-obs:0xc0000112d8]

[2021-04-01T21:56:03+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/service.go] [service.loadDefaultStorageClass():270] [PID:11] Ext2IntTierMap:map[OpenSDS:0xc0000112c0 alibaba-oss:0xc000011320 aws-s3:0xc0000112b0 azure-blob:0xc0000112d0 ceph-s3:0xc000011300 fusionstorage-object:0xc000011310 gcp-s3:0xc0000112f0 hw-obs:0xc0000112e0]

[2021-04-01T21:56:03+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/service.go] [service.loadDefaultTransition():288] [PID:11] loadDefaultTransition:map[1:[1] 99:[1 99] 999:[1 99 999]]

[2021-04-01T21:56:03+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/cache.go] [meta.newMetaCache():49] [PID:11] Setting Up Metadata Cache: NOCACHE

[2021-04-01T21:56:03+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/client.go] [tidb.NewTidbClient():47] [PID:11] connected to tidb ...
2021-04-01 21:56:03 file=v2@v2.9.1/service.go:200 level=info Starting [service] s3
2021-04-01 21:56:03 file=grpc/grpc.go:864 level=info Server [grpc] Listening on [::]:38743
2021-04-01 21:56:03 file=grpc/grpc.go:697 level=info Registry [mdns] Registering node: s3-69e9ca08-0b60-4d35-a192-1d395a89bee7
[2021-04-01T21:56:31+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/object.go] [service.(*s3Service).ListObjects():1087] [PID:11] ListObject is called in s3 service, bucket is pravin-jshjhsjhdshddjdhf.

[2021-04-01T21:56:31+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).GetBucket():37] [PID:11] get bucket[pravin-jshjhsjhdshddjdhf] from tidb ...

[2021-04-01T21:56:31+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).GetBucketSSE():708] [PID:11] list bucket SSE info from tidb ...
[2021-04-01T21:56:31+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/bucket.go] [meta.(*Meta).GetBucket.func1():38] [PID:11] GetBucket CacheMiss. bucket:pravin-jshjhsjhdshddjdhf
[2021-04-01T21:56:31+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/util/util.go] [util.GetCredentialFromCtx():46] [PID:11] isAdmin=false, tenantId=94b280022d0c4401bcf3b0ea85870519, userId=558057c4256545bd8a307c37464003c9, err=

[2021-04-01T21:56:31+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/object.go] [service.(*s3Service).ListObjectsInternal():1165] [PID:11] Prefix: Marker: MaxKeys: 1000 Delimiter: Version: 1 keyMarker: versionIdMarker:
[2021-04-01T21:56:31+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).ListObjects():333] [PID:11] sqltext:select bucketname,name,version from objects where bucketName=? and deletemarker=0 order by bucketname,name,version limit ?, args:[pravin-jshjhsjhdshddjdhf 10000]

[2021-04-01T21:56:33+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).DeleteBucket():219] [PID:11] DeleteBucket is called in s3 service, bucketName is pravin-jshjhsjhdshddjdhf.

[2021-04-01T21:56:33+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).GetBucket():37] [PID:11] get bucket[pravin-jshjhsjhdshddjdhf] from tidb ...

[2021-04-01T21:56:33+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).GetBucketSSE():708] [PID:11] list bucket SSE info from tidb ...
[2021-04-01T21:56:33+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/bucket.go] [meta.(*Meta).GetBucket.func1():38] [PID:11] GetBucket CacheMiss. bucket:pravin-jshjhsjhdshddjdhf
[2021-04-01T21:56:33+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/util/util.go] [util.GetCredentialFromCtx():46] [PID:11] isAdmin=false, tenantId=94b280022d0c4401bcf3b0ea85870519, userId=558057c4256545bd8a307c37464003c9, err=

[2021-04-01T21:56:33+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).ListObjects():333] [PID:11] sqltext:select bucketname,name,version from objects where bucketName=? and deletemarker=0 order by bucketname,name,version limit ?, args:[pravin-jshjhsjhdshddjdhf 10000]

[2021-04-01T21:56:33+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/utils/utils.go] [utils.GetBackend():140] [PID:11] backendName is aws:

[2021-04-01T21:56:33+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/utils/utils.go] [utils.GetBackend():145] [PID:11] backendErr is :
[2021-04-01T21:56:33+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).DeleteBucket():255] [PID:11] The backend is:
aws
[2021-04-01T21:56:33+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/datastore/aws/aws.go] [aws.(*AwsAdapter).BucketDelete():193] [PID:11] Bucket delete is called in aws s3 service
[2021-04-01T21:56:34+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/datastore/aws/aws.go] [aws.(*AwsAdapter).BucketDelete():204] [PID:11] {

}
[2021-04-01T21:56:34+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).DeleteBucket():277] [PID:11] Delete bucket[pravin-jshjhsjhdshddjdhf] successfully

[2021-04-01T21:56:34+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).ListBuckets():45] [PID:11] ListBuckets is called in s3 service.
[2021-04-01T21:56:34+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).GetBuckets():138] [PID:11] list buckets from tidb ...
[2021-04-01T21:56:34+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/util/util.go] [util.GetCredentialFromCtx():46] [PID:11] isAdmin=false, tenantId=94b280022d0c4401bcf3b0ea85870519, userId=558057c4256545bd8a307c37464003c9, err=

[2021-04-01T21:56:34+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).ListBuckets():73] [PID:11] out.Buckets:[]

[2021-04-01T21:56:34+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).ListBuckets():45] [PID:11] ListBuckets is called in s3 service.
[2021-04-01T21:56:34+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).GetBuckets():138] [PID:11] list buckets from tidb ...
[2021-04-01T21:56:34+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/util/util.go] [util.GetCredentialFromCtx():46] [PID:11] isAdmin=false, tenantId=94b280022d0c4401bcf3b0ea85870519, userId=558057c4256545bd8a307c37464003c9, err=

[2021-04-01T21:56:34+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).ListBuckets():73] [PID:11] out.Buckets:[]

[2021-04-01T21:57:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():68] [PID:11] list gc objects ...
[2021-04-01T21:57:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/gc.go] [tidb.(*TidbClient).ListGcObjs():89] [PID:11] sqltext:select bucketname,name,version,location,objectid,storageMeta from gcobjs order by bucketname,name,version limit ?,?;, args:[0 1000]

[2021-04-01T21:57:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():93] [PID:11] total=0, deleted=0, offset=0

[2021-04-01T21:57:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():96] [PID:11] break this round of gc
[2021-04-01T21:58:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():68] [PID:11] list gc objects ...
[2021-04-01T21:58:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/gc.go] [tidb.(*TidbClient).ListGcObjs():89] [PID:11] sqltext:select bucketname,name,version,location,objectid,storageMeta from gcobjs order by bucketname,name,version limit ?,?;, args:[0 1000]

[2021-04-01T21:58:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():93] [PID:11] total=0, deleted=0, offset=0

[2021-04-01T21:58:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():96] [PID:11] break this round of gc
[2021-04-01T21:58:25+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).CreateBucket():78] [PID:11] CreateBucket is called in s3 service, in:name:"pravin-ytyfudfuf" tenantId:"94b280022d0c4401bcf3b0ea85870519" userId:"558057c4256545bd8a307c37464003c9" acl:<cannedAcl:"private" > createTime:1617294505 versioning:<Status:"Disabled" > defaultLocation:"aws"

[2021-04-01T21:58:25+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/utils/utils.go] [utils.GetBackend():140] [PID:11] backendName is aws:

[2021-04-01T21:58:25+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/utils/utils.go] [utils.GetBackend():145] [PID:11] backendErr is :
[2021-04-01T21:58:25+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/datastore/aws/aws.go] [aws.(*AwsAdapter).BucketCreate():68] [PID:11] Bucket create is called in aws service and input request is:name:"pravin-ytyfudfuf" tenantId:"94b280022d0c4401bcf3b0ea85870519" userId:"558057c4256545bd8a307c37464003c9" acl:<cannedAcl:"private" > createTime:1617294505 versioning:<Status:"Disabled" > defaultLocation:"aws"
[2021-04-01T21:58:26+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/datastore/aws/aws.go] [aws.(*AwsAdapter).BucketCreate():80] [PID:11] The bucket creation successful in aws-s3 service with output:%s{
Location: "http://pravin-ytyfudfuf.s3.amazonaws.com/"
}
[2021-04-01T21:58:26+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).CreateBucket():110] [PID:11] Bucket created successfully in s3 service
[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).GetBucket():37] [PID:11] get bucket[pravin-ytyfudfuf] from tidb ...

[2021-04-01T21:58:26+05:30] [ERROR] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/client.go] [tidb.handleDBError():59] [PID:11] db error:sql: no rows in result set

[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).CheckAndPutBucket():287] [PID:11] insert bucket[pravin-ytyfudfuf] into database.

[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/types/bucket.go] [types.Bucket.GetCreateSql():113] [PID:11] createTime=2021-04-01 21:58:26

[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).CreateBucket():118] [PID:11] Create bucket[pravin-ytyfudfuf] in database succeed, processed=true.

[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).CreateBucketSSE():694] [PID:11] create bucket[pravin-ytyfudfuf] SSE info[NONE] into tidb ...

[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).ListBuckets():45] [PID:11] ListBuckets is called in s3 service.
[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).GetBuckets():138] [PID:11] list buckets from tidb ...
[2021-04-01T21:58:26+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/util/util.go] [util.GetCredentialFromCtx():46] [PID:11] isAdmin=false, tenantId=94b280022d0c4401bcf3b0ea85870519, userId=558057c4256545bd8a307c37464003c9, err=

[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).GetBucketSSE():708] [PID:11] list bucket SSE info from tidb ...
[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).ListBuckets():73] [PID:11] out.Buckets:[name:"pravin-ytyfudfuf" tenantId:"94b280022d0c4401bcf3b0ea85870519" createTime:1617294506 serverSideEncryption:<sseType:"NONE" > versioning:<Status:"Disabled" > defaultLocation:"aws" ]

[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).ListBuckets():45] [PID:11] ListBuckets is called in s3 service.
[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).GetBuckets():138] [PID:11] list buckets from tidb ...
[2021-04-01T21:58:26+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/util/util.go] [util.GetCredentialFromCtx():46] [PID:11] isAdmin=false, tenantId=94b280022d0c4401bcf3b0ea85870519, userId=558057c4256545bd8a307c37464003c9, err=

[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/bucket.go] [tidb.(*TidbClient).GetBucketSSE():708] [PID:11] list bucket SSE info from tidb ...
[2021-04-01T21:58:26+05:30] [INFO] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/service/bucket.go] [service.(*s3Service).ListBuckets():73] [PID:11] out.Buckets:[name:"pravin-ytyfudfuf" tenantId:"94b280022d0c4401bcf3b0ea85870519" createTime:1617294506 serverSideEncryption:<sseType:"NONE" > versioning:<Status:"Disabled" > defaultLocation:"aws" ]

[2021-04-01T21:59:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():68] [PID:11] list gc objects ...
[2021-04-01T21:59:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/gc.go] [tidb.(*TidbClient).ListGcObjs():89] [PID:11] sqltext:select bucketname,name,version,location,objectid,storageMeta from gcobjs order by bucketname,name,version limit ?,?;, args:[0 1000]

[2021-04-01T21:59:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():93] [PID:11] total=0, deleted=0, offset=0

[2021-04-01T21:59:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():96] [PID:11] break this round of gc
[2021-04-01T22:00:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():68] [PID:11] list gc objects ...
[2021-04-01T22:00:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/meta/db/drivers/tidb/gc.go] [tidb.(*TidbClient).ListGcObjs():89] [PID:11] sqltext:select bucketname,name,version,location,objectid,storageMeta from gcobjs order by bucketname,name,version limit ?,?;, args:[0 1000]

[2021-04-01T22:00:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():93] [PID:11] total=0, deleted=0, offset=0

[2021-04-01T22:00:03+05:30] [DEBUG] [/root/gopath/src/github.com/sodafoundation/multi-cloud/s3/pkg/gc/gc.go] [gc.Run():96] [PID:11] break this round of gc


Special notes for your reviewer:

s3/pkg/service/bucket.go Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Apr 1, 2021

Codecov Report

Merging #1241 (f6d626e) into master (7afa15d) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #1241   +/-   ##
=======================================
  Coverage   48.00%   48.00%           
=======================================
  Files          10       10           
  Lines        1450     1450           
=======================================
  Hits          696      696           
  Misses        702      702           
  Partials       52       52           
Impacted Files Coverage Δ
backend/pkg/service/service.go 82.94% <100.00%> (ø)

Copy link
Collaborator

@kumarashit kumarashit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@skdwriting skdwriting left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@skdwriting skdwriting merged commit 4029704 into sodafoundation:master Apr 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants