Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

objstore : implement Aliyun OSS #1234

Closed
wants to merge 48 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
d67a69d
Implemented AliYun OSS for objstore support
Jun 7, 2019
f973be4
restore github.com/improbable-eng/thanos/blob/master/go.mod & go.sum
Jun 7, 2019
4abb631
Implemented AliYun OSS for objstore support following Object Storage->
Jun 7, 2019
4538290
aliyun oss objstore works well
Jun 7, 2019
86c5548
put img under 'img' path
Jun 7, 2019
714b5d3
correct method Iter for pageing view objects
Jun 11, 2019
9dd19b7
Update config.yml
snowbubblyzone Jun 11, 2019
5202497
change storage.md
snowbubblyzone Jun 11, 2019
9b64327
change storage.md
snowbubblyzone Jun 11, 2019
3240df3
Update storage.md
snowbubblyzone Jun 11, 2019
aec4ee4
Update Makefile
snowbubblyzone Jun 11, 2019
1182217
Update config.yml
snowbubblyzone Jun 11, 2019
a827af3
Update config.yml
snowbubblyzone Jun 11, 2019
495908d
Update config.yml
snowbubblyzone Jun 11, 2019
a783bb3
Update oss.go
snowbubblyzone Jun 11, 2019
b47347f
Update config.yml
snowbubblyzone Jun 11, 2019
64f024f
correct access_key_id & access_key_secret
snowbubblyzone Jun 12, 2019
8ac2d43
Update CHANGELOG.md
snowbubblyzone Jun 14, 2019
7e47e41
delete 'make docs' , re-order the method Iter() in oss.go file
snowbubblyzone Jun 14, 2019
3c1f6e1
Update config.yml
snowbubblyzone Jun 14, 2019
d926d6e
Delete .config.yml.swp
snowbubblyzone Jun 14, 2019
2adae0c
Update docs/storage.md
snowbubblyzone Jun 14, 2019
b00d8a5
Update docs/storage.md
snowbubblyzone Jun 14, 2019
e0bd68a
Update pkg/objstore/client/factory.go
snowbubblyzone Jun 14, 2019
ecaa38b
Update proxy.go
snowbubblyzone Jun 14, 2019
0380d47
change object store from 'OSS' to 'AliYunOSS'
snowbubblyzone Jun 14, 2019
b66ad90
Update CHANGELOG.md
snowbubblyzone Jun 17, 2019
04f416c
add 1097 PR in CHANGELOG.md
snowbubblyzone Jun 18, 2019
e895342
add 1097 PR in CHANGELOG.md
snowbubblyzone Jun 18, 2019
a2ad5c5
add #1097 PR in CHANGELOG.md
snowbubblyzone Jun 18, 2019
400a927
Merge branch 'master' into master
snowbubblyzone Jun 18, 2019
1c90eb2
delete select code , rename AccessKeyID , delete
snowbubblyzone Jun 19, 2019
4a6854f
add error wrapper, change AliYunOSS to ALIYUNOSS provider, add @wujin…
snowbubblyzone Jun 21, 2019
b7ba6af
Merge branch 'master' into master
snowbubblyzone Jun 21, 2019
4a14d92
change aliyun oss code to support bucket creating & bucket reusing
snowbubblyzone Jun 25, 2019
dca0fb7
update code for multi-part upload , change improbable-eng to thanos-io
Jul 27, 2019
feed309
update code for multi-part upload , change improbable-eng to thanos-io
Jul 27, 2019
1091424
Merge branch 'master' into master
snowbubblyzone Jul 27, 2019
540fc5b
Update main.go
snowbubblyzone Jul 27, 2019
c1bab13
Update pool_test.go
snowbubblyzone Jul 27, 2019
f6f50d3
Update handler.go
snowbubblyzone Jul 27, 2019
1130f3d
Update proxy.go
snowbubblyzone Jul 27, 2019
656e1fb
Update bucket.go
snowbubblyzone Jul 27, 2019
037f411
Update oss.go
snowbubblyzone Jul 27, 2019
0f89466
Update oss.go
snowbubblyzone Jul 27, 2019
efa50b1
Update bucket.go
snowbubblyzone Jul 27, 2019
04c9098
update code for multi-part upload
Jul 27, 2019
6b6a70c
remove blank line , add PR to CHANGELOG.md
Aug 2, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ jobs:
echo "Skipping SWIFT tests."
export THANOS_SKIP_TENCENT_COS_TESTS="true"
echo "Skipping TENCENT COS tests."

export THANOS_SKIP_ALIYUN_OSS_TESTS="true"
This conversation was marked as resolved.
Show resolved Hide resolved
echo "Skipping ALIYUN OSS tests."
make test

# Cross build is needed for publish_release but needs to be done outside of docker.
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ We use *breaking* word for marking changes that are not backward compatible (rel
## Unreleased.

- [#1338](https://github.com/thanos-io/thanos/pull/1338) Querier still warns on store API duplicate, but allows a single one from duplicated set. This is gracefully warn about the problematic logic and not disrupt immediately.
- [#1234](https://github.com/thanos-io/thanos/pull/1234) `AliYun OSS` object storage, see [documents](docs/storage.md#aliyun-oss-configuration) for further information.

### Fixed

Expand All @@ -23,6 +24,7 @@ We use *breaking* word for marking changes that are not backward compatible (rel

### Added


- [#1097](https://github.com/thanos-io/thanos/pull/1097) Added `thanos check rules` linter for Thanos rule rules files.

- [#1253](https://github.com/thanos-io/thanos/pull/1253) Add support for specifying a maximum amount of retries when using Azure Blob storage (default: no retries).
Expand Down
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -149,8 +149,8 @@ docs: $(EMBEDMD) build
.PHONY: check-docs
check-docs: $(EMBEDMD) $(LICHE) build
@EMBEDMD_BIN="$(EMBEDMD)" scripts/genflagdocs.sh check
@$(LICHE) --recursive docs --exclude "cloud.tencent.com" --document-root .
@$(LICHE) --exclude "cloud.tencent.com" --document-root . *.md
@$(LICHE) --recursive docs --exclude "(cloud.tencent.com|alibabacloud.com)" --document-root .
@$(LICHE) --exclude "(cloud.tencent.com|alibabacloud.com)" --document-root . *.md
This conversation was marked as resolved.
Show resolved Hide resolved

# format formats the code (including imports format).
.PHONY: format
Expand Down
18 changes: 18 additions & 0 deletions docs/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Current object storage client implementations:
| Azure Storage Account | Stable (production usage) | yes | @vglafirov |
| OpenStack Swift | Beta (working PoCs, testing usage) | no | @sudhi-vm |
| Tencent COS | Beta (testing usage) | no | @jojohappy |
| AliYun OSS | Beta (testing usage) | no | @shaulboozhiao,@wujinhu |

NOTE: Currently Thanos requires strong consistency (write-read) for object store implementation.

Expand Down Expand Up @@ -308,3 +309,20 @@ config:
```

Set the flags `--objstore.config-file` to reference to the configuration file.

## AliYun OSS Configuration
In order to use AliYun OSS object storage, you should first create a bucket with proper Storage Class , ACLs and get the access key on the AliYun cloud. Go to [https://www.alibabacloud.com/product/oss](https://www.alibabacloud.com/product/oss) for more detail.

To use AliYun OSS object storage, please specify following yaml configuration file in `objstore.config*` flag.

[embedmd]:# (flags/config_aliyunoss.txt $)
```$
type: ALIYUNOSS
config:
endpoint: ""
bucket: ""
access_key_id: ""
access_key_secret: ""
```

Use --objstore.config-file to reference to this configuration file.
8 changes: 8 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ require (
cloud.google.com/go v0.34.0
github.com/Azure/azure-storage-blob-go v0.7.0
github.com/NYTimes/gziphandler v1.1.1
github.com/aliyun/aliyun-oss-go-sdk v1.9.8
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da
github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f // indirect
github.com/cespare/xxhash v1.1.0
github.com/fatih/structtag v1.0.0
github.com/fortytw2/leaktest v1.3.0
Expand All @@ -15,6 +17,7 @@ require (
github.com/google/martian v2.1.0+incompatible // indirect
github.com/googleapis/gax-go v2.0.2+incompatible // indirect
github.com/gophercloud/gophercloud v0.0.0-20190301152420-fca40860790e
github.com/gopherjs/gopherjs v0.0.0-20181103185306-d547d1d9531e // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.0.0
github.com/grpc-ecosystem/go-grpc-prometheus v0.0.0-20181025070259-68e3a13e4117
github.com/hashicorp/golang-lru v0.5.1
Expand All @@ -33,18 +36,23 @@ require (
github.com/pkg/errors v0.8.1
github.com/prometheus/client_golang v1.0.0
github.com/prometheus/common v0.6.0
github.com/prometheus/procfs v0.0.3 // indirect
github.com/prometheus/prometheus v2.9.2+incompatible
github.com/prometheus/tsdb v0.8.0
github.com/uber-go/atomic v1.4.0 // indirect
github.com/uber/jaeger-client-go v2.16.0+incompatible
github.com/uber/jaeger-lib v2.0.0+incompatible
go.uber.org/atomic v1.4.0 // indirect
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4 // indirect
golang.org/x/net v0.0.0-20190628185345-da137c7871d7 // indirect
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421
golang.org/x/sync v0.0.0-20190423024810-112230192c58
golang.org/x/sys v0.0.0-20190626221950-04f50cda93cb // indirect
golang.org/x/text v0.3.2
google.golang.org/api v0.3.2
google.golang.org/grpc v1.19.1
gopkg.in/alecthomas/kingpin.v2 v2.2.6
gopkg.in/fsnotify.v1 v1.4.7
gopkg.in/fsnotify/fsnotify.v1 v1.4.7 // indirect
gopkg.in/yaml.v2 v2.2.2
)
19 changes: 19 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,17 @@ github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc h1:cAKDfWh5Vpd
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf h1:qet1QNfXsQxTZqLG4oE62mJzwPIB8+Tee4RNCL9ulrY=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/aliyun/aliyun-oss-go-sdk v1.9.8 h1:BOflvK0Zs/zGmoabyFIzTg5c3kguktWTXEwewwbuba0=
github.com/aliyun/aliyun-oss-go-sdk v1.9.8/go.mod h1:T/Aws4fEfogEE9v+HPhhw+CntffsBHJ8nXQCwKr0/g8=
github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da h1:8GUt8eRujhVEGZFFEjBj46YV4rDjvGrNxb0KMWYkL2I=
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=
github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=
github.com/aws/aws-sdk-go v0.0.0-20180507225419-00862f899353 h1:qFKf58XUUvHaEz0zFkLJsQ4dzoAyrQ8QyhK4nHGHBI4=
github.com/aws/aws-sdk-go v0.0.0-20180507225419-00862f899353/go.mod h1:ZRmQr0FajVIyZ4ZzBYKG5P3ZqPz9IHG41ZoMu1ADI3k=
github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f h1:ZNv7On9kyUzm7fvRZumSyy/IUiSC7AzL0I1jKKtwooA=
github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f/go.mod h1:AuiFmCCPBSrqvVMvuqFuk0qogytodnVFVSN5CeJB8Gc=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973 h1:xJ4a3vCFaGF/jqvzLMYoU8P317H5OQ+Via4RmuPwCS0=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0 h1:HWo1m869IqiPhD389kmkxeTalrjNbbJTC8LXupb+sl0=
Expand Down Expand Up @@ -108,6 +112,8 @@ github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c h1:964Od4U6p2jUkFxvCy
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0 h1:+dTQ8DZQJz0Mb/HjFlkptS1FeQ4cWSnN941F8aEG4SQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/gofuzz v0.0.0-20150304233714-bbcb9da2d746 h1:M6d2zDTA4cKXT6OwFsJxlo5tWrAukj3KfvJ1zcBatnA=
Expand All @@ -123,6 +129,8 @@ github.com/gophercloud/gophercloud v0.0.0-20190301152420-fca40860790e h1:hQpY0g0
github.com/gophercloud/gophercloud v0.0.0-20190301152420-fca40860790e/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gopherjs/gopherjs v0.0.0-20181103185306-d547d1d9531e h1:JKmoR8x90Iww1ks85zJ1lfDGgIiMDuIptTOhJq+zKyg=
github.com/gopherjs/gopherjs v0.0.0-20181103185306-d547d1d9531e/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 h1:pdN6V1QBWetyv/0+wjACpqVH+eVULgEjkurDLq3goeM=
Expand Down Expand Up @@ -284,6 +292,8 @@ github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1 h1:/K3IL0Z1quvmJ
github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.2 h1:6LJUbpNm42llc4HRCuvApCSWB/WfhuNo9K98Q9sNGfs=
github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/procfs v0.0.3 h1:CTwfnzjQ+8dS6MhHHu4YswVAD99sL2wjPqP+VkURmKE=
github.com/prometheus/procfs v0.0.3/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ=
github.com/prometheus/prometheus v2.9.2+incompatible h1:5QVnXpkSsbbG59TyZ99clRfaHQy2QuIlTv6dEgS66C4=
github.com/prometheus/prometheus v2.9.2+incompatible/go.mod h1:vdLuLLM0uqhLSofrQ7Nev2b/rQUyZ+pkT3zF7LB/i1g=
github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
Expand All @@ -298,6 +308,7 @@ github.com/ryanuber/columnize v2.1.0+incompatible/go.mod h1:sm1tb6uqfes/u+d4ooFo
github.com/samuel/go-zookeeper v0.0.0-20161028232340-1d7be4effb13 h1:4AQBn5RJY4WH8t8TLEMZUsWeXHAUcoao42TCAfpEJJE=
github.com/samuel/go-zookeeper v0.0.0-20161028232340-1d7be4effb13/go.mod h1:gi+0XIa01GRL2eRQVjQkKGqKF3SF9vZR/HnPullcV2E=
github.com/sasha-s/go-deadlock v0.0.0-20161201235124-341000892f3d/go.mod h1:StQn567HiB1fF2yJ44N9au7wOhrPS3iZqiDbRupzT10=
github.com/satori/go.uuid v1.2.0 h1:0uYX9dsZ2yD7q2RtLRtPSdGDWzjeM3TbMJP9utgA0ww=
github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=
github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24/go.mod h1:M+9NzErvs504Cn4c5DxATwIqPbtswREoFCre64PpcG4=
Expand Down Expand Up @@ -337,6 +348,8 @@ golang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnf
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190513172903-22d7a77e9e5f h1:R423Cnkcp5JABoeemiGEPlt9tHXFfw5kvc0yqlxRPWo=
golang.org/x/crypto v0.0.0-20190513172903-22d7a77e9e5f/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4 h1:HuIa8hRrWRSrqYzx1qI49NNxhdi2PrY7gxVSq1JjLDc=
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
Expand All @@ -357,6 +370,8 @@ golang.org/x/net v0.0.0-20190522155817-f3200d17e092 h1:4QSRKanuywn15aTZvI/mIDEgP
golang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190613194153-d28f0bde5980 h1:dfGZHvZk057jK2MCeWus/TowKpJ8y4AmooUzdBSR9GU=
golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190628185345-da137c7871d7 h1:rTIdg5QFRR7XCaK4LCjBiPbx8j4DQRpdYMnGn/bJUEU=
golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421 h1:Wo7BWFiOk0QRFMLYMqJGFMd9CgUAcGx7V+qEg/h5IBI=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
Expand All @@ -382,6 +397,8 @@ golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894 h1:Cz4ceDQGXuKRnVBDTS23GTn/pU5OE2C0WrNTOYK1Uuc=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190626221950-04f50cda93cb h1:fgwFCsaw9buMuxNd6+DQfAuSFqbNiQZpcgJQAgJsK6k=
golang.org/x/sys v0.0.0-20190626221950-04f50cda93cb/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
Expand Down Expand Up @@ -416,6 +433,8 @@ gopkg.in/fsnotify.v1 v1.4.7 h1:xOHLXZwVvI9hhs+cLKq5+I5onOuwQLhQwiu63xxlHs4=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/fsnotify/fsnotify.v1 v1.3.1 h1:2fkCHbPQZNYRAyRyIV9VX0bpRkxIorlQDiYRmufHnhA=
gopkg.in/fsnotify/fsnotify.v1 v1.3.1/go.mod h1:Fyux9zXlo4rWoMSIzpn9fDAYjalPqJ/K1qJ27s+7ltE=
gopkg.in/fsnotify/fsnotify.v1 v1.4.7 h1:XNNYLJHt73EyYiCZi6+xjupS9CpvmiDgjPTAjrBlQbo=
gopkg.in/fsnotify/fsnotify.v1 v1.4.7/go.mod h1:Fyux9zXlo4rWoMSIzpn9fDAYjalPqJ/K1qJ27s+7ltE=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/ini.v1 v1.42.0 h1:7N3gPTt50s8GuLortA00n8AqRTk75qOP98+mTPpgzRk=
Expand Down
14 changes: 9 additions & 5 deletions pkg/objstore/client/factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
"github.com/thanos-io/thanos/pkg/objstore/azure"
"github.com/thanos-io/thanos/pkg/objstore/cos"
"github.com/thanos-io/thanos/pkg/objstore/gcs"
"github.com/thanos-io/thanos/pkg/objstore/oss"
"github.com/thanos-io/thanos/pkg/objstore/s3"
"github.com/thanos-io/thanos/pkg/objstore/swift"
yaml "gopkg.in/yaml.v2"
Expand All @@ -21,11 +22,12 @@ import (
type ObjProvider string

const (
GCS ObjProvider = "GCS"
S3 ObjProvider = "S3"
AZURE ObjProvider = "AZURE"
SWIFT ObjProvider = "SWIFT"
COS ObjProvider = "COS"
GCS ObjProvider = "GCS"
S3 ObjProvider = "S3"
AZURE ObjProvider = "AZURE"
SWIFT ObjProvider = "SWIFT"
COS ObjProvider = "COS"
ALIYUNOSS ObjProvider = "ALIYUNOSS"
)

type BucketConfig struct {
Expand Down Expand Up @@ -59,6 +61,8 @@ func NewBucket(logger log.Logger, confContentYaml []byte, reg prometheus.Registe
bucket, err = swift.NewContainer(logger, config)
case string(COS):
bucket, err = cos.NewBucket(logger, config, component)
case string(ALIYUNOSS):
bucket, err = oss.NewBucket(logger, config, component)
default:
return nil, errors.Errorf("bucket with type %s is not supported", bucketConf.Type)
}
Expand Down
17 changes: 17 additions & 0 deletions pkg/objstore/objtesting/foreach.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"github.com/thanos-io/thanos/pkg/objstore/cos"
"github.com/thanos-io/thanos/pkg/objstore/gcs"
"github.com/thanos-io/thanos/pkg/objstore/inmem"
"github.com/thanos-io/thanos/pkg/objstore/oss"
"github.com/thanos-io/thanos/pkg/objstore/s3"
"github.com/thanos-io/thanos/pkg/objstore/swift"
"github.com/thanos-io/thanos/pkg/testutil"
Expand Down Expand Up @@ -117,4 +118,20 @@ func ForeachStore(t *testing.T, testFn func(t testing.TB, bkt objstore.Bucket))
} else {
t.Log("THANOS_SKIP_TENCENT_COS_TESTS envvar present. Skipping test against Tencent COS.")
}

// Optional OSS.
if _, ok := os.LookupEnv("THANOS_SKIP_ALIYUN_OSS_TESTS"); !ok {
bkt, closeFn, err := oss.NewTestBucket(t)
testutil.Ok(t, err)

ok := t.Run("AliYun oss", func(t *testing.T) {
testFn(t, bkt)
})
closeFn()
if !ok {
return
}
} else {
t.Log("THANOS_SKIP_ALIYUN_OSS_TESTS envvar present. Skipping test against AliYun OSS.")
}
}
Loading