Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Commit

Permalink
GC: Beta Client Package (#2205)
Browse files Browse the repository at this point in the history
## Description
To add needed features, Kiota-Generated Beta Connector created
<!-- Insert PR description-->

## Does this PR need a docs update or release note?

- [x] ✅ Yes, it's included


## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes  #2086<issue>
* closes #2174 
* closes #2223

## Test Plan
Compilation and current CI tests will fail if the package does not align with current operational standards. 
- [x] ⚡ Unit test
- [x] 💚 E2E
  • Loading branch information
Danny authored Jan 31, 2023
1 parent 14be75e commit 3a52115
Show file tree
Hide file tree
Showing 105 changed files with 9,803 additions and 252 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ jobs:
working-directory: src
steps:
- uses: actions/checkout@v3

# single setup and sum cache handling here.
# the results will cascade onto both testing and linting.
- name: Setup Golang with cache
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed

- Check if the user specified for an exchange backup operation has a mailbox.

### Changed

- BetaClient introduced. Enables Corso to be able to interact with SharePoint Page objects. Package located `/internal/connector/graph/betasdk`
- Handle case where user's drive has not been initialized
- Inline attachments (e.g. copy/paste ) are discovered and backed up correctly ([#2163](https://github.com/alcionai/corso/issues/2163))
- Guest and External users (for cloud accounts) and non-on-premise users (for systems that use on-prem AD syncs) are now excluded from backup and restore operations.
Expand Down
12 changes: 11 additions & 1 deletion src/.golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ run:

linters:
enable:
- gci
- gofmt
- gofumpt
- errcheck
Expand Down Expand Up @@ -106,3 +105,14 @@ issues:
linters:
- forbidigo
text: "context.(Background|TODO)"
- path: internal/connector/graph/betasdk
linters:
- wsl
- revive
- gci
- lll
- gofmt
- gofumpt
- misspell
- errcheck

14 changes: 7 additions & 7 deletions src/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ require (
github.com/google/uuid v1.3.0
github.com/hashicorp/go-multierror v1.1.1
github.com/kopia/kopia v0.12.2-0.20230123092305-e5387cec0acb
github.com/microsoft/kiota-abstractions-go v0.15.2
github.com/microsoft/kiota-authentication-azure-go v0.5.0
github.com/microsoft/kiota-http-go v0.11.0
github.com/microsoft/kiota-abstractions-go v0.16.0
github.com/microsoft/kiota-authentication-azure-go v0.6.0
github.com/microsoft/kiota-http-go v0.13.0
github.com/microsoft/kiota-serialization-json-go v0.7.2
github.com/microsoftgraph/msgraph-sdk-go v0.50.0
github.com/microsoftgraph/msgraph-sdk-go-core v0.31.1
github.com/microsoftgraph/msgraph-sdk-go v0.53.0
github.com/microsoftgraph/msgraph-sdk-go-core v0.33.0
github.com/pkg/errors v0.9.1
github.com/rudderlabs/analytics-go v3.3.3+incompatible
github.com/spatialcurrent/go-lazy v0.0.0-20211115014721-47315cc003d1
Expand Down Expand Up @@ -53,8 +53,8 @@ require (
)

require (
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.2.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.0.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.3.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.1.2 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v0.7.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869 // indirect
Expand Down
28 changes: 14 additions & 14 deletions src/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,12 @@ cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RX
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
cloud.google.com/go/storage v1.14.0/go.mod h1:GrKmX003DSIwi9o29oFT7YDnHYwZoctc3fOKtUw0Xmo=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.2.0 h1:sVW/AFBTGyJxDaMYlq0ct3jUXTtj12tQ6zE2GZUgVQw=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.2.0/go.mod h1:uGG2W01BaETf0Ozp+QxxKJdMBNRWPdstHG0Fmdwn1/U=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.3.0 h1:VuHAcMq8pU1IWNT/m5yRaGqbK0BiQKHT8X4DTp9CHdI=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.3.0/go.mod h1:tZoQYdDZNOiIjdSn0dVWVfl0NEPGOJqVLzSrcFk4Is0=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.2.0 h1:t/W5MYAuQy81cvM8VUNfRLzhtKpXhVUAN7Cd7KVbTyc=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.2.0/go.mod h1:NBanQUfSWiWn3QEpWDTCU0IjBECKOYvl2R8xdRtMtiM=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.0.1 h1:XUNQ4mw+zJmaA2KXzP9JlQiecy1SI+Eog7xVkPiqIbg=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.0.1/go.mod h1:eWRD7oawr1Mu1sLCawqVc0CUiF43ia3qQMxLscsKQ9w=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.1.2 h1:+5VZ72z0Qan5Bog5C+ZkgSqUbeVUd9wgtHOrIKuc5b8=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.1.2/go.mod h1:eWRD7oawr1Mu1sLCawqVc0CUiF43ia3qQMxLscsKQ9w=
github.com/AzureAD/microsoft-authentication-library-for-go v0.7.0 h1:VgSJlZH5u0k2qxSpqyghcFQKmvYckj46uymKK5XzkBM=
github.com/AzureAD/microsoft-authentication-library-for-go v0.7.0/go.mod h1:BDJ5qMFKx9DugEg3+uQSDCdbYPr5s9vBTrL9P8TpqOU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
Expand Down Expand Up @@ -269,22 +269,22 @@ github.com/matttproud/golang_protobuf_extensions v1.0.2 h1:hAHbPm5IJGijwng3PWk09
github.com/matttproud/golang_protobuf_extensions v1.0.2/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d h1:5PJl274Y63IEHC+7izoQE9x6ikvDFZS2mDVS3drnohI=
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
github.com/microsoft/kiota-abstractions-go v0.15.2 h1:Pp78BbqPvkF2mAMH0Ph37ymwfSH7uF9iYfY1fZ8g630=
github.com/microsoft/kiota-abstractions-go v0.15.2/go.mod h1:RT/s9sCzg49i4iO7e2qhyWmX+DlJDgC0P+Wp8fKQQfo=
github.com/microsoft/kiota-authentication-azure-go v0.5.0 h1:RVA/tTgMnDIN3u4qPZtvYvVRsQDOFkd3yvi6KXjZJko=
github.com/microsoft/kiota-authentication-azure-go v0.5.0/go.mod h1:1Io6h+88FlDRmrajdjSnXPz8oyObUVjNuQZLhrF9kQk=
github.com/microsoft/kiota-http-go v0.11.0 h1:0K0y/wZcTvEEX2Xdj5tngJqknqYQpArLdtjB/fo88Dc=
github.com/microsoft/kiota-http-go v0.11.0/go.mod h1:4D6vMjT7jQ3IRAJrpFoaDtfS9eMaSjQKHY6ETSV2cc0=
github.com/microsoft/kiota-abstractions-go v0.16.0 h1:DZ1L4YsRsQw39iPGnVq2fQkqLXMsazdPwmWsnaH4EZg=
github.com/microsoft/kiota-abstractions-go v0.16.0/go.mod h1:RT/s9sCzg49i4iO7e2qhyWmX+DlJDgC0P+Wp8fKQQfo=
github.com/microsoft/kiota-authentication-azure-go v0.6.0 h1:Il9bLO34J6D8DY89xYAXoGh9muvlphayqG4eihyT6B8=
github.com/microsoft/kiota-authentication-azure-go v0.6.0/go.mod h1:EJCHiLWLXW1/mSgX7lYReAhVO37MzRT5Xi2mcPTwCRQ=
github.com/microsoft/kiota-http-go v0.13.0 h1:CZSC+UrBSwjIvLlVh+AEPsDbD0c17AWd/QPRHIljd8k=
github.com/microsoft/kiota-http-go v0.13.0/go.mod h1:aWtBlFhCetH0JmouvN3hiSaJoqCEdVHPqlrcLYExs3k=
github.com/microsoft/kiota-serialization-form-go v0.2.0 h1:jgPE+8DtrWhL+KwnAwRm13HnMNydbelC/NP9wRGwDUo=
github.com/microsoft/kiota-serialization-form-go v0.2.0/go.mod h1:chOuh09tO7IrNtubAumdlG5wnrcYdMkjV7joVVGDyGs=
github.com/microsoft/kiota-serialization-json-go v0.7.2 h1:DSb4fNDi5O+DqJwrHo+vRy2kSvfxG5VtN6m1EHzn5Vw=
github.com/microsoft/kiota-serialization-json-go v0.7.2/go.mod h1:Ojum5prlijopyCOZ2XctRcVlE2pU8h+43r3tMdiWoDU=
github.com/microsoft/kiota-serialization-text-go v0.6.0 h1:3N2vftYZlwKdog69AN7ha+FZT0QxPG7xp/hLv0/W2OQ=
github.com/microsoft/kiota-serialization-text-go v0.6.0/go.mod h1:OUA4dNH+f6afiJUs+rQAatJos7QVF5PJkyrqoD89lx4=
github.com/microsoftgraph/msgraph-sdk-go v0.50.0 h1:yfPBDr7+tSdq8jKiNCvY5XzQji1kZzOHXvxQ9XxQ4E4=
github.com/microsoftgraph/msgraph-sdk-go v0.50.0/go.mod h1:XoTT9lzRSersVV4/lsFup3sOLfOLEf2dMsLckiAAKq8=
github.com/microsoftgraph/msgraph-sdk-go-core v0.31.1 h1:aVvnO5l8qLCEcvELc5n9grt7UXhAVtpog1QeQKLMlTE=
github.com/microsoftgraph/msgraph-sdk-go-core v0.31.1/go.mod h1:RE4F2qGCTehGtQGc9Txafc4l+XMpbjYuO4amDLFgOWE=
github.com/microsoftgraph/msgraph-sdk-go v0.53.0 h1:HpQd1Nvr8yQNeqhDuiVSbqn1fkHsFbRFDmnuhhXJXOQ=
github.com/microsoftgraph/msgraph-sdk-go v0.53.0/go.mod h1:BZLyon4n4T4EuLIAlX+kJ5JgneFTXVQDah1AJuq3FRY=
github.com/microsoftgraph/msgraph-sdk-go-core v0.33.0 h1:cDL3ov/IZ2ZarUJdGGPsdR+46ALdd3CRAiDBIylLCoA=
github.com/microsoftgraph/msgraph-sdk-go-core v0.33.0/go.mod h1:d0mU3PQAWnN/C4CwPJEZz2QhesrnR5UDnqRu2ODWPkI=
github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=
github.com/minio/minio-go/v7 v7.0.45 h1:g4IeM9M9pW/Lo8AGGNOjBZYlvmtlE1N5TQEYWXRWzIs=
Expand Down
86 changes: 86 additions & 0 deletions src/internal/connector/graph/betasdk/beta_client.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
package betasdk

import (
i1a3c1a5501c5e41b7fd169f2d4c768dce9b096ac28fb5431bf02afcc57295411 "github.com/alcionai/corso/src/internal/connector/graph/betasdk/sites"
absser "github.com/microsoft/kiota-abstractions-go"
kioser "github.com/microsoft/kiota-abstractions-go/serialization"
kform "github.com/microsoft/kiota-serialization-form-go"
kw "github.com/microsoft/kiota-serialization-json-go"
ktext "github.com/microsoft/kiota-serialization-text-go"
msgraphsdk "github.com/microsoftgraph/msgraph-sdk-go"
)

// BetaClient the main entry point of the SDK, exposes the configuration and the fluent API.
// Minimal Beta Connector:
// Details on how the Code was generated is present in `kioter-lock.json`.
// NOTE: kiota gen file is altered to indicate what files are included in the created
//
// Changes to Sites Directory:
// Access files send requests with an adapter's with ASync() support.
// This feature is not enabled in v1.0. Manually changed in remaining files.
// Additionally, only calls that begin as client.SitesBy(siteID).Pages() have an endpoint.
//
// The use case specific to Pages(). All other requests should be routed to the /internal/connector/graph.Servicer
// Specifics on `betaClient.SitesById(siteID).Pages` are located: sites/site_item_request_builder.go
//
// Required model files are identified as `modelFiles` in kiota-lock.json. Directory -> betasdk/models
// Required access files are identified as `sitesFiles` in kiota-lock.json. Directory -> betasdk/sites
//
// BetaClient minimal msgraph-beta-sdk-go for connecting to msgraph-beta-sdk-go
// for retrieving `SharePoint.Pages`. Code is generated from kiota.dev.
// requestAdapter is registered with the following the serializers:
// -- "Microsoft.Kiota.Serialization.Json.JsonParseNodeFactory",
// -- "Microsoft.Kiota.Serialization.Text.TextParseNodeFactory",
// -- "Microsoft.Kiota.Serialization.Form.FormParseNodeFactory"
type BetaClient struct {
// Path parameters for the request
pathParameters map[string]string
// The request adapter to use to execute the requests.
requestAdapter *msgraphsdk.GraphRequestAdapter
// Url template to use to build the URL for the current request builder
urlTemplate string
}

// NewBetaClient instantiates a new BetaClient and sets the default values.
// func NewBetaClient(requestAdapter i2ae4187f7daee263371cb1c977df639813ab50ffa529013b7437480d1ec0158f.RequestAdapter)(*BetaClient) {
func NewBetaClient(requestAdapter *msgraphsdk.GraphRequestAdapter) *BetaClient {
m := &BetaClient{}
m.pathParameters = make(map[string]string)
m.urlTemplate = "{+baseurl}"
m.requestAdapter = requestAdapter
absser.RegisterDefaultSerializer(func() kioser.SerializationWriterFactory {
return kw.NewJsonSerializationWriterFactory()
})
absser.RegisterDefaultSerializer(func() kioser.SerializationWriterFactory {
return ktext.NewTextSerializationWriterFactory()
})
absser.RegisterDefaultSerializer(func() kioser.SerializationWriterFactory {
return kform.NewFormSerializationWriterFactory()
})
absser.RegisterDefaultDeserializer(func() kioser.ParseNodeFactory {
return kw.NewJsonParseNodeFactory()
})
absser.RegisterDefaultDeserializer(func() kioser.ParseNodeFactory {
return ktext.NewTextParseNodeFactory()
})
absser.RegisterDefaultDeserializer(func() kioser.ParseNodeFactory {
return kform.NewFormParseNodeFactory()
})

if m.requestAdapter.GetBaseUrl() == "" {
m.requestAdapter.SetBaseUrl("https://graph.microsoft.com/beta")
}
return m
}

// SitesById provides operations to manage the collection of site entities.
func (m *BetaClient) SitesById(id string) *i1a3c1a5501c5e41b7fd169f2d4c768dce9b096ac28fb5431bf02afcc57295411.SiteItemRequestBuilder {
urlTplParams := make(map[string]string)
for idx, item := range m.pathParameters {
urlTplParams[idx] = item
}
if id != "" {
urlTplParams["site%2Did"] = id
}
return i1a3c1a5501c5e41b7fd169f2d4c768dce9b096ac28fb5431bf02afcc57295411.NewSiteItemRequestBuilderInternal(urlTplParams, m.requestAdapter)
}
80 changes: 80 additions & 0 deletions src/internal/connector/graph/betasdk/beta_client_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
package betasdk

import (
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"

"github.com/alcionai/corso/src/internal/connector/graph"
"github.com/alcionai/corso/src/internal/tester"
"github.com/alcionai/corso/src/pkg/account"
)

type BetaClientSuite struct {
suite.Suite
credentials account.M365Config
}

func TestBetaClientSuite(t *testing.T) {
suite.Run(t, new(BetaClientSuite))
}

func (suite *BetaClientSuite) SetupSuite() {
t := suite.T()
a := tester.NewM365Account(t)
m365, err := a.M365Config()
require.NoError(t, err)

suite.credentials = m365
}

func (suite *BetaClientSuite) TestCreateBetaClient() {
t := suite.T()
adpt, err := graph.CreateAdapter(
suite.credentials.AzureTenantID,
suite.credentials.AzureClientID,
suite.credentials.AzureClientSecret,
)

require.NoError(t, err)

client := NewBetaClient(adpt)
assert.NotNil(t, client)
}

// TestBasicClientGetFunctionality. Tests that adapter is able
// to parse retrieved Site Page. Additional tests should
// be handled within the /internal/connector/sharepoint when
// additional features are added.
func (suite *BetaClientSuite) TestBasicClientGetFunctionality() {
ctx, flush := tester.NewContext()
defer flush()
t := suite.T()
adpt, err := graph.CreateAdapter(
suite.credentials.AzureTenantID,
suite.credentials.AzureClientID,
suite.credentials.AzureClientSecret,
)

require.NoError(t, err)
client := NewBetaClient(adpt)
require.NotNil(t, client)

siteID := tester.M365SiteID(t)
// TODO(dadams39) document allowable calls in main
collection, err := client.SitesById(siteID).Pages().Get(ctx, nil)
// Ensures that the client is able to receive data from beta
// Not Registered Error: content type application/json does not have a factory registered to be parsed
require.NoError(t, err)

for _, page := range collection.GetValue() {
assert.NotNil(t, page, "betasdk call for page does not return value.")

if page != nil {
t.Logf("Page :%s ", *page.GetName())
assert.NotNil(t, page.GetId())
}
}
}
Loading

0 comments on commit 3a52115

Please sign in to comment.