Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add integration with: Sharepoint #693

Merged
merged 9 commits into from
Sep 13, 2024
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,17 @@ export class SharepointConnectionService extends AbstractBaseConnectionService {
'OAuth credentials : sharepoint filestorage ' + JSON.stringify(data),
);

// get site_id from tenant and sitename
const site_details = await axios.get(
`https://graph.microsoft.com/v1.0/sites/${tenant}.sharepoint.com:/sites/${site}`,
{
headers: {
Authorization: `Bearer ${data.access_token}`,
},
},
);
const site_id = site_details.data.id;
Comment on lines +142 to +151
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve the changes with a suggestion to add error handling.

The code changes introduce a new mechanism to dynamically fetch the site_id using the Microsoft Graph API, which is an improvement over using the site parameter directly. This change aligns with the best practices for accessing SharePoint resources by using the correct identifiers in API calls.

The axios library is being used to make the API call and the access_token from the OAuth response is being used for authentication, which looks good.

Consider adding error handling for the API call to gracefully handle any errors that may occur during the request. You can use a try-catch block to catch any errors and log them using the logger service.


let db_res;
const connection_token = uuidv4();

Expand All @@ -153,7 +164,7 @@ export class SharepointConnectionService extends AbstractBaseConnectionService {
account_url: (
CONNECTORS_METADATA['filestorage']['sharepoint'].urls
.apiUrl as DynamicApiUrl
)(site),
)(site_id),
expiration_timestamp: new Date(
new Date().getTime() + Number(data.expires_in) * 1000,
),
Expand All @@ -172,7 +183,7 @@ export class SharepointConnectionService extends AbstractBaseConnectionService {
account_url: (
CONNECTORS_METADATA['filestorage']['sharepoint'].urls
.apiUrl as DynamicApiUrl
)(site),
)(site_id),
access_token: this.cryptoService.encrypt(data.access_token),
refresh_token: this.cryptoService.encrypt(data.refresh_token),
expiration_timestamp: new Date(
Expand Down
1 change: 1 addition & 0 deletions packages/api/src/@core/sync/sync.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -546,6 +546,7 @@ export class CoreSyncService {
try {
await task();
} catch (error) {
console.log(error);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider removing console.log for error logging in production.

Using console.log can lead to performance issues and does not align with best practices for error handling in production environments. It is recommended to rely on the existing logger.error method, which integrates with the application's centralized logging system.

this.logger.error(`File Storage Task failed: ${error.message}`, error);
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
import {
BoxSharedLinkInput,
BoxSharedLinkOutput,
} from '@filestorage/sharedlink/services/box/types';
/* INPUT */

import {
OnedriveSharedLinkInput,
OnedriveSharedLinkOutput,
Expand Down Expand Up @@ -37,7 +35,40 @@ import {
OnedriveDriveOutput,
} from '@filestorage/drive/services/onedrive/types';

/* INPUT */
import {
SharepointSharedLinkInput,
SharepointSharedLinkOutput,
} from '@filestorage/sharedlink/services/sharepoint/types';

import {
SharepointPermissionInput,
SharepointPermissionOutput,
} from '@filestorage/permission/services/sharepoint/types';

import {
SharepointGroupInput,
SharepointGroupOutput,
} from '@filestorage/group/services/sharepoint/types';

import {
SharepointUserInput,
SharepointUserOutput,
} from '@filestorage/user/services/sharepoint/types';

import {
SharepointFolderInput,
SharepointFolderOutput,
} from '@filestorage/folder/services/sharepoint/types';

import {
SharepointFileInput,
SharepointFileOutput,
} from '@filestorage/file/services/sharepoint/types';

import {
SharepointDriveInput,
SharepointDriveOutput,
} from '@filestorage/drive/services/sharepoint/types';

import {
BoxFileInput,
Expand All @@ -56,6 +87,9 @@ import {
BoxUserOutput,
} from '@filestorage/user/services/box/types';
import {
BoxSharedLinkInput,
BoxSharedLinkOutput,
} from '@filestorage/sharedlink/services/box/types';
GoogleDriveFileInput,
GoogleDriveFileOutput,
} from '@filestorage/file/services/googledrive/types';
Expand All @@ -72,28 +106,39 @@ import {
export type OriginalFileInput =
| BoxFileInput
| OnedriveFileInput
| SharepointFileInput;
| GoogleDriveFileInput;

/* folder */
export type OriginalFolderInput =
| BoxFolderInput
| OnedriveFolderInput
| SharepointFolderInput;
| GoogleDriveFolderInput;

/* permission */
export type OriginalPermissionInput = any | OnedrivePermissionInput;
export type OriginalPermissionInput =
| any
| OnedrivePermissionInput
| SharepointPermissionInput;

Comment on lines +120 to +123
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider removing the any type from the union.

The addition of SharepointPermissionInput to the OriginalPermissionInput union type is consistent with the overall objective of integrating SharePoint types into the existing type definitions. It allows for a unified approach to handling permission inputs from different providers.

However, the presence of the any type in the union may lead to a lack of type safety. Consider removing the any type and replacing it with a more specific type or creating a separate type for the cases where any is currently used.

/* shared link */
export type OriginalSharedLinkInput = any;

/* drive */
export type OriginalDriveInput = GoogleDriveDriveInput | OnedriveDriveInput;
export type OriginalDriveInput = GoogleDriveDriveInput | OnedriveDriveInput | SharepointDriveInput;

/* group */
export type OriginalGroupInput = BoxGroupInput | OnedriveGroupInput;
export type OriginalGroupInput =
| BoxGroupInput
| OnedriveGroupInput
| SharepointGroupInput;

/* user */
export type OriginalUserInput = BoxUserInput | OnedriveUserInput;
export type OriginalUserInput =
| BoxUserInput
| OnedriveUserInput
| SharepointUserInput;

export type FileStorageObjectInput =
| OriginalFileInput
Expand All @@ -110,28 +155,39 @@ export type FileStorageObjectInput =
export type OriginalFileOutput =
| BoxFileOutput
| OnedriveFileOutput
| SharepointFileOutput;
| GoogleDriveFileOutput;

/* folder */
export type OriginalFolderOutput =
| BoxFolderOutput
| OnedriveFolderOutput
| SharepointFolderOutput;
| GoogleDriveFolderOutput;

/* permission */
export type OriginalPermissionOutput = any | OnedrivePermissionOutput;
export type OriginalPermissionOutput =
| any
| OnedrivePermissionOutput
| SharepointPermissionOutput;

Comment on lines +169 to +172
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider removing the any type from the union.

The addition of SharepointPermissionOutput to the OriginalPermissionOutput union type is consistent with the overall objective of integrating SharePoint types into the existing type definitions. It allows for a unified approach to handling permission outputs from different providers.

However, the presence of the any type in the union may lead to a lack of type safety. Consider removing the any type and replacing it with a more specific type or creating a separate type for the cases where any is currently used.

/* shared link */
export type OriginalSharedLinkOutput = any;

/* drive */
export type OriginalDriveOutput = GoogleDriveDriveOutput | OnedriveDriveOutput;
export type OriginalDriveOutput = GoogleDriveDriveOutput | OnedriveDriveOutput | SharepointDriveOutput;

/* group */
export type OriginalGroupOutput = BoxGroupOutput | OnedriveGroupOutput;
export type OriginalGroupOutput =
| BoxGroupOutput
| OnedriveGroupOutput
| SharepointGroupOutput;

/* user */
export type OriginalUserOutput = BoxUserOutput | OnedriveUserOutput;
export type OriginalUserOutput =
| BoxUserOutput
| OnedriveUserOutput
| SharepointUserOutput;

export type FileStorageObjectOutput =
| OriginalFileOutput
Expand All @@ -144,8 +200,10 @@ export type FileStorageObjectOutput =

export type OriginalSharedlinkInput =
| BoxSharedLinkInput
| OnedriveSharedLinkInput;
| OnedriveSharedLinkInput
| SharepointSharedLinkInput;

export type OriginalSharedlinkOutput =
| BoxSharedLinkOutput
| OnedriveSharedLinkOutput;
| OnedriveSharedLinkOutput
| SharepointSharedLinkOutput;
7 changes: 7 additions & 0 deletions packages/api/src/filestorage/drive/drive.module.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
import { OnedriveDriveMapper } from './services/onedrive/mappers';
import { OnedriveService } from './services/onedrive';
import { SharepointDriveMapper } from './services/sharepoint/mappers';
import { SharepointService } from './services/sharepoint';
import { BullQueueModule } from '@@core/@core-services/queues/queue.module';
import { IngestDataService } from '@@core/@core-services/unification/ingest-data.service';
import { WebhookService } from '@@core/@core-services/webhooks/panora-webhooks/webhook.service';
import { Utils } from '@filestorage/@lib/@utils';
Expand All @@ -24,6 +29,8 @@ import { SyncService } from './sync/sync.service';
GoogleDriveService,
GoogleDriveMapper,
OnedriveDriveMapper,
SharepointService,
SharepointDriveMapper,
],
exports: [SyncService],
})
Expand Down
71 changes: 71 additions & 0 deletions packages/api/src/filestorage/drive/services/sharepoint/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
import { EncryptionService } from '@@core/@core-services/encryption/encryption.service';
import { LoggerService } from '@@core/@core-services/logger/logger.service';
import { PrismaService } from '@@core/@core-services/prisma/prisma.service';
import { ApiResponse } from '@@core/utils/types';
import { SyncParam } from '@@core/utils/types/interface';
import { FileStorageObject } from '@filestorage/@lib/@types';
import { IDriveService } from '@filestorage/drive/types';
import { Injectable } from '@nestjs/common';
import axios from 'axios';
import { ServiceRegistry } from '../registry.service';
import { SharepointDriveOutput } from './types';
import { DesunifyReturnType } from '@@core/utils/types/desunify.input';
import { OriginalDriveOutput } from '@@core/utils/types/original/original.file-storage';

@Injectable()
export class SharepointService implements IDriveService {
constructor(
private prisma: PrismaService,
private logger: LoggerService,
private cryptoService: EncryptionService,
private registry: ServiceRegistry,
) {
this.logger.setContext(
`${FileStorageObject.file.toUpperCase()}:${SharepointService.name}`,
);
this.registry.registerService('sharepoint', this);
}

async addDrive(
driveData: DesunifyReturnType,
linkedUserId: string,
): Promise<ApiResponse<OriginalDriveOutput>> {
// No API to add drive in Sharepoint
return;
}
Comment on lines +29 to +35
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarify the stub for addDrive.

The addDrive method is a stub with a comment indicating no API for adding drives in SharePoint. If this is accurate, consider adding more detail or documentation to explain how this situation is handled or if alternative methods are used.


async sync(data: SyncParam): Promise<ApiResponse<SharepointDriveOutput[]>> {
try {
const { linkedUserId } = data;

const connection = await this.prisma.connections.findFirst({
where: {
id_linked_user: linkedUserId,
provider_slug: 'sharepoint',
vertical: 'filestorage',
},
});

const resp = await axios.get(`${connection.account_url}/drives`, {
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${this.cryptoService.decrypt(
connection.access_token,
)}`,
},
});

const drives: SharepointDriveOutput[] = resp.data.value;
this.logger.log(`Synced sharepoint drives !`);

return {
data: drives,
message: 'Sharepoint drives retrived',
statusCode: 200,
};
} catch (error) {
console.log(error.response);
throw error;
}
}
}
86 changes: 86 additions & 0 deletions packages/api/src/filestorage/drive/services/sharepoint/mappers.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
import { MappersRegistry } from '@@core/@core-services/registries/mappers.registry';
import { CoreUnification } from '@@core/@core-services/unification/core-unification.service';
import { Utils } from '@filestorage/@lib/@utils';
import {
UnifiedFilestorageDriveInput,
UnifiedFilestorageDriveOutput,
} from '@filestorage/drive/types/model.unified';
import { Injectable } from '@nestjs/common';
import { SharepointDriveInput, SharepointDriveOutput } from './types';
import { IDriveMapper } from '@filestorage/drive/types';

@Injectable()
export class SharepointDriveMapper implements IDriveMapper {
constructor(
private mappersRegistry: MappersRegistry,
private utils: Utils,
private coreUnificationService: CoreUnification,
) {
this.mappersRegistry.registerService(
'filestorage',
'drive',
'sharepoint',
this,
);
}

async desunify(
source: UnifiedFilestorageDriveInput,
customFieldMappings?: {
slug: string;
remote_id: string;
}[],
): Promise<SharepointDriveInput> {
return;
Comment on lines +27 to +34
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incomplete implementation of desunify method.

The desunify method lacks implementation, which might be intentional as a placeholder or an oversight. Please confirm if this method is supposed to be implemented later or if it was missed.

Would you like me to help draft the implementation or should this be tracked as a task in your project management tool?

}

async unify(
source: SharepointDriveOutput | SharepointDriveOutput[],
connectionId: string,
customFieldMappings?: {
slug: string;
remote_id: string;
}[],
): Promise<UnifiedFilestorageDriveOutput | UnifiedFilestorageDriveOutput[]> {
if (!Array.isArray(source)) {
return await this.mapSingleDriveToUnified(
source,
connectionId,
customFieldMappings,
);
}
// Handling array of SharepointDriveOutput
return Promise.all(
source.map((drive) =>
this.mapSingleDriveToUnified(drive, connectionId, customFieldMappings),
),
);
}

private async mapSingleDriveToUnified(
drive: SharepointDriveOutput,
connectionId: string,
customFieldMappings?: {
slug: string;
remote_id: string;
}[],
): Promise<UnifiedFilestorageDriveOutput> {
const field_mappings: { [key: string]: any } = {};
if (customFieldMappings) {
for (const mapping of customFieldMappings) {
field_mappings[mapping.slug] = drive[mapping.remote_id];
}
}

const result: UnifiedFilestorageDriveOutput = {
remote_id: drive.id,
remote_data: drive,
name: drive.name,
remote_created_at: drive.createdDateTime,
drive_url: drive.webUrl,
field_mappings,
};

return result;
}
}
Loading
Loading