Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gmail: update documentation #917

Merged
merged 7 commits into from
Jan 18, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/four-dots-tickle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@openfn/language-gmail': patch
---

Tweak docs
27 changes: 16 additions & 11 deletions packages/gmail/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,23 +16,22 @@ Gmail for messages and identify desired attachments and metadata.

## Usage

The adaptor's primary function is `getContentsFromMessages`
The adaptor's primary function is `getContentsFromMessages`:
josephjclark marked this conversation as resolved.
Show resolved Hide resolved

```js
getContentsFromMessages(userId, query, desiredContents, callback);
getContentsFromMessages(accountId, options);
```

1. Set userId with the Gmail account to query.
2. Customize the query to contain filters as needed. This is the same format a
the query in the Gmail UI.
1. Set accountId with the Gmail account to query.
2. Set a query in options in the same format used in the Gmail UI.
3. Specify what content to retrieve from messages (body, subject, attachments,
etc)

## Extracting Message Contents

The `desiredContents` array should list what content to retrieve from each
message. Each item should be a string (ie, `"body"` or an object describing an
attachment)
message. Each item should be a string (ie, `"body"`) or an object describing an
attachment.

### Metadata

Expand Down Expand Up @@ -82,7 +81,7 @@ The query variable is constructed to filter Gmail messages:
## Example

```js
const userId = 'tester@gmail.com';
const accountId = 'tester@gmail.com';

const querySubject = encodeURIComponent('device data summary');

Expand Down Expand Up @@ -116,9 +115,15 @@ const desiredContents = [
dataFile,
];

getContentsFromMessages(userId, query, desiredContents, state =>
console.log(state.data)
);
getContentsFromMessages(accountId, {
query,
desiredContents,
});

fn(state => {
console.log(state.data);
return state;
});
```

## Sample Output
Expand Down
18 changes: 7 additions & 11 deletions packages/gmail/ast.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
{
"name": "getContentsFromMessages",
"params": [
"userId",
"userOptions"
"accountId",
"options"
],
"docs": {
"description": "Used to isolate the type of content to retrieve from the message.",
Expand All @@ -16,7 +16,7 @@
"type": "NameExpression",
"name": "Object"
},
"name": "DesiredContent"
"name": "MessageContent"
},
{
"title": "public",
Expand All @@ -34,16 +34,12 @@
},
{
"title": "property",
"description": "A custom description for the content type. Optional.",
"description": "A custom description for the content type.",
"type": {
"type": "OptionalType",
"expression": {
"type": "NullableType",
"expression": {
"type": "NameExpression",
"name": "string"
},
"prefix": false
"type": "NameExpression",
"name": "string"
}
},
"name": "name",
Expand Down Expand Up @@ -93,7 +89,7 @@
},
{
"title": "property",
"description": "Maximum number of characters to retrieve from the content. Optional.",
"description": "Maximum number of characters to retrieve from the content.",
"type": {
"type": "OptionalType",
"expression": {
Expand Down
65 changes: 38 additions & 27 deletions packages/gmail/src/Adaptor.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,70 +19,81 @@ import {

/**
* Used to isolate the type of content to retrieve from the message.
* @typedef {Object} DesiredContent
* @typedef {Object} MessageContent
josephjclark marked this conversation as resolved.
Show resolved Hide resolved
* @public
* @property {string} type - Message content type. Valid types: from, date, subject, body, archive, file.
* @property {string?} [name=null] - A custom description for the content type. Optional.
* @property {string} [name] - A custom description for the content type.
josephjclark marked this conversation as resolved.
Show resolved Hide resolved
* @property {RegExp|string} [archive] - Identifier to isolate the desired attachment when type is 'archive'.
* Use a regular expression for pattern matching or a string for a literal match. Required if type is 'archive'.
* @property {RegExp|string} [file] - Identifier to isolate the desired attachment when type is 'file' or 'archive'.
* Use a regular expression for pattern matching or a string for a literal match. Required if type is 'file' or 'archive'.
* @property {number?} [maxLength=null] - Maximum number of characters to retrieve from the content. Optional.
* @property {number?} [maxLength] - Maximum number of characters to retrieve from the content.
*/

/**
* Configurable options provided to the Gmail adaptor.
* @typedef {Object} Options
* @public
* @property {string} userId - The email address of the Gmail account.
* @property {string?} [query=null] - Custom query to limit the messages result. Adheres to the Gmail search syntax. Optional.
* @property {Array<string|DesiredContent>?} [desiredContents=['from', 'date', 'subject', 'body']]
* An array of strings or DesiredContent objects used to specify which parts of the message to retrieve. Optional, default is `['from', 'date', 'subject', 'body']`.
* @property {Array<string>?} [processedIds=null] - Ignore message ids which have already been processed. Optional.
* @property {string?} [query] - Gmail search query string.
* @property {Array<string|MessageContent>} [desiredContents=['from', 'date', 'subject', 'body']]
* An array of strings or MessageContent objects used to specify which parts of the message to retrieve.
* @property {Array<string>} [processedIds] - Ignore message ids which have already been processed.
*/

/**
* Requests contents from messages of a Gmail account.
* Downloads contents from messages of a Gmail account.
* @public
* @function
* @param {string} userId - The email address of the account to retrieve messages from.
* @param {Options} userOptions - Customized options including desired contents and query.
* @returns {Function} A function that processes the state.
* @example
* @param {string} accountId - The email address of the account to retrieve messages from.
josephjclark marked this conversation as resolved.
Show resolved Hide resolved
* @param {Options} options - Customized options including desired contents and query.
josephjclark marked this conversation as resolved.
Show resolved Hide resolved
* @state {Array} data - The returned message objects, of the form `{ id, contents } `
* @state {Array<string>} processedIds - An array of string ids processed by this request
* @returns {Operation}
* @example <caption>Get a message with a specific subject</caption>
* getContentsFromMessages(
* 'user123@gmail.com',
* {
* query: 'subject:my+test+message'
* }
* )
* @example <caption>Get messages after a specific date, with subject and report.txt attachment</caption>
* getContentsFromMessages(
* 'test@tester.com',
* 'user123@gmail.com',
* {
* query: 'in:inbox subject:my+test+message',
* desiredContents: ['date', 'from', 'subject', { type: 'body', maxLength: 50 }]
* query: 'after:15/01/2025',
* desiredContents: [
* 'subject',
* { type: 'file', name: 'metadata', file: 'report.txt'}
* ]
* }
* )
*/
export function getContentsFromMessages(userId, userOptions) {
export function getContentsFromMessages(accountId, options) {
return async state => {
const [resolvedUserId, resolvedUserOptions] = expandReferences(
const [resolvedAccountId, resolvedOptions] = expandReferences(
state,
userId,
userOptions
accountId,
options
);

const defaultOptions = {
desiredContents: ['from', 'date', 'subject', 'body'],
};

const options = { ...defaultOptions, ...(resolvedUserOptions || {}) };
const opts = { ...defaultOptions, ...(resolvedOptions || {}) };

const contents = [];
const currentIds = [];
const previousIds = Array.isArray(options.processedIds)
? options.processedIds
const previousIds = Array.isArray(opts.processedIds)
? opts.processedIds
: [];

let nextPageToken = null;

do {
const messagesResult = await getMessagesResult(
resolvedUserId,
options.query,
resolvedAccountId,
opts.query,
nextPageToken
);

Expand All @@ -104,9 +115,9 @@ export function getContentsFromMessages(userId, userOptions) {
messageId: messageId,
};

const messageResult = await getMessageResult(userId, messageId);
const messageResult = await getMessageResult(accountId, messageId);

for (const desiredContentHint of options.desiredContents) {
for (const desiredContentHint of opts.desiredContents) {
const desiredContent = getDesiredContent(desiredContentHint);

const messageContent = await getMessageContent(
Expand Down
Loading