Introduce Datasource Filtering on sync command based on Name and Slug

README.md

@@ -22,35 +22,39 @@ $ npm i storyblok -g
 
 ## Commands
 
-
 ### login
 
 Login to the Storyblok cli
 
 ```sh
 $ storyblok login
 ```
+
 #### Login options
 
 ##### Options for Login with email and password
+
 * `email`: your user's email address
 * `password`: your user's password
 
 ##### Options for Login with token (Recomended to SSO user's but works with all user accounts)
+
 * `token`: your personal access token
 
 **Get your personal access token**
+
 * Go to [https://app.storyblok.com/#/me/account?tab=token](https://app.storyblok.com/#/me/account?tab=token) and click on Generate new token.
 
 **For Both login options you nedd to pass the region**
 
-* `region`: region you would like to work in. Please keep in mind that the region must match the region of your space. You can use `us`, `cn` or `eu`, if left empty, default is `eu`. This region flag will be used for the other cli's commands. 
+* `region`: region you would like to work in. Please keep in mind that the region must match the region of your space. You can use `us`, `cn` or `eu`, if left empty, default is `eu`. This region flag will be used for the other cli's commands.
 
 #### Login with token flag
+
 You can also add the token directly from the login’s command, like the example below:
 
 ```sh
-$ storyblok login --token <PERSONAL_ACCESS_TOKEN> --region eu 
+$ storyblok login --token <PERSONAL_ACCESS_TOKEN> --region eu
 ```
 
 ### logout
@@ -60,6 +64,7 @@ Logout from the Storyblok cli
 ```sh
 $ storyblok logout
 ```
+
 ### user
 
 Get the currently logged in user
@@ -68,7 +73,6 @@ Get the currently logged in user
 $ storyblok user
 ```
 
-
 ### select
 
 Usage to kickstart a boilerplate, fieldtype or theme
@@ -166,14 +170,17 @@ storyblok delete-component <component> --space <SPACE_ID>
 ```
 
 #### Parameters
+
 * `component`: The name or id of the component
 
 #### Options
+
 * `space_id`: the space where the command should be executed.
 
 #### Examples
 
 Delete a component on your space.
+
 ```sh
 storyblok delete-component 111111 --space 67819
 ```
@@ -192,6 +199,7 @@ storyblok delete-components <SOURCE> --space <SPACE_ID>
 ```
 
 #### Parameters
+
 * `source`: can be a URL or path to JSON file, the path to a json file could be to a single or multiple files separated by comma, like `./pages-1234.json,../User/components/grid-1234.json`
 
 Using an **URL**
@@ -213,23 +221,27 @@ $ storyblok push-components ./page.json,../grid.json,./feature.json --space 6781
 ```
 
 #### Options
+
 * `space_id`: the space where the command should be executed.
 * `reverse`: When passed as an argument, deletes only those components on your space that do not appear in the JSON.
 * `dryrun`: when passed as an argument, does not perform any changes on the given space.
 
 #### Examples
 
 Delete all components on a certain space that occur in your local JSON.
+
 ```sh
 storyblok delete-components ./components.json --space 67819
 ```
 
 Delete only those components which do not occur in your local json from your space.
+
 ```sh
 storyblok delete-components ./components.json --space 67819 --reverse
 ```
 
 To see the result in your console output but to not perform the command on your space, use the `--dryrun` argument.
+
 ```sh
 storyblok delete-components ./components.json --space 67819 --reverse --dryrun
 ```
@@ -247,10 +259,9 @@ $ storyblok sync --type <COMMAND> --source <SPACE_ID> --target <SPACE_ID>
 * `type`: describe the command type to execute. Can be: `folders`, `components`, `stories`, `datasources` or `roles`. It's possible pass multiple types separated by comma (`,`).
 * `source`: the source space to use to sync
 * `target`: the target space to use to sync
-* `starts-with`: sync only stories that starts with the given string
-* `filter`: sync stories based on the given filter. Required Options: Required options: `--keys`, `--operations`, `--values`
-* `keys`: Multiple keys should be separated by comma. Example: `--keys key1,key2`, `--keys key1`
-* `operations`: Operations to be used for filtering. Can be: `is`, `in`, `not_in`, `like`, `not_like`, `any_in_array`, `all_in_array`, `gt_date`, `lt_date`, `gt_int`, `lt_int`, `gt_float`, `lt_float`. Multiple operations should be separated by comma.
+* `components-groups`: Synchronize components based on their group UUIDs separated by commas. Example: `--components-groups ******-****`
+* `datasources-starts-with-slug`: Synchronize datasources that starts with the given slug. Example: `--datasources-starts-with-slug global-translations`
+* `datasources-starts-with-name`: Synchronize datasources that starts with the given name. Example: `--datasources-starts-with-name Translations`
 
 #### Examples
 
@@ -261,14 +272,15 @@ $ storyblok sync --type components --source 00001 --target 00002
 # Sync components and stories from `00001` space to `00002` space
 $ storyblok sync --type components,stories --source 00001 --target 00002
 
-# Sync only stories that starts with `myStartsWithString` from `00001` space to `00002` space
-$ storyblok sync --type stories --source 00001 --target 00002 --starts-with myStartsWithString
+# Synchronize components based on their group UUIDs separated by commas
+$ storyblok sync --type components --source 00001 --target 00002 --components-groups xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx,xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+
+# Synchronize datasources that starts with the given slug
+$ storyblok sync --type datasources  --source 00001 --target 00002 --datasources-starts-with-slug global-translations
 
-# Sync only stories with a category field like `reference` from `00001` space to `00002` space
-$ storyblok sync --type stories --source 00001 --target 00002 --filter --keys category --operations like --values reference
+# Synchronize datasources that starts with the given name (input is converted to lowercase)
+$ storyblok sync --type datasources  --source 00001 --target 00002 --datasources-starts-with-name Global Translations
 
-# Sync only stories with a category field like `reference` and a name field not like `demo` from `00001` space to `00002` space
-$ storyblok sync --type stories --source 00001 --target 00002 --filter --keys category,name --operations like,not_like --values reference,demo
 ```
 
 ### quickstart
@@ -286,6 +298,7 @@ Create a migration file (with the name `change_<COMPONENT>_<FIELD>.js`) inside t
 ```sh
 $ storyblok generate-migration --space <SPACE_ID> --component <COMPONENT_NAME> --field <FIELD>
 ```
+
 It's important to note that the `component` and `field` parameters are required and must be spelled exactly as they are in Storyblok. You can check the exact name by looking at the `Block library` inside your space.
 
 #### Options
@@ -338,7 +351,6 @@ $ storyblok rollback-migration --space 1234 --component Product --field title
 
 ### spaces
 
-
 List all spaces of the logged account
 
 ```sh
@@ -380,7 +392,7 @@ this-is-my-title;This is my title;"Lorem ipsum dolor sit amet";https://a.storybl
 A json file need to have following format:
 
 ```json
-[ 
+[
   {
     "path": "this-is-my-title",
     "title": "This is my title",
@@ -445,10 +457,9 @@ The created file will have the following content:
 module.exports = function (block) {
   // Example to change a string to boolean
   // block.subtitle = !!(block.subtitle)
-
   // Example to transfer content from other field
   // block.subtitle = block.other_field
-}
+};
 ```
 
 In the migration function you can manipulate the block variable to add or modify existing fields of the component.
@@ -520,7 +531,6 @@ $ storyblok run-migration --space 00000 --component product --field image --dryr
 
 #### 2. Transform a Markdown field into a Richtext field
 
-
 To transform a markdown or html field into a richtext field you first need to install a converter library.
 
 ```sh

package.json

@@ -42,7 +42,7 @@
     "p-series": "^2.1.0",
     "path": "^0.12.7",
     "simple-uuid": "^0.0.1",
-    "storyblok-js-client": "^4.5.6",
+    "storyblok-js-client": "^5.12.0",
     "update-notifier": "^5.1.0",
     "xml-js": "^1.6.11"
   },
@@ -65,4 +65,4 @@
       "master"
     ]
   }
-}
+}

src/cli.js

@@ -12,7 +12,7 @@ const updateNotifier = require('update-notifier')
 const pkg = require('../package.json')
 
 const tasks = require('./tasks')
-const { getQuestions, lastStep, api, creds, buildFilterQuery } = require('./utils')
+const { getQuestions, lastStep, api, creds } = require('./utils')
 const { SYNC_TYPES, COMMANDS } = require('./constants')
 
 clear()
@@ -24,16 +24,14 @@ console.log()
 
 // non-intrusive notify users if an update available
 const notifyOptions = {
-  isGlobal: true
+  isGlobal: true,
 }
 
-updateNotifier({ pkg })
-  .notify(notifyOptions)
+updateNotifier({ pkg }).notify(notifyOptions)
 
 program.version(pkg.version)
 
-program
-  .option('-s, --space [value]', 'space ID')
+program.option('-s, --space [value]', 'space ID')
 
 // login
 program
@@ -94,7 +92,7 @@ program
 // pull-languages
 program
   .command('pull-languages')
-  .description("Download your space's languages schema as json")
+  .description('Download your space\'s languages schema as json')
   .action(async () => {
     console.log(`${chalk.blue('-')} Executing pull-languages task`)
     const space = program.space
@@ -122,7 +120,7 @@ program
   .option('--sf, --separate-files [value]', 'Argument to create a single file for each component')
   .option('-p, --path <path>', 'Path to save the component files')
   .option('-f, --file-name <fileName>', 'custom name to be used in file(s) name instead of space id')
-  .description("Download your space's components schema as json")
+  .description('Download your space\'s components schema as json')
   .action(async (options) => {
     console.log(`${chalk.blue('-')} Executing pull-components task`)
     const space = program.space
@@ -150,7 +148,7 @@ program
 program
   .command(COMMANDS.PUSH_COMPONENTS + ' <source>')
   .option('-p, --presets-source <presetsSource>', 'Path to presets file')
-  .description("Download your space's components schema as json. The source parameter can be a URL to your JSON file or a path to it")
+  .description('Download your space\'s components schema as json. The source parameter can be a URL to your JSON file or a path to it')
   .action(async (source, options) => {
     console.log(`${chalk.blue('-')} Executing push-components task`)
     const space = program.space
@@ -271,12 +269,9 @@ program
   .requiredOption('--type <TYPE>', 'Define what will be sync. Can be components, folders, stories, datasources or roles')
   .requiredOption('--source <SPACE_ID>', 'Source space id')
   .requiredOption('--target <SPACE_ID>', 'Target space id')
-  .option('--starts-with <STARTS_WITH>', 'Sync only stories that starts with the given string')
-  .option('--filter', 'Enable filter options to sync only stories that match the given filter. Required options: --keys; --operations; --values')
-  .option('--keys <KEYS>', 'Field names in your story object which should be used for filtering. Multiple keys should separated by comma.')
-  .option('--operations <OPERATIONS>', 'Operations to be used for filtering. Can be: is, in, not_in, like, not_like, any_in_array, all_in_array, gt_date, lt_date, gt_int, lt_int, gt_float, lt_float. Multiple operations should be separated by comma.')
-  .option('--values <VALUES>', 'Values to be used for filtering. Any string or number. If you want to use multiple values, separate them with a comma. Multiple values should be separated by comma.')
   .option('--components-groups <UUIDs>', 'Synchronize components based on their group UUIDs separated by commas')
+  .option('--datasources-starts-with-slug <SLUG>', 'Synchronize datasources that starts with the given slug')
+  .option('--datasources-starts-with-name <NAME>', 'Synchronize datasources that starts with the given name')
   .action(async (options) => {
     console.log(`${chalk.blue('-')} Sync data between spaces\n`)
 
@@ -287,36 +282,34 @@ program
 
       const {
         type,
-        source,
         target,
-        startsWith,
-        filter,
-        keys,
-        operations,
-        values,
-        componentsGroups
+        source,
+        componentsGroups,
+        datasourcesStartsWithSlug,
+        datasourcesStartsWithName,
       } = options
 
-      const _componentsGroups = componentsGroups ? componentsGroups.split(',') : null
+      const _componentsGroups = componentsGroups
+        ? componentsGroups.split(',')
+        : null
+
+      const token = creds.get().token || null
 
       const _types = type.split(',') || []
-      _types.forEach(_type => {
+      _types.forEach((_type) => {
         if (!SYNC_TYPES.includes(_type)) {
           throw new Error(`The type ${_type} is not valid`)
         }
       })
 
-      const filterQuery = filter ? buildFilterQuery(keys, operations, values) : undefined
-
-      const token = creds.get().token || null
       await tasks.sync(_types, {
         api,
         token,
-        source,
         target,
-        startsWith,
-        filterQuery,
-        _componentsGroups
+        source,
+        _componentsGroups,
+        datasourcesStartsWithSlug,
+        datasourcesStartsWithName,
       })
 
       console.log('\n' + chalk.green('✓') + ' Sync data between spaces successfully completed')
@@ -399,7 +392,7 @@ program
     const publishOptionsAvailable = [
       'all',
       'published',
-      'published-with-changes'
+      'published-with-changes',
     ]
     if (publish && !publishOptionsAvailable.includes(publish)) {
       console.log(chalk.red('X') + ' Please provide a correct publish option: all, published, or published-with-changes')
@@ -449,7 +442,7 @@ program
 
       await tasks.rollbackMigration(api, field, component)
     } catch (e) {
-      console.log(chalk.red('X') + ' An error ocurred when run rollback-migration: ' + e.message)
+      console.log(chalk.red('X') + ' An error ocurred when run rollback-migration: ' +e.message)
       process.exit(1)
     }
   })
@@ -496,9 +489,7 @@ program
       api.setSpaceId(space)
       await tasks.importFiles(api, options)
 
-      console.log(
-        `${chalk.green('✓')} The import process was executed with success!`
-      )
+      console.log(`${chalk.green('✓')} The import process was executed with success!`)
     } catch (e) {
       console.log(chalk.red('X') + ' An error ocurred to import data : ' + e.message)
       process.exit(1)
@@ -511,7 +502,7 @@ if (program.rawArgs.length <= 2) {
   program.help()
 }
 
-function errorHandler (e, command) {
+function errorHandler(e, command) {
   if (/404/.test(e.message)) {
     console.log(chalk.yellow('/!\\') + ' If your space was created under US or CN region, you must provide the region us or cn upon login.')
   } else {

src/constants.js

@@ -28,12 +28,18 @@ const COMMANDS = {
   SYNC: 'sync'
 }
 
+const DEFAULT_AGENT = {
+  SB_Agent: 'SB-CLI',
+  SB_Agent_Version: process.env.npm_package_version || '3.0.0'
+}
+
 module.exports = {
   LOGIN_URL,
   SIGNUP_URL,
   API_URL,
   SYNC_TYPES,
   US_API_URL,
   CN_API_URL,
-  COMMANDS
+  COMMANDS,
+  DEFAULT_AGENT
 }