Deep merge token files override tokens if inner structure is same.

[Charlene-Bx]

That's an issue that keep me digging for at least 3days now. Any help will be welcome:

If design token file on component level with some breakpoints exeptions so I end up with this kind of case:

/component/surface/carousel/basic/self-small.json:

"component": {
    "surface": {
      "carousel": {
        "basic": {
          "self": {
            "normal": {
              "base": {
                "default": {
                  "gap": {
                    "value": "{spacing.3xs}",
                    "type": "spacing"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

/component/surface/carousel/basic/self-medium.json:

"component": {
    "surface": {
      "carousel": {
        "basic": {
          "self": {
            "normal": {
              "base": {
                "default": {
                  "gap": {
                    "value": "{spacing.4xs}",
                    "type": "spacing"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

They have exact same structure with differents value for breakpoints exception so my dictionary will end-up with just one value overwritted with the last readed files from "source":

[
  {
    value: '8px',
    type: 'dimension',
    filePath: 'tokens/component/surface/carousel/basic/self-small.json',
    isSource: true,
    '$extensions': { 'studio.tokens': [Object] },
    original: {
      value: '{spacing.4xs}',
      type: 'dimension',
      '$extensions': [Object]
    },
    name: 'carousel-self-base-default-gap',
    attributes: {
      ap_breakpoint: 'small',
      ap_component: 'carousel',
      ap_variation: 'basic',
      ap_item: null,
      ap_mode: 'normal',
      ap_status: 'base',
      ap_state: 'default'
    },
    path: [
      'component', 'surface',
      'carousel',  'basic',
      'self',      'normal',
      'base',      'default',
      'gap'
    ]
  }
].

My first approach was to divide build from breakpoint, and then perform an action to merge them in on single files. But have just noticied that the build run in paralel, so output can vary - I've no security that it wille first manage the small break point, then the medium and so one, ...
So my new tentative is to build all tokens in once... But they are overwrited sometimes. What approach did you suggest in these cases? I will giving up and create a script that will run after the style dictionary build... But I look like I'm not doing the right thing - I mean, it will work for sure... But custom script to run after a library script that should handle the job looks bad. Before doing it, I want to make sure I didn't miss any others better options.

Thanks for any help!

[Charlene-Bx]

Improvement could be to add a meta data on token output for such case instead just displaying a warning message and overwrite it.
something like:

'colision': {
  'filePath': 'tokens/component/surface/carousel/basic/self-medium.json',
  'original':'{ value: '{spacing.3xs}', type: 'spacing' }' 
  }

so token will look like:

  {
    value: '8px',
    type: 'dimension',
    filePath: 'tokens/component/surface/carousel/basic/self-small.json',
    isSource: true,
    '$extensions': { 'studio.tokens': [Object] },
    original: {
      value: '{spacing.4xs}',
      type: 'dimension',
      '$extensions': [Object]
    },
    **'colision': [{
      'filePath': 'tokens/component/surface/carousel/basic/self-small.json',
      'original':'{ value: '{spacing.3xs}', type: 'spacing' }' 
      }]**
    name: 'carousel-self-base-default-gap',
    attributes: {
      ap_breakpoint: 'small',
      ap_component: 'carousel',
      ap_variation: 'basic',
      ap_item: null,
      ap_mode: 'normal',
      ap_status: 'base',
      ap_state: 'default'
    },
    path: [
      'component', 'surface',
      'carousel',  'basic',
      'self',      'normal',
      'base',      'default',
      'gap'
    ]
  }

[jorenbroekema]

Yeah so in your case you have a token collision that is actually not to be ignored and causes an issue (you lose whichever tokens get overwritten by having identical structure with tokens in other sets)

I would look into this approach https://github.com/amzn/style-dictionary/tree/main/examples/advanced/multi-brand-multi-platform

Setting up your breakpoints as "themes" essentially, where breakpoint is a theme dimension and e.g. "mobile" "tablet" "desktop" could be variants within that theme dimension. Some tokensets will be included for each variant (e.g. your primitives/globals/references tokens) and some tokensets will be theme variant specific and only included if we're building for that variant.

This approach can be used for multiple theming dimensions as well if you later need to add other dimensions such as light/dark mode, or high/low emphasis, high/low contrast, different subbrands.

---

If you are using Tokens Studio, this will be useful https://github.com/tokens-studio/sd-transforms?tab=readme-ov-file#theming
As Tokens Studio, we have quite a lot of thoughts on how we think folks should approach theming, we're even contributing a proposal to the DTCG spec for theme resolution and we published some docs on that https://resolver-spec.netlify.app/ , regardless of being a Tokens Studio user or not, this will be an interesting read

[jorenbroekema]

Another approach that you can take is running a Style Dictionary for each breakpoint, outputting them to memory instead of writing to a file, and then combining them or doing whatever you want with the CSS output strings, this gives you way more control than SD's default action which is writing to the filesystem for you without being able to control the timings:

const cfgs = ['small', 'medium'].map(breakpoint => ({
  source: [`tokens/**/*-${breakpoint}.json`],
  platforms: {
    css: {
      transformGroup: 'css',
      files: [{
        format: 'css/variables',
        destination: `output/css/${breakpoint}.css`
      }]
    }
  }
}));

async function runSD(cfg) {
  const sd = new StyleDictionary(cfg);
   // array destructure, since you can have multiple "files" in platform config
  const [file] = await sd.formatPlatform('css');
  return [file.destination, file.output];
}

const outputs = Object.fromEntries(await Promise.all(cfgs.map(runSD)));
/**
 * Now you have your outputs in JS memory, as an object
 * You can write them to the filesystem yourself, you can combine them into
 * a file, it's all up to you!
 * {
 *   "output/css/small.css": ":root {...}",
 *   "output/css/medium.css": ":root {...}"
 * }
 */