Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a filter or changing encoding type causes Error: Arrow field values can not be overwritten. #327

Open
tjader opened this issue Jan 3, 2024 · 1 comment

Comments

@tjader
Copy link

tjader commented Jan 3, 2024

Hi, It looks like the arrow loader fails if the chart is having some transforms (at least the filter transform) or if the field type is changed in the encoding. The error thrown is Error: Arrow field values can not be overwritten.

It seems that the vega library is in many cases writing back into the object in-place, even though the in-place writing seems write back the same data.
Commenting out the line throwing the error made the figures render, but I'm not sure it is a good solution moving forward.

        set: function() {
          //throw Error('Arrow field values can not be overwritten.');  <--commented out line
        },
        enumerable: true
      });
   ...

This baseline chart works out of the box (using the current master of vega-loader-arrow)

{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "data": {
    "url": "https://gist.githubusercontent.com/domoritz/0f53a5abde95564c36dfaac623a7a922/raw/cce3719b853e25d5dfff97a270283ba83af3c0e6/flights-10k.arrow",
    "format": {
      "type": "arrow"
    }
  },
  "encoding": {
    "x": {
      "field": "FL_DATE"
    },
    "y": {
      "aggregate": "sum",
      "field": "ARR_DELAY",
      "type": "quantitative"
    }
  },
  "mark": {
    "type": "bar"
  }
}

But adding a filter causes the error.

{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "data": {
    "url": "https://gist.githubusercontent.com/domoritz/0f53a5abde95564c36dfaac623a7a922/raw/cce3719b853e25d5dfff97a270283ba83af3c0e6/flights-10k.arrow",
    "format": {
      "type": "arrow"
    }
  },
  "transform": [
    { "filter": {
      "field": "ARR_DELAY",
      "gt": 360.0
    }}
    ],
  "encoding": {
    "x": {
      "field": "FL_DATE"
    },
    "y": {
      "aggregate": "sum",
      "field": "ARR_DELAY",
      "type": "quantitative"
    }
  },
  "mark": {
    "type": "bar"
  }
}

I have attached a few other examples of failing renders. They all work if the exception is commented out.
[flight10k_filter.json](https://github.com/vega/vega-loader-arrow/files/13818262/flight10k_filter.json)
[flight10k_ordinal.json](https://github.com/vega/vega-loader-arrow/files/13818263/flight10k_ordinal.json)
[flight10k_timeunit.json](https://github.com/vega/vega-loader-arrow/files/13818264/flight10k_timeunit.json)
[flight10k_temporal.json](https://github.com/vega/vega-loader-arrow/files/13818265/flight10k_temporal.json)
[flight10k_ok.json](https://github.com/vega/vega-loader-arrow/files/13818267/flight10k_ok.json)

I have tested these against the latest master (the latest release 0.1.0 uses arrow 6 which is not compatible with the dataset), where I also could confirm the "fix" of removing raising the error.

Btw, it would be nice to have a new release with a more up-to-date version of arrow supported.

@domoritz
Copy link
Member

domoritz commented Jan 5, 2024

I made a release for vega-loader-arrow@0.2.0.

The error is coming from

throw Error('Arrow field values can not be overwritten.');
as you said. Writing back to arrow shouldn't really be happening. I'd suggest finding out why the set is happening by putting a breakpoint I the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants