Enhancement: import agenda from CSV #550

michaelmhoffman · 2020-07-14T19:31:17Z

I often need to do bulk editing of my calendar which is not super convenient with the Google Calendar web interface. A spreadsheet interface would be much better.

What would you think about a way to do roundtrip updates through gcalcli agenda --tsv and a paired command agendaupdate for writing any changes back? We would need:

Some refactoring of gcalcli.gcal.GoogleCalendarInterface._tsv(). I would like to replace the current long list of if statements with a modular design based around new class gcalcli.details.Handler. Each Handler represents the set of event properties that are captured by each item of gcalcli.argparsers.DETAILS. These objects would have get() handlers for TSV. They would be stored in a collections.OrderedDict called gcalcli.details.HANDLERS.
Set gcalcli.argparsers.DETAILS to a superset of gcalcli.details.HANDLERS.keys() to avoid duplication.
Add a header row to TSV. This would probably break old code so would require a major release. The alternative would be to add an option like --header.
Implement a Handler for id to make it trivial to find the right Event later for the round trip.
Implement Handler.patch() for a limited set of details. Throw error if there is a column not in the implemented set.
Implement Handler.patch() for the rest of the details.

Future extensions or subsequent refactoring could include:

Refactoring other uses of patch to call details.Handler subclasses
Add Detail.format() handlers that could be used for gcalcli.gcal.GoogleCalendarInterface._PrintEvent().
A feature to add new events through this interface instead of only updating events with existing IDs (Enhancement: import agenda from CSV #550)
Adding ability to support multiple calendars in agendaupdate
Adding ability to move events between calendars by changing the calendar field (see commit message for d7589ed)
A feature to do the round-trip through a Google Sheets instead of a local TSV

The text was updated successfully, but these errors were encountered:

michaelmhoffman · 2020-07-27T22:15:41Z

@jcrowgey Could you let me know what you think of this design?

jcrowgey · 2020-07-28T16:59:55Z

Overall this looks good to me. Your recent work on this project has been very helpful and appreciated. I'm sorry for the long delays. In general, I'm not going to be able to work on gcalcli much for the foreseeable future.

…

On Mon, Jul 27, 2020 at 03:15:57PM -0700, Michael Hoffman wrote: @jcrowgey Could you let me know what you think of this design? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #550 (comment)

Refactor `gcalcli.gcal.GoogleCalendarInterface._tsv()`, replacing the long list of if statements with a modular design using a series of `gcalcli.details.Handler` classes. Each `Handler` represents the set of event properties that are captured by each item of `gcalcli.argparsers.DETAILS`. So far only the `header` class attribute and the `get()` methods are implemented. This is what is necessary for TSV output. Preparatory work for insanum#550.

Set `gcalcli.argparsers.DETAILS` to a superset of `gcalcli.details.HANDLERS.keys()` to avoid duplication. Preparatory work for insanum#550.

BREAKING CHANGE: having a header row will break any software that assumes that the output of `gcalcli agenda --tsv` doesn't have one. Also slightly refactored the output printing to use print() like in most parts of the code. Partially implements insanum#550.

Partially implements insanum#550.

argparsers.py, cli.py: add `agendaupdate` subcommand gcal.py: add `AgendaUpdate()` details.py: - `Title`, `SimpleSingleFieldHandler`: add `patch()` - add `FIELD_HANDLERS` Minimum viable product for insanum#550.

This will enable `Handler`s with multiple fields to work for patching. Necessary for `Time.patch()` and `Conference.patch()` Also introduce `SingleFieldHandler.patch(cls,` ...`)` which dispatches to `cls._patch()`. Preparatory for further implementation of insanum#550.

Round trips don't work for all day events. The current output format is to present them as events starting and ending at midnight of the same day. But this keeps us from having events starting and ending at midnight of the same day. In the next commit I plan to change the output for all day events so the time fields are blank. Also: - change signature of `Handler.patch()` to include `cal`, which is necessary to extract the timezone - change default of `Handler.fieldnames` from `None` to `[]` to silence a mypy warning. Partially implements insanum#550.

michaelmhoffman · 2020-08-10T18:52:03Z

To get the TSV output of gcalcli agenda to work automatically in Microsoft Excel with non-ASCII characters, it is necessary to set PYTHONIOENCODING=utf_16. Things that don't work utf_16_le (no BOM) or utf_8_sig (BOM but for whatever reason Excel ignores the tabs by default). This does not seem like a sensible default but if you intend to use these files with Excel you should know about it.

michaelmhoffman · 2020-08-10T20:32:59Z

The easiest way to use a UTF-16 file.tsv is to use input redirection with the PYTHONIOENCODING environment variable set. There's no user-accessible way to change the encoding of what's read with a file argument.

PYTHONIOENCODING=utf_16 gcalcli agendaupdate < file.tsv

michaelmhoffman · 2020-08-15T23:53:36Z

The output of PYTHONIOENCODING=utf_16 gcalcli agenda isn't easy to use some other Unix tools like grep with. Instead you can use iconv --to-code=utf-16 as part of a pipeline. For example:

gcalcli agenda | grep Title | iconv --to-code=utf-16 > file.csv

Partially implements insanum#522 and insanum#550.

This treats calendar as a readonly field. Actually moving calendar would be a bit more involved, and would necessitate: - changing agendaupdate from its current one-calendar-only restriction - using the events().move() method to do the move. It's not done by events().patch() That can be a future feature. Partially implements insanum#550.

* feat: add agendaupdate for calendar This treats calendar as a readonly field. Actually moving calendar would be a bit more involved, and would necessitate: - changing agendaupdate from its current one-calendar-only restriction - using the events().move() method to do the move. It's not done by events().patch() That can be a future feature. Partially implements #550. * feat!: add header row for `gcalcli agenda --tsv` BREAKING CHANGE: having a header row will break any software that assumes that the output of `gcalcli agenda --tsv` doesn't have one. Also slightly refactored the output printing to use print() like in most parts of the code. * feat!: gcal agenda --tsv prints empty time fields for all-day events BREAKING CHANGE: this changes the previous behavior which was to depict all-day events as starting and ending at midnight. * feat: add `--detail id` for `gcalcli agenda --tsv` * feat: add agendaupdate for title, id, location, description argparsers.py, cli.py: add `agendaupdate` subcommand gcal.py: add `AgendaUpdate()` details.py: - `Title`, `SimpleSingleFieldHandler`: add `patch()` - add `FIELD_HANDLERS` * feat: add agendaupdate for time Round trips don't work for all day events. The current output format is to present them as events starting and ending at midnight of the same day. But this keeps us from having events starting and ending at midnight of the same day. Also: - change signature of `Handler.patch()` to include `cal`, which is necessary to extract the timezone - change default of `Handler.fieldnames` from `None` to `[]` to silence a mypy warning. * feat: add agendaupdate for url Both these fields are read-only. If html_link is in the input, always fail. If hangout_link is in the input, check against the current value. If they don't match, fail. * feat: add agendaupdate for conference Partially implements #522. * refactor: move _tsv() implementation to gcalcli.details Refactor `gcalcli.gcal.GoogleCalendarInterface._tsv()`, replacing the long list of if statements with a modular design using a series of `gcalcli.details.Handler` classes. Each `Handler` represents the set of event properties that are captured by each item of `gcalcli.argparsers.DETAILS`. So far only the `header` class attribute and the `get()` methods are implemented. This is what is necessary for TSV output. * fix: change SimpleSingleColumnHandler.header to a simple list Can't use a property on a classmethod without making a custom metaclass. Without that, the property is only enacted upon class instantiation, which we aren't doing here. A custom metaclass is too magic, and thus far we have avoided instantiating what would be singleton objects. Also changed `SimpleSingleColumnHandler._get()` to use `cls.header[0]` instead of `cls._key` which is removed. * ci: eliminate flake8-docstrings method docstring checks for details.py These are mostly overriding pure virtual classes and adding a docstring to each one would be superfluous and make the code harder to read. * refactor: derive `argparsers.DETAILS` from `details.HANDLERS` Set `gcalcli.argparsers.DETAILS` to a superset of `gcalcli.details.HANDLERS.keys()` to avoid duplication. * refactor: change `header` to `fieldnames` and `column` to `field` * refactor: change `Handler.patch()` to include `fieldname` argument This will enable `Handler`s with multiple fields to work for patching. Necessary for `Time.patch()` and `Conference.patch()` Also introduce `SingleFieldHandler.patch(cls,` ...`)` which dispatches to `cls._patch()`. * refactor: move gcal.GoogleCalendarInterface._isallday() to utils.is_all_day() This enables its use by details.py. * refactor: `details.Url` uses `details.URL_PROPS` to identify properties to get The new `details.URL_PROPS` constant `OrderedDict` will also work for use in `Url.patch()`, which should be implemented in the next commit. This will allow automated handling of two property/fieldname pairs for this `Handler`. * refactor: add `exceptions.ReadonlyError`, `exceptions.ReadonlyCheckError`

Implement #550. Includes BREAKING CHANGES. Perhaps next release should increase major version number. * feat!: add header row for `gcalcli agenda --tsv` BREAKING CHANGE: having a header row will break any software that assumes that the output of `gcalcli agenda --tsv` doesn't have one. Also slightly refactored the output printing to use print() like in most parts of the code. * feat!: gcal agenda --tsv prints empty time fields for all-day events BREAKING CHANGE: this changes the previous behavior which was to depict all-day events as starting and ending at midnight. * feat: add `--detail id` for `gcalcli agenda --tsv` * feat: add agendaupdate for title, id, location, description argparsers.py, cli.py: add `agendaupdate` subcommand gcal.py: add `AgendaUpdate()` details.py: - `Title`, `SimpleSingleFieldHandler`: add `patch()` - add `FIELD_HANDLERS` * feat: add agendaupdate for time Round trips don't work for all day events. The current output format is to present them as events starting and ending at midnight of the same day. But this keeps us from having events starting and ending at midnight of the same day. In the next commit I plan to change the output for all day events so the time fields are blank. Also: - change signature of `Handler.patch()` to include `cal`, which is necessary to extract the timezone - change default of `Handler.fieldnames` from `None` to `[]` to silence a mypy warning. * feat: add detail for `action` for `gcalcli agenda`. Column is ignored by `gcalcli agendaupdate`. * feat: add agendaupdate action dispatch and `patch` action Refactor most of existing agendaupdate into new module `gcalcli.actions` in `patch()`. * feat: add agendaupdate for calendar This treats calendar as a readonly field. Actually moving calendar would be a bit more involved, and would necessitate: - changing agendaupdate from its current one-calendar-only restriction - using the events().move() method to do the move. It's not done by events().patch() That can be a future feature. * feat: add agendaupdate `ignore` action * feat: add agendaupdate `delete` action * feat: add agendaupdate `insert` action * feat: `agendaupdate` patch action uses insert if no id supplied * feat: error when unsupported agendaupdate action used Also limits actions to a defined set. Before one could have specified action to be name of some other object in `gcalcli.actions` with unpredictable results. * feat: add agendaupdate for url Both these fields are read-only. If html_link is in the input, always fail. If hangout_link is in the input, check against the current value. If they don't match, fail. * feat: add agendaupdate for conference Partially implements #522. * fix: change SimpleSingleColumnHandler.header to a simple list Can't use a property on a classmethod without making a custom metaclass. Without that, the property is only enacted upon class instantiation, which we aren't doing here. A custom metaclass is too magic, and thus far we have avoided instantiating what would be singleton objects. Also changed `SimpleSingleColumnHandler._get()` to use `cls.header[0]` instead of `cls._key` which is removed. * fix: gcalcli agenda to work with refactor 72710b4 * refactor: move _tsv() implementation to gcalcli.details Refactor `gcalcli.gcal.GoogleCalendarInterface._tsv()`, replacing the long list of if statements with a modular design using a series of `gcalcli.details.Handler` classes. Each `Handler` represents the set of event properties that are captured by each item of `gcalcli.argparsers.DETAILS`. So far only the `header` class attribute and the `get()` methods are implemented. This is what is necessary for TSV output. * refactor: derive `argparsers.DETAILS` from `details.HANDLERS` Set `gcalcli.argparsers.DETAILS` to a superset of `gcalcli.details.HANDLERS.keys()` to avoid duplication. * refactor: change `header` to `fieldnames` and `column` to `field` * refactor: change `Handler.patch()` to include `fieldname` argument This will enable `Handler`s with multiple fields to work for patching. Necessary for `Time.patch()` and `Conference.patch()` Also introduce `SingleFieldHandler.patch(cls,` ...`)` which dispatches to `cls._patch()`. * refactor: move gcal.GoogleCalendarInterface._isallday() to utils.is_all_day() This enables its use by details.py. * refactor: `details.Url` uses `details.URL_PROPS` to identify properties to get The new `details.URL_PROPS` constant `OrderedDict` will also work for use in `Url.patch()`, which should be implemented in the next commit. This will allow automated handling of two property/fieldname pairs for this `Handler`. * refactor: add `exceptions.ReadonlyError`, `exceptions.ReadonlyCheckError` * refactor: add `gcal.GoogleCalendarInterface.get_events()` change previous uses of `get_cal_service().events()` to `get_events()` * refactor: add `gcal.GoogleCalendarInterface.delete()` * refactor: add gcalcli.actions._iter_field_handlers() * style: add extra line of whitespace * ci: eliminate flake8-docstrings method docstring checks for details.py These are mostly overriding pure virtual classes and adding a docstring to each one would be superfluous and make the code harder to read.

* feat: add agendaupdate for calendar This treats calendar as a readonly field. Actually moving calendar would be a bit more involved, and would necessitate: - changing agendaupdate from its current one-calendar-only restriction - using the events().move() method to do the move. It's not done by events().patch() That can be a future feature. Partially implements insanum#550. * feat!: add header row for `gcalcli agenda --tsv` BREAKING CHANGE: having a header row will break any software that assumes that the output of `gcalcli agenda --tsv` doesn't have one. Also slightly refactored the output printing to use print() like in most parts of the code. * feat!: gcal agenda --tsv prints empty time fields for all-day events BREAKING CHANGE: this changes the previous behavior which was to depict all-day events as starting and ending at midnight. * feat: add `--detail id` for `gcalcli agenda --tsv` * feat: add agendaupdate for title, id, location, description argparsers.py, cli.py: add `agendaupdate` subcommand gcal.py: add `AgendaUpdate()` details.py: - `Title`, `SimpleSingleFieldHandler`: add `patch()` - add `FIELD_HANDLERS` * feat: add agendaupdate for time Round trips don't work for all day events. The current output format is to present them as events starting and ending at midnight of the same day. But this keeps us from having events starting and ending at midnight of the same day. Also: - change signature of `Handler.patch()` to include `cal`, which is necessary to extract the timezone - change default of `Handler.fieldnames` from `None` to `[]` to silence a mypy warning. * feat: add agendaupdate for url Both these fields are read-only. If html_link is in the input, always fail. If hangout_link is in the input, check against the current value. If they don't match, fail. * feat: add agendaupdate for conference Partially implements insanum#522. * refactor: move _tsv() implementation to gcalcli.details Refactor `gcalcli.gcal.GoogleCalendarInterface._tsv()`, replacing the long list of if statements with a modular design using a series of `gcalcli.details.Handler` classes. Each `Handler` represents the set of event properties that are captured by each item of `gcalcli.argparsers.DETAILS`. So far only the `header` class attribute and the `get()` methods are implemented. This is what is necessary for TSV output. * fix: change SimpleSingleColumnHandler.header to a simple list Can't use a property on a classmethod without making a custom metaclass. Without that, the property is only enacted upon class instantiation, which we aren't doing here. A custom metaclass is too magic, and thus far we have avoided instantiating what would be singleton objects. Also changed `SimpleSingleColumnHandler._get()` to use `cls.header[0]` instead of `cls._key` which is removed. * ci: eliminate flake8-docstrings method docstring checks for details.py These are mostly overriding pure virtual classes and adding a docstring to each one would be superfluous and make the code harder to read. * refactor: derive `argparsers.DETAILS` from `details.HANDLERS` Set `gcalcli.argparsers.DETAILS` to a superset of `gcalcli.details.HANDLERS.keys()` to avoid duplication. * refactor: change `header` to `fieldnames` and `column` to `field` * refactor: change `Handler.patch()` to include `fieldname` argument This will enable `Handler`s with multiple fields to work for patching. Necessary for `Time.patch()` and `Conference.patch()` Also introduce `SingleFieldHandler.patch(cls,` ...`)` which dispatches to `cls._patch()`. * refactor: move gcal.GoogleCalendarInterface._isallday() to utils.is_all_day() This enables its use by details.py. * refactor: `details.Url` uses `details.URL_PROPS` to identify properties to get The new `details.URL_PROPS` constant `OrderedDict` will also work for use in `Url.patch()`, which should be implemented in the next commit. This will allow automated handling of two property/fieldname pairs for this `Handler`. * refactor: add `exceptions.ReadonlyError`, `exceptions.ReadonlyCheckError`

Implement insanum#550. Includes BREAKING CHANGES. Perhaps next release should increase major version number. * feat!: add header row for `gcalcli agenda --tsv` BREAKING CHANGE: having a header row will break any software that assumes that the output of `gcalcli agenda --tsv` doesn't have one. Also slightly refactored the output printing to use print() like in most parts of the code. * feat!: gcal agenda --tsv prints empty time fields for all-day events BREAKING CHANGE: this changes the previous behavior which was to depict all-day events as starting and ending at midnight. * feat: add `--detail id` for `gcalcli agenda --tsv` * feat: add agendaupdate for title, id, location, description argparsers.py, cli.py: add `agendaupdate` subcommand gcal.py: add `AgendaUpdate()` details.py: - `Title`, `SimpleSingleFieldHandler`: add `patch()` - add `FIELD_HANDLERS` * feat: add agendaupdate for time Round trips don't work for all day events. The current output format is to present them as events starting and ending at midnight of the same day. But this keeps us from having events starting and ending at midnight of the same day. In the next commit I plan to change the output for all day events so the time fields are blank. Also: - change signature of `Handler.patch()` to include `cal`, which is necessary to extract the timezone - change default of `Handler.fieldnames` from `None` to `[]` to silence a mypy warning. * feat: add detail for `action` for `gcalcli agenda`. Column is ignored by `gcalcli agendaupdate`. * feat: add agendaupdate action dispatch and `patch` action Refactor most of existing agendaupdate into new module `gcalcli.actions` in `patch()`. * feat: add agendaupdate for calendar This treats calendar as a readonly field. Actually moving calendar would be a bit more involved, and would necessitate: - changing agendaupdate from its current one-calendar-only restriction - using the events().move() method to do the move. It's not done by events().patch() That can be a future feature. * feat: add agendaupdate `ignore` action * feat: add agendaupdate `delete` action * feat: add agendaupdate `insert` action * feat: `agendaupdate` patch action uses insert if no id supplied * feat: error when unsupported agendaupdate action used Also limits actions to a defined set. Before one could have specified action to be name of some other object in `gcalcli.actions` with unpredictable results. * feat: add agendaupdate for url Both these fields are read-only. If html_link is in the input, always fail. If hangout_link is in the input, check against the current value. If they don't match, fail. * feat: add agendaupdate for conference Partially implements insanum#522. * fix: change SimpleSingleColumnHandler.header to a simple list Can't use a property on a classmethod without making a custom metaclass. Without that, the property is only enacted upon class instantiation, which we aren't doing here. A custom metaclass is too magic, and thus far we have avoided instantiating what would be singleton objects. Also changed `SimpleSingleColumnHandler._get()` to use `cls.header[0]` instead of `cls._key` which is removed. * fix: gcalcli agenda to work with refactor 72710b4 * refactor: move _tsv() implementation to gcalcli.details Refactor `gcalcli.gcal.GoogleCalendarInterface._tsv()`, replacing the long list of if statements with a modular design using a series of `gcalcli.details.Handler` classes. Each `Handler` represents the set of event properties that are captured by each item of `gcalcli.argparsers.DETAILS`. So far only the `header` class attribute and the `get()` methods are implemented. This is what is necessary for TSV output. * refactor: derive `argparsers.DETAILS` from `details.HANDLERS` Set `gcalcli.argparsers.DETAILS` to a superset of `gcalcli.details.HANDLERS.keys()` to avoid duplication. * refactor: change `header` to `fieldnames` and `column` to `field` * refactor: change `Handler.patch()` to include `fieldname` argument This will enable `Handler`s with multiple fields to work for patching. Necessary for `Time.patch()` and `Conference.patch()` Also introduce `SingleFieldHandler.patch(cls,` ...`)` which dispatches to `cls._patch()`. * refactor: move gcal.GoogleCalendarInterface._isallday() to utils.is_all_day() This enables its use by details.py. * refactor: `details.Url` uses `details.URL_PROPS` to identify properties to get The new `details.URL_PROPS` constant `OrderedDict` will also work for use in `Url.patch()`, which should be implemented in the next commit. This will allow automated handling of two property/fieldname pairs for this `Handler`. * refactor: add `exceptions.ReadonlyError`, `exceptions.ReadonlyCheckError` * refactor: add `gcal.GoogleCalendarInterface.get_events()` change previous uses of `get_cal_service().events()` to `get_events()` * refactor: add `gcal.GoogleCalendarInterface.delete()` * refactor: add gcalcli.actions._iter_field_handlers() * style: add extra line of whitespace * ci: eliminate flake8-docstrings method docstring checks for details.py These are mostly overriding pure virtual classes and adding a docstring to each one would be superfluous and make the code harder to read.

michaelmhoffman · 2024-09-18T22:13:22Z

The main parts of this are done. I haven't closed the issue because each of the "Future extensions or subsequent refactoring" should either be (1) implemented, (2) made into issues for future discussion or implementation, or (3) decided as WONTFIX.

dbarnett · 2024-09-25T20:28:18Z

K, I've forked off some new issues for the loose ends:

I'll close out this base issue and we can follow up on subtasks in those or fresh bugs.

michaelmhoffman · 2024-11-16T14:13:32Z

Thank you @dbarnett for cleaning this up.

michaelmhoffman mentioned this issue Aug 2, 2020

Add agendaupdate subcommand #553

Merged

michaelmhoffman added a commit to michaelmhoffman/gcalcli that referenced this issue Aug 3, 2020

feat: add --detail id for gcalcli agenda --tsv

784d812

Partially implements insanum#550.

michaelmhoffman mentioned this issue Aug 8, 2020

gcalcli agendaupdate column for action #557

Open

5 tasks

michaelmhoffman added a commit to michaelmhoffman/gcalcli that referenced this issue Sep 6, 2020

feat: add agendaupdate for conference

f14e6f3

Partially implements insanum#522 and insanum#550.

michaelmhoffman mentioned this issue Jan 3, 2021

feat: add agendaupdate action column #577

Merged

michaelmhoffman mentioned this issue Sep 16, 2024

Release v4.4.0 #706

Closed

This was referenced Sep 25, 2024

agendaupdate should be able to create new events vs. just updating existing events #772

Closed

Refactor to use more helpers from details #773

Open

agendaupdate should support multiple calendars #774

Open

dbarnett closed this as completed Sep 25, 2024

dbarnett mentioned this issue Sep 25, 2024

Docs/examples for agendaupdate to make it more discoverable #776

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhancement: import agenda from CSV #550

Enhancement: import agenda from CSV #550

michaelmhoffman commented Jul 14, 2020 •

edited

Loading

michaelmhoffman commented Jul 27, 2020

jcrowgey commented Jul 28, 2020 via email

michaelmhoffman commented Aug 10, 2020

michaelmhoffman commented Aug 10, 2020 •

edited

Loading

michaelmhoffman commented Aug 15, 2020

michaelmhoffman commented Sep 18, 2024

dbarnett commented Sep 25, 2024 •

edited

Loading

michaelmhoffman commented Nov 16, 2024

Enhancement: import agenda from CSV #550

Enhancement: import agenda from CSV #550

Comments

michaelmhoffman commented Jul 14, 2020 • edited Loading

michaelmhoffman commented Jul 27, 2020

jcrowgey commented Jul 28, 2020 via email

michaelmhoffman commented Aug 10, 2020

michaelmhoffman commented Aug 10, 2020 • edited Loading

michaelmhoffman commented Aug 15, 2020

michaelmhoffman commented Sep 18, 2024

dbarnett commented Sep 25, 2024 • edited Loading

michaelmhoffman commented Nov 16, 2024

michaelmhoffman commented Jul 14, 2020 •

edited

Loading

michaelmhoffman commented Aug 10, 2020 •

edited

Loading

dbarnett commented Sep 25, 2024 •

edited

Loading