Avoid Sync message overflow #3389
Conversation
PBENCH-1120 A SQL error was observed in deployment where `pbench-index` logged an error on the `INDEX` sync object because a tarball was somehow not present. The message string generated by `indexing_tarballs.py` exceeded the `VARCHAR(255)` column specification. This isn't an attempt to address the root problem, but to address the symptom of overloading the operation table message column in the future so at least errors are properly recorded. This reworks some of the `indexing_tarballs.py` messages to avoid redundancy (e.g., naming the dataset or tarball isn't necessary as the records are linked to the `Dataset`), but also removes the limit on the message column as a precaution.
|
Hey @dbutenhof just to understand this fix. We have created a separate column for the messages which doesn't have any limit, right? unrelated to this change where were the messages stored earlier? |
All this PR does is expand the existing column, and "tweak" some of the messages to avoid unnecessarily long values. The column isn't added, or moved, and none of the When I added the |
PBENCH-1120 A SQL error was observed in deployment where `pbench-index` logged an error on the `INDEX` sync object because a tarball was somehow not present. The message string generated by `indexing_tarballs.py` exceeded the `VARCHAR(255)` column specification. This isn't an attempt to address the root problem, but to address the symptom of overloading the operation table message column in the future so at least errors are properly recorded. This reworks some of the `indexing_tarballs.py` messages to avoid redundancy (e.g., naming the dataset or tarball isn't necessary as the records are linked to the `Dataset`), but also removes the limit on the message column as a precaution.
* Avoid Sync message overflow (#3389) PBENCH-1120 A SQL error was observed in deployment where `pbench-index` logged an error on the `INDEX` sync object because a tarball was somehow not present. The message string generated by `indexing_tarballs.py` exceeded the `VARCHAR(255)` column specification. This isn't an attempt to address the root problem, but to address the symptom of overloading the operation table message column in the future so at least errors are properly recorded. This reworks some of the `indexing_tarballs.py` messages to avoid redundancy (e.g., naming the dataset or tarball isn't necessary as the records are linked to the `Dataset`), but also removes the limit on the message column as a precaution.
PBENCH-1120
A SQL error was observed in deployment where
pbench-indexlogged an error on theINDEXsync object because a tarball was somehow not present. The message string generated byindexing_tarballs.pyexceeded theVARCHAR(255)column specification.This isn't an attempt to address the root problem, but to address the symptom of overloading the operation table message column in the future so at least errors are properly recorded.
This reworks some of the
indexing_tarballs.pymessages to avoid redundancy (e.g., naming the dataset or tarball isn't necessary as the records are linked to theDataset), but also removes the limit on the message column as a precaution.(NOTE: it also adds some unit test cases, although these are more documentation than "real tests" as sqlite3, unlike PostgreSQL, doesn't implement column limits.)
Resolves #3366