Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql: Add telemetry counters for succeeded, failed, canceled jobs #59711

Closed
vy-ton opened this issue Feb 2, 2021 · 0 comments · Fixed by #61482
Closed

sql: Add telemetry counters for succeeded, failed, canceled jobs #59711

vy-ton opened this issue Feb 2, 2021 · 0 comments · Fixed by #61482
Assignees
@fqazi
Labels
A-telemetry C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@vy-ton
Copy link
Contributor

vy-ton commented Feb 2, 2021 •
edited by jordanlewis
Loading

With features telemetry, there are counters for different SQL commands, such as ALTER TABLE or ALTER TABLE...ADD COLUMN. For schema changes, we should add counters for when they succeed, fail, or are canceled.

As a first pass, we can start with a counter per job type - meaning, schema changes / CDC /backup / etc.

Jobs can also have pending, running, or paused statuses. Since those are transient statuses, they are not as important to capture in telemetry data.

Once this is closed, please add a new issue for grouping these telemetry counters by schema change type.
@vy-ton vy-ton added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-telemetry labels Feb 2, 2021
@jordanlewis jordanlewis changed the title sql: Add telemetry counters for succeeded, failed, canceled schema changes sql: Add telemetry counters for succeeded, failed, canceled jobs Mar 2, 2021
@fqazi fqazi mentioned this issue Mar 4, 2021
Merged
fqazi added a commit to fqazi/cockroach that referenced this issue Mar 4, 2021
@fqazi
jobs: add job metrics per-type to track success, failure, and cancel
a5df1c8 
Fixes: cockroachdb#59711

Previously, there were only over all counters tracking how many
jobs were completed, cancelled, or failed. This was inadequate
because it didn't make it easy to tell in aggregate what job
types they were. To address this, this patch will add counters
for different job types for tracking success, failure, and
cancellation.

Release justification: Low risk change only adding a metric inside
the crdb_internal.feature_usage table
Release note: None
fqazi added a commit to fqazi/cockroach that referenced this issue Mar 5, 2021
@fqazi
jobs: add job metrics per-type to track success, failure, and cancel
ca37efc 
Fixes: cockroachdb#59711

Previously, there were only over all counters tracking how many
jobs were completed, cancelled, or failed. This was inadequate
because it didn't make it easy to tell in aggregate what job
types they were. To address this, this patch will add counters
for different job types for tracking success, failure, and
cancellation.

Release justification: Low risk change only adding a metric inside
the crdb_internal.feature_usage table
Release note: None
fqazi added a commit to fqazi/cockroach that referenced this issue Mar 5, 2021
@fqazi
jobs: add job metrics per-type to track success, failure, and cancel
a81afb0 
Fixes: cockroachdb#59711

Previously, there were only over all counters tracking how many
jobs were completed, cancelled, or failed. This was inadequate
because it didn't make it easy to tell in aggregate what job
types they were. To address this, this patch will add counters
for different job types for tracking success, failure, and
cancellation.

Release justification: Low risk change only adding a metric inside
the crdb_internal.feature_usage table
Release note: None
craig bot pushed a commit that referenced this issue Mar 5, 2021
@irfansharif @fqazi
@RaduBerinde @mgartner @stevendanna
Merge #61359 #61482 #61491 #61509 #61522 #61553
2264789 
61359: tracing: use byte-limits for logs/structured events per span r=irfansharif a=irfansharif

Touches #59188. Follow-on work from #60678. We can introduce byte-limits for
verbose logging and structured events, instead of limiting things based on
count.

This PR also:
- adds a _dropped tag to recordings with dropped logs/structured events.
- squashes a bug where reset spans (as used in SessionTracing) still
  held onto earlier structured events
- moves away from the internal usage of the opentracing.LogRecord type,
  it's unnecessary

Release justification: low risk, high benefit changes to existing
functionality

Release note: None

---

+cc @knz / @erikgrinaker / @angelapwen for pod-visibility.

61482: jobs: add job metrics per-type to track success, failure, and cancel r=fqazi a=fqazi

Fixes: #59711

Previously, there were only over all counters tracking how many
jobs were completed, cancelled, or failed. This was inadequate
because it didn't make it easy to tell in aggregate what job
types they were. To address this, this patch will add counters
for different job types for tracking success, failure, and
cancellation.

Release justification: Low risk change only adding a metric inside
the crdb_internal.feature_usage table
Release note: None

61491: sqlsmith: add support for computed columns r=RaduBerinde a=RaduBerinde

This changes the random table generator to also create computed
columns (either STORED or VIRTUAL). Some example of definitions:
 - `col1_14 STRING NOT NULL AS (lower(CAST(col1_8 AS STRING))) VIRTUAL`
 - `col2_6 DECIMAL NOT NULL AS (col2_2 + 1:::DECIMAL) STORED`
 - `col1_13 INT4 AS (col1_0 + col1_10) STORED`

Release justification: non-production code change.

Release note: None

Informs #57608.

61509: sql: add a regression test r=RaduBerinde a=RaduBerinde

This commit adds a regression test for #58104 (the problem was already
fixed).

Resolves #58104.

Release justification: non-production code change.

Release note: None

61522: opt: fix fetch scope in UPDATE..FROM statements r=mgartner a=mgartner

Previously, the fetch scope incorrectly included columns in the FROM
clause of an UPDATE..FROM statement. As a result, column names shared by
the FROM clause and the mutating table lead to ambiguity when resolving
partial index DEL column expressions. This commit ensures that the fetch
scope does not include columns in the FROM clause.

Fixes #61284

Release justification: This is a low-risk bug fix to existing
functionality.

Release note (bug fix): An UPDATE..FROM statement where the FROM clause
contained column names that match table column names erred if the table
had a partial index predicate referencing those columns. This bug,
present since partial indexes were released in version 20.2.0, has been
fixed.

61553: ccl,server: error.Wrap on previously handled errors r=[dt,miretskiy] a=stevendanna

These errors.Wrap calls are wrapping errors that are nil and thus will
always return a nil error.

Release justification: Minor error handling fixes
Release note: None

Co-authored-by: irfan sharif <irfanmahmoudsharif@gmail.com>
Co-authored-by: Faizan Qazi <faizan@cockroachlabs.com>
Co-authored-by: Radu Berinde <radu@cockroachlabs.com>
Co-authored-by: Marcus Gartner <marcus@cockroachlabs.com>
Co-authored-by: Steven Danna <danna@cockroachlabs.com>
@craig craig bot closed this as completed in 1b39a7f Mar 5, 2021
@craig craig bot closed this as completed in #61482 Mar 5, 2021
@fqazi fqazi mentioned this issue Mar 5, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fqazi fqazi

Labels
A-telemetry C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

jobs: add job metrics per-type to track success, failure, and cancel fqazi/cockroach
2 participants
@fqazi @vy-ton