Skip to content
This repository has been archived by the owner on Mar 11, 2021. It is now read-only.

Improve migration 110 #2319

Merged
merged 2 commits into from
Oct 15, 2018
Merged

Conversation

kwk
Copy link
Collaborator

@kwk kwk commented Oct 15, 2018

Use Postgres CTE to update iteration numbers

When explaining the updae before it looks like this:

postgres@172:postgres> explain UPDATE iterations iter SET number = seq.row_number
                       FROM (
                           SELECT id, space_id, created_at, row_number() OVER (PARTITION BY space_id ORDER BY created_at ASC)
                           FROM iterations
                       ) AS seq
                       WHERE iter.space_id = seq.space_id AND iter.id = seq.id;
+-----------------------------------------------------------------------------------------+
| QUERY PLAN                                                                              |
|-----------------------------------------------------------------------------------------|
| Update on iterations iter  (cost=2.06..2.16 rows=1 width=263)                           |
|   ->  Merge Join  (cost=2.06..2.16 rows=1 width=263)                                    |
|         Merge Cond: (seq.space_id = iter.space_id)                                      |
|         Join Filter: (iter.id = seq.id)                                                 |
|         ->  Subquery Scan on seq  (cost=1.03..1.09 rows=2 width=112)                    |
|               ->  WindowAgg  (cost=1.03..1.07 rows=2 width=48)                          |
|                     ->  Sort  (cost=1.03..1.03 rows=2 width=40)                         |
|                           Sort Key: iterations.space_id, iterations.created_at          |
|                           ->  Seq Scan on iterations  (cost=0.00..1.02 rows=2 width=40) |
|         ->  Sort  (cost=1.03..1.03 rows=2 width=187)                                    |
|               Sort Key: iter.space_id                                                   |
|               ->  Seq Scan on iterations iter  (cost=0.00..1.02 rows=2 width=187)       |
+-----------------------------------------------------------------------------------------+

Now the update looks better (notice absence of nested seq scans):

postgres@172:postgres> explain WITH iteration_numbers AS (
                           SELECT *, ROW_NUMBER() OVER(PARTITION BY space_id ORDER BY created_at ASC) AS num
                           FROM iterations
                       )
                       UPDATE iterations SET number = (SELECT num FROM iteration_numbers WHERE iteration_numbers.id = iterations.id);
+---------------------------------------------------------------------------------------------+
| QUERY PLAN                                                                                  |
|---------------------------------------------------------------------------------------------|
| Update on iterations  (cost=1.07..2.18 rows=2 width=191)                                    |
|   CTE iteration_numbers                                                                     |
|     ->  WindowAgg  (cost=1.03..1.07 rows=2 width=193)                                       |
|           ->  Sort  (cost=1.03..1.03 rows=2 width=185)                                      |
|                 Sort Key: iterations_1.space_id, iterations_1.created_at                    |
|                 ->  Seq Scan on iterations iterations_1  (cost=0.00..1.02 rows=2 width=185) |
|   ->  Seq Scan on iterations  (cost=0.00..1.11 rows=2 width=191)                            |
|         SubPlan 2                                                                           |
|           ->  CTE Scan on iteration_numbers  (cost=0.00..0.04 rows=1 width=8)               |
|                 Filter: (id = iterations.id)                                                |
+---------------------------------------------------------------------------------------------+

@alien-ike
Copy link

Ike Plugins (test-keeper)

Thank you @kwk for this contribution!

It appears that no tests have been added or updated in this PR.

Automated tests give us confidence in shipping reliable software. Please add some as part of this change.

If you are an admin or the reviewer of this PR and you are sure that no test is needed then you can use the command /ok-without-tests as a comment to make the status green.

For more information please head over to official documentation. You can find there how to configure the plugin.

When explaining the updae before it looks like this:

```
postgres@172:postgres> explain UPDATE iterations iter SET number = seq.row_number
                       FROM (
                           SELECT id, space_id, created_at, row_number() OVER (PARTITION BY space_id ORDER BY created_at ASC)
                           FROM iterations
                       ) AS seq
                       WHERE iter.space_id = seq.space_id AND iter.id = seq.id;
+-----------------------------------------------------------------------------------------+
| QUERY PLAN                                                                              |
|-----------------------------------------------------------------------------------------|
| Update on iterations iter  (cost=2.06..2.16 rows=1 width=263)                           |
|   ->  Merge Join  (cost=2.06..2.16 rows=1 width=263)                                    |
|         Merge Cond: (seq.space_id = iter.space_id)                                      |
|         Join Filter: (iter.id = seq.id)                                                 |
|         ->  Subquery Scan on seq  (cost=1.03..1.09 rows=2 width=112)                    |
|               ->  WindowAgg  (cost=1.03..1.07 rows=2 width=48)                          |
|                     ->  Sort  (cost=1.03..1.03 rows=2 width=40)                         |
|                           Sort Key: iterations.space_id, iterations.created_at          |
|                           ->  Seq Scan on iterations  (cost=0.00..1.02 rows=2 width=40) |
|         ->  Sort  (cost=1.03..1.03 rows=2 width=187)                                    |
|               Sort Key: iter.space_id                                                   |
|               ->  Seq Scan on iterations iter  (cost=0.00..1.02 rows=2 width=187)       |
+-----------------------------------------------------------------------------------------+
```

Now the update looks better (notice absence of nested seq scans):

```
postgres@172:postgres> explain WITH iteration_numbers AS (
                           SELECT *, ROW_NUMBER() OVER(PARTITION BY space_id ORDER BY created_at ASC) AS num
                           FROM iterations
                       )
                       UPDATE iterations SET number = (SELECT num FROM iteration_numbers WHERE iteration_numbers.id = iterations.id);
+---------------------------------------------------------------------------------------------+
| QUERY PLAN                                                                                  |
|---------------------------------------------------------------------------------------------|
| Update on iterations  (cost=1.07..2.18 rows=2 width=191)                                    |
|   CTE iteration_numbers                                                                     |
|     ->  WindowAgg  (cost=1.03..1.07 rows=2 width=193)                                       |
|           ->  Sort  (cost=1.03..1.03 rows=2 width=185)                                      |
|                 Sort Key: iterations_1.space_id, iterations_1.created_at                    |
|                 ->  Seq Scan on iterations iterations_1  (cost=0.00..1.02 rows=2 width=185) |
|   ->  Seq Scan on iterations  (cost=0.00..1.11 rows=2 width=191)                            |
|         SubPlan 2                                                                           |
|           ->  CTE Scan on iteration_numbers  (cost=0.00..0.04 rows=1 width=8)               |
|                 Filter: (id = iterations.id)                                                |
+---------------------------------------------------------------------------------------------+
```
@kwk kwk force-pushed the improve-migration-110 branch from afc88d3 to 4e4e2ee Compare October 15, 2018 11:21
@codecov
Copy link

codecov bot commented Oct 15, 2018

Codecov Report

Merging #2319 into master will increase coverage by 0.03%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2319      +/-   ##
==========================================
+ Coverage   69.99%   70.03%   +0.03%     
==========================================
  Files         171      171              
  Lines       16610    16610              
==========================================
+ Hits        11626    11632       +6     
+ Misses       3861     3854       -7     
- Partials     1123     1124       +1
Impacted Files Coverage Δ
workitem/workitem_repository.go 67.79% <0%> (-0.23%) ⬇️
remoteworkitem/scheduler.go 60.97% <0%> (+7.31%) ⬆️
remoteworkitem/jira.go 100% <0%> (+25%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 37a913a...19d48f5. Read the comment docs.

@kwk kwk requested a review from jarifibrahim October 15, 2018 12:13
@jarifibrahim
Copy link
Member

/ok-without-tests

@kwk kwk merged commit e09b805 into fabric8-services:master Oct 15, 2018
@kwk kwk deleted the improve-migration-110 branch October 15, 2018 12:44
kwk added a commit to kwk/fabric8-wit that referenced this pull request Oct 23, 2018
kwk added a commit that referenced this pull request Oct 23, 2018
Currently the database migration 110 won't go through on prod-preview. I'm postponing it for now.

* Reverts "update number for existing iterations (#2314)"
   * This reverts commit 8674b7b.
* Revert "Use Postgres CTE to update iteration numbers (#2319)"
  * This reverts commit e09b805.
kwk pushed a commit to openshiftio/saas-openshiftio that referenced this pull request Oct 30, 2018
# Changes

**Commit:** fabric8-services/fabric8-wit@11904c7
**Author:** Konrad Kleine (193408+kwk@users.noreply.github.com)
**Date:** 2018-10-10T13:49:39+02:00

Assign number to new areas/iterations and allow searching by it (fabric8-services/fabric8-wit#2311)

----

**Commit:** fabric8-services/fabric8-wit@8674b7b
**Author:** Konrad Kleine (193408+kwk@users.noreply.github.com)
**Date:** 2018-10-10T16:58:17+02:00

update number for existing iterations (fabric8-services/fabric8-wit#2314)

----

**Commit:** fabric8-services/fabric8-wit@e67b5e2
**Author:** nurali-techie (nurali.techie@gmail.com)
**Date:** 2018-10-15T14:00:33+05:30

stop calling analytics api for cve scan reg/de-reg (fabric8-services/fabric8-wit#2316)

----

**Commit:** fabric8-services/fabric8-wit@37a913a
**Author:** Baiju Muthukadan (baiju.m.mail@gmail.com)
**Date:** 2018-10-15T17:07:20+05:30

Include fabric8-common as a dependency (fabric8-services/fabric8-wit#2318)

----

**Commit:** fabric8-services/fabric8-wit@e09b805
**Author:** Konrad Kleine (193408+kwk@users.noreply.github.com)
**Date:** 2018-10-15T14:44:35+02:00

Use Postgres CTE to update iteration numbers (fabric8-services/fabric8-wit#2319)

----

**Commit:** fabric8-services/fabric8-wit@1724c15
**Author:** Ibrahim Jarif (jarifibrahim@gmail.com)
**Date:** 2018-10-23T10:12:56+05:30

Rearrage search repository blackbox tests (fabric8-services/fabric8-wit#2322)

----

**Commit:** fabric8-services/fabric8-wit@7fda6af
**Author:** Ibrahim Jarif (jarifibrahim@gmail.com)
**Date:** 2018-10-23T13:49:11+05:30

Make agile default template for space creation (fabric8-services/fabric8-wit#2323)

----

**Commit:** fabric8-services/fabric8-wit@7b78cf6
**Author:** Konrad Kleine (193408+kwk@users.noreply.github.com)
**Date:** 2018-10-23T11:54:01+02:00

Postpone iteration number update (fabric8-services/fabric8-wit#2325)

----

**Commit:** fabric8-services/fabric8-wit@22e3d5d
**Author:** Dhriti Shikhar (dhriti.shikhar.rokz@gmail.com)
**Date:** 2018-10-23T16:06:35+05:30

Enable the delete workitem endpoint (fabric8-services/fabric8-wit#2305)

----

**Commit:** fabric8-services/fabric8-wit@3d163c7
**Author:** Ibrahim Jarif (jarifibrahim@gmail.com)
**Date:** 2018-10-23T17:45:11+05:30

Increase deployment timeout to 2X (fabric8-services/fabric8-wit#2327)

----

**Commit:** fabric8-services/fabric8-wit@9879836
**Author:** Ibrahim Jarif (jarifibrahim@gmail.com)
**Date:** 2018-10-24T11:30:20+05:30

Do not throw 500 on invalid value type in search API (fabric8-services/fabric8-wit#2321)

----

**Commit:** fabric8-services/fabric8-wit@44739f8
**Author:** Ibrahim Jarif (jarifibrahim@gmail.com)
**Date:** 2018-10-25T10:57:03+05:30

Remove redundant code and fix typo (fabric8-services/fabric8-wit#2329)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants