Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More caching is required #6

Open
SphericalKat opened this issue Dec 2, 2019 · 1 comment
Open

More caching is required #6

SphericalKat opened this issue Dec 2, 2019 · 1 comment
Assignees
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@SphericalKat
Copy link
Collaborator

SphericalKat commented Dec 2, 2019

We require caching for all DB stuff if we're gonna scale. Need some help doing this.
I also wouldn't mind making volunteers collaborators for this project.

@SphericalKat SphericalKat added enhancement New feature or request help wanted Extra attention is needed labels Dec 2, 2019
@SphericalKat SphericalKat self-assigned this Dec 3, 2019
@yurinnick
Copy link

I would suggest moving to a different DB ORM driver first since gorm considered as very slow and inefficient for a reason.

Steps to reproduce:

  1. $ docker run --name some-postgres -e POSTGRES_USER=bench -e POSTGRES_PASSWORD=pass -e POSTGRES_DB=benchdb -p 5432:5432 -d postgres
  2. Build and run github.com/derkan/gobenchorm

Reports: 

  2000 times - Insert
       raw:     2.74s      1369317 ns/op     720 B/op     21 allocs/op
       qbs:     2.79s      1393293 ns/op    5698 B/op    126 allocs/op
 beego_orm:     2.95s      1474103 ns/op    2427 B/op     58 allocs/op
       dbr:     2.95s      1475117 ns/op    2982 B/op     75 allocs/op
        pg:     3.21s      1604382 ns/op    1013 B/op     10 allocs/op
      modl:     3.85s      1925094 ns/op    1702 B/op     46 allocs/op
      xorm:     3.92s      1958806 ns/op    3290 B/op    108 allocs/op
      hood:     4.07s      2032934 ns/op    7187 B/op    183 allocs/op
      gorp:     4.08s      2038453 ns/op    1702 B/op     47 allocs/op
      sqlx:     4.13s      2063763 ns/op    2352 B/op     55 allocs/op
      godb:     4.21s      2104967 ns/op    4746 B/op    118 allocs/op
      gorm:     6.06s      3031673 ns/op    8140 B/op    185 allocs/op
    genmai:     8.66s      4330461 ns/op    4616 B/op    159 allocs/op
     db.v3:    15.76s      7880378 ns/op   36688 B/op   1625 allocs/op
       pop:     0.00s      1.35 ns/op       0 B/op      0 allocs/op

   500 times - BulkInsert 100 row
       dbr:     0.01s        20492 ns/op    2081 B/op     39 allocs/op
 beego_orm:     1.64s      3285343 ns/op  186967 B/op   2845 allocs/op
       raw:     1.84s      3671072 ns/op  140489 B/op   1423 allocs/op
        pg:     1.88s      3760384 ns/op   15574 B/op    213 allocs/op
      xorm:     2.36s      4715292 ns/op  304202 B/op   7795 allocs/op
      godb:     2.48s      4967324 ns/op  290630 B/op   6096 allocs/op
    genmai:     2.49s      4988987 ns/op  209408 B/op   3423 allocs/op
     db.v3:     4.69s      9376891 ns/op  481083 B/op  20006 allocs/op
      sqlx:     benchmark not implemeted yet - https://github.com/jmoiron/sqlx/issues/134
      gorm:     Don't support bulk insert - https://github.com/jinzhu/gorm/issues/255
       pop:     Problematic bulk insert, too slow
       qbs:     Don't support bulk insert, err driver: bad connection
      hood:     Problematic bulk insert, too slow
      modl:     Don't support bulk insert
      gorp:     Problematic bulk insert, too slow

  2000 times - Update
       raw:     0.99s       493008 ns/op     728 B/op     21 allocs/op
       dbr:     1.07s       533120 ns/op    2627 B/op     58 allocs/op
 beego_orm:     2.56s      1278918 ns/op    1809 B/op     49 allocs/op
        pg:     2.90s      1449887 ns/op     808 B/op     11 allocs/op
      modl:     3.79s      1894455 ns/op    1320 B/op     43 allocs/op
      xorm:     3.81s      1905579 ns/op    3313 B/op    134 allocs/op
      godb:     3.93s      1966325 ns/op    5411 B/op    158 allocs/op
      gorp:     3.94s      1968135 ns/op    1368 B/op     42 allocs/op
       qbs:     4.06s      2029909 ns/op    5932 B/op    153 allocs/op
      sqlx:     4.32s      2159638 ns/op    1056 B/op     26 allocs/op
    genmai:     4.91s      2454522 ns/op    3635 B/op    157 allocs/op
      hood:     5.15s      2573940 ns/op   13091 B/op    207 allocs/op
      gorm:     5.96s      2982313 ns/op    8215 B/op    208 allocs/op
     db.v3:    16.87s      8434619 ns/op   41694 B/op   1917 allocs/op
       pop:     0.00s      2.30 ns/op       0 B/op      0 allocs/op

  4000 times - Read
 beego_orm:     1.99s       496539 ns/op    2145 B/op     80 allocs/op
       raw:     2.02s       504116 ns/op     920 B/op     28 allocs/op
      sqlx:     2.05s       512452 ns/op    1776 B/op     41 allocs/op
       dbr:     2.10s       524550 ns/op    2176 B/op     36 allocs/op
        pg:     2.17s       542181 ns/op     993 B/op     16 allocs/op
      gorp:     2.57s       642300 ns/op    3984 B/op    191 allocs/op
      modl:     4.14s      1033755 ns/op    1824 B/op     46 allocs/op
      xorm:     4.33s      1081818 ns/op    8919 B/op    262 allocs/op
      gorm:     4.36s      1091192 ns/op    7411 B/op    157 allocs/op
     db.v3:     4.46s      1115106 ns/op    6692 B/op    286 allocs/op
      godb:     4.56s      1139612 ns/op    4242 B/op    107 allocs/op
    genmai:     5.91s      1476869 ns/op    3345 B/op    174 allocs/op
      hood:     reflect: call of reflect.Value.Bytes on string Value
       pop:     0.00s      0.82 ns/op       0 B/op      0 allocs/op
       qbs:     reflect: call of reflect.Value.Bytes on string Value

  2000 times - MultiRead limit 1000
       raw:     6.51s      3254673 ns/op  282842 B/op  12912 allocs/op
        pg:     6.78s      3392484 ns/op  246021 B/op   6025 allocs/op
      modl:     8.32s      4161184 ns/op  353245 B/op  17930 allocs/op
      sqlx:     9.07s      4532737 ns/op  330116 B/op  14946 allocs/op
 beego_orm:     9.21s      4604459 ns/op  586776 B/op  34983 allocs/op
       dbr:     9.30s      4647836 ns/op  534243 B/op  17960 allocs/op
      gorp:     9.60s      4801609 ns/op  470963 B/op  16109 allocs/op
     db.v3:     9.67s      4833362 ns/op  462990 B/op  15289 allocs/op
    genmai:    10.86s      5427716 ns/op  431664 B/op  14102 allocs/op
      godb:    12.94s      6468530 ns/op  828594 B/op  32993 allocs/op
      xorm:    20.67s     10333918 ns/op 1277765 B/op  57114 allocs/op
      gorm:    23.11s     11555336 ns/op 2425411 B/op  61037 allocs/op
      hood:     reflect: call of reflect.Value.Bytes on string Value
       pop:     0.00s      2.15 ns/op       0 B/op      0 allocs/op
       qbs:     reflect: call of reflect.Value.Bytes on string Value

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants