Skip to content

WIP: Port crates#index to use Diesel #600

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,11 @@ rustc-serialize = "0.3"
license-exprs = "^1.3"
dotenv = "0.8.0"
toml = "0.2"
diesel = { version = "0.11.0", features = ["postgres", "serde_json"] }
diesel = { version = "0.11.0", features = ["postgres", "serde_json", "deprecated-time"] }
diesel_codegen = { version = "0.11.0", features = ["postgres"] }
r2d2-diesel = "0.11.0"
diesel_full_text_search = "0.11.0"
serde_json = "0.9.0"

conduit = "0.8"
conduit-conditional-get = "0.8"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ALTER TABLE versions ALTER COLUMN yanked DROP NOT NULL;
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ALTER TABLE versions ALTER COLUMN yanked SET NOT NULL;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 👍 👍 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inb4 prod has a ton of rows where this is null

3 changes: 2 additions & 1 deletion src/app.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@ impl App {
.helper_threads(if config.env == ::Env::Production {3} else {1})
.build();
let diesel_db_config = r2d2::Config::builder()
.pool_size(if config.env == ::Env::Production {1} else {1})
.pool_size(if config.env == ::Env::Production {50} else {1})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This 50 is going to have to come down as we discovered yesterday

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought prod had 120 connections available? I told it to not try to keep more than 5 idles at a time so it should never get bad on staging.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see. I thought this would mean I couldn't deploy it to staging.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait... min_idle means "not more than 5 idles"? That sounds like max_idles to me...?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, r2d2's naming/behavior/defaults are all really questionable here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could totally take down staging by sending 20 concurrent requests though, but I figured that's not a huge concern.

.min_idle(if config.env == ::Env::Production {5} else {1})
.helper_threads(if config.env == ::Env::Production {3} else {1})
.build();

Expand Down
21 changes: 18 additions & 3 deletions src/badge.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
use util::CargoResult;
use krate::Crate;
use Model;
use krate::Crate;
use schema::badges;
use util::CargoResult;

use std::collections::HashMap;
use diesel::prelude::*;
use diesel::pg::Pg;
use pg::GenericConnection;
use pg::rows::Row;
use rustc_serialize::json::Json;
use serde_json;
use std::collections::HashMap;

#[derive(Debug, PartialEq, Clone)]
pub enum Badge {
Expand All @@ -26,6 +30,17 @@ pub struct EncodableBadge {
pub attributes: HashMap<String, String>,
}

impl Queryable<badges::SqlType, Pg> for Badge {
type Row = (i32, String, serde_json::Value);

fn build((_, badge_type, attributes): Self::Row) -> Self {
let attributes = serde_json::from_value::<HashMap<String, String>>(attributes)
.expect("attributes was not a map in the database");
Self::from_attributes(&badge_type, &attributes)
.expect("invalid badge in the database")
}
}

impl Model for Badge {
fn from_row(row: &Row) -> Badge {
let attributes: Json = row.get("attributes");
Expand Down
14 changes: 13 additions & 1 deletion src/category.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,13 @@ use pg::rows::Row;

use {Model, Crate};
use db::RequestTransaction;
use schema::*;
use util::{RequestUtils, CargoResult, ChainError};
use util::errors::NotFound;

#[derive(Clone)]
#[derive(Clone, Identifiable, Associations)]
#[has_many(crates_categories)]
#[table_name="categories"]
pub struct Category {
pub id: i32,
pub category: String,
Expand All @@ -21,6 +24,15 @@ pub struct Category {
pub crates_cnt: i32,
}

#[derive(Associations)]
#[belongs_to(Category)]
#[table_name="crates_categories"]
#[allow(dead_code)] // FIXME: Hey @sgrif add better many-to-many support to Diesel, wouldya?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

struct CrateCategory {
crate_id: i32,
category_id: i32,
}

#[derive(RustcEncodable, RustcDecodable)]
pub struct EncodableCategory {
pub id: String,
Expand Down
13 changes: 12 additions & 1 deletion src/keyword.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,28 @@ use pg::rows::Row;

use {Model, Crate};
use db::RequestTransaction;
use schema::*;
use util::{RequestUtils, CargoResult, ChainError, internal};
use util::errors::NotFound;

#[derive(Clone)]
#[derive(Clone, Identifiable, Associations)]
#[has_many(crates_keywords)]
pub struct Keyword {
pub id: i32,
pub keyword: String,
pub created_at: Timespec,
pub crates_cnt: i32,
}

#[derive(Associations)]
#[belongs_to(Keyword)]
#[table_name="crates_keywords"]
#[allow(dead_code)]
struct CrateKeyword {
crate_id: i32,
keyword_id: i32,
}

#[derive(RustcEncodable, RustcDecodable)]
pub struct EncodableKeyword {
pub id: String,
Expand Down
Loading