-
Notifications
You must be signed in to change notification settings - Fork 218
Remove DocBuilder, part 1 #1016
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Those options aren't used at all anywhere in the codebase, so it's safe to remove them.
As far as I'm aware this flag was never used since the Rust project started to take care of docs.rs, and it's not much useful. Basically, every time a build happened the DocBuilder recorded the crate name and version in a text file, and the flag prevented a build if the related names and versions were present in such file. Checking if the build is successful in the database is more reliable.
| debug!("10 builds in a row; pinging pubsubhubhub"); | ||
| status = BuilderState::QueueInProgress(0); | ||
|
|
||
| match pubsubhubbub::ping_hubs() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't know we had this 😆 do we know anyone using this? I'd rather have #809 I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Dunno why it exists, but let's not remove it for now until we get more insights on who's using it.
| "SELECT 1 FROM crates, releases, builds | ||
| WHERE crates.id = releases.crate_id AND releases.id = builds.rid |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs an INNER JOIN or you'll get duplicates (the id won't match the crate that succeeded).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm? This is working as expected when testing manually. Why would you expect it to fail?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Huh, I expected it to fail when you have three different crates, each of which satisfies one of these conditions.
CREATE TEMPORARY TABLE crates (
id SERIAL PRIMARY KEY,
name VARCHAR(255) UNIQUE NOT NULL
);
CREATE TEMPORARY TABLE releases (
id SERIAL PRIMARY KEY,
crate_id INT NOT NULL REFERENCES crates(id),
version VARCHAR(100)
);
CREATE TEMPORARY TABLE builds (
id SERIAL,
rid INT NOT NULL REFERENCES releases(id),
build_status BOOL NOT NULL
);
insert into crates (name) values ('a'), ('b');
insert into releases (crate_id, version) values (1, '0.1.0'), (2, '0.1.0');
insert into builds (rid, build_status) values (1, true), (2, true);
SELECT 1 FROM crates, releases, builds
WHERE crates.id = releases.crate_id AND releases.id = builds.rid
AND crates.name = 'a' AND releases.version = '0.1.0' AND builds.build_status = TRUE;
But that doesn't happen - the WHERE id = crate_id AND id = rid filters out the duplicates. It's more clear when you select everything instead of 1:
select * from crates, releases, builds WHERE crates.id = releases.crate_id AND releases.id = builds.rid;
id | name | id | crate_id | version | id | rid | build_status
----+------+----+----------+---------+----+-----+--------------
1 | a | 1 | 1 | 0.1.0 | 1 | 1 | f
2 | b | 2 | 2 | 0.1.0 | 2 | 2 | t
joshua=> select * from crates, releases, builds WHERE crates.name = 'a' AND releases.version = '0.1.0' AND builds.build_status = TRUE;
id | name | id | crate_id | version | id | rid | build_status
----+------+----+----------+---------+----+-----+--------------
1 | a | 1 | 1 | 0.1.0 | 2 | 2 | t
1 | a | 2 | 2 | 0.1.0 | 2 | 2 | t
joshua=> select * from crates, releases, builds WHERE crates.name = 'a' AND releases.version = '0.1.0' AND builds.build_status = TRUE AND crates.id = releases.crate_id AND releases.id = builds.rid;
id | name | id | crate_id | version | id | rid | build_status
----+------+----+----------+---------+----+-----+--------------
(0 rows)
So this works, it's just confusing.
Co-authored-by: Joshua Nelson <joshua@yottadb.com>
This PR is the first of the two PRs I plan to send to remove
DocBuilderandDocBuilderOptionsfrom the docs.rs codebase: both those structs serve as "intermediaries" betweenRustwideBuilderand the rest of the application, but now that we haveConfigandContextthat's not really needed anymore. Not having the middle layer will help with building buildtests a lot.Notable changes:
cratesfyi build --skip-if-log-exists <command>was removed: the--skip-if-log-existsflag was really fragile, relying on a text log of all the builds updated every 10 builds. There is also no need I can see for it, as the practically equivalent (and better)--skip-if-existsalready exists.cratesfyi build --skip-if-exists <command>was improved: now it checks if a successful build is present, instead of checking whether the release metadata is stored in the database.RustwideBuilderconfiguration was moved over toConfig.DocBuilderOptionswas moved over toConfig.Indexis now stored inContextand a single instance is reused across the whole application. This won't affect the file descriptors problem, as the underlying git instance is still re-created every time it is accessed.In the next PR I'll move away all the rest of
DocBuilder. This PR is best reviewed commit-by-commit.