Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[red-knot]: Add a VendoredFileSystem implementation #11863

Merged
merged 43 commits into from
Jun 18, 2024
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
41a1aff
Refactor `file` and `vendored` methods and make them standalone funct…
MichaReiser Jun 10, 2024
d3c8e83
Make `file_system_path_to_file` return an Option
MichaReiser Jun 10, 2024
1a69745
Improve documentation, rewrite inline comment, rename `file_system_pa…
MichaReiser Jun 13, 2024
43fa7bf
Update crates/ruff_db/src/vfs.rs
MichaReiser Jun 13, 2024
b054315
Don't use non-platform agnostic `is_absolute` method
MichaReiser Jun 13, 2024
383691e
Add `ModuleName` struct with tests
MichaReiser Jun 10, 2024
5a1da62
Module resolver
MichaReiser Jun 10, 2024
01041cd
Port module resolver to salsa
MichaReiser Jun 11, 2024
f16c045
Add `remove_file` and `remove_directory` tests
MichaReiser Jun 11, 2024
56f2caf
Suppress warning about unused `Os` file system that are only used on …
MichaReiser Jun 11, 2024
bcaea19
Change `ModuleName::new` to return an `Option`
MichaReiser Jun 13, 2024
4df3bea
Use `touch` over `set_permission` and `set_revision` in Tests
MichaReiser Jun 13, 2024
4441c9e
[red-knot]: Add a `VendoredFileSystem` implementation
AlexWaygood Jun 13, 2024
ae49c71
Fix many easily addressed review comments
AlexWaygood Jun 14, 2024
bd94c60
More relatively easily addressed comments
AlexWaygood Jun 14, 2024
43e3626
Just use the file hash for `FileRevision`
AlexWaygood Jun 14, 2024
13ee2d2
simplify
AlexWaygood Jun 14, 2024
05846ae
Refactor `file` and `vendored` methods and make them standalone funct…
MichaReiser Jun 10, 2024
88a6aa4
Make `file_system_path_to_file` return an Option
MichaReiser Jun 10, 2024
64b67ef
Improve documentation, rewrite inline comment, rename `file_system_pa…
MichaReiser Jun 13, 2024
cbb8992
Update crates/ruff_db/src/vfs.rs
MichaReiser Jun 13, 2024
00477c7
Add `ModuleName` struct with tests
MichaReiser Jun 10, 2024
de1ec29
Module resolver
MichaReiser Jun 10, 2024
92b3465
Port module resolver to salsa
MichaReiser Jun 11, 2024
2ee6973
Add `remove_file` and `remove_directory` tests
MichaReiser Jun 11, 2024
96d75a7
Suppress warning about unused `Os` file system that are only used on …
MichaReiser Jun 11, 2024
e5e8da6
Change `ModuleName::new` to return an `Option`
MichaReiser Jun 13, 2024
2348fe6
Use `touch` over `set_permission` and `set_revision` in Tests
MichaReiser Jun 13, 2024
eea6023
Merge branch 'salsa-memory-resolver' into vendored-filesystem
AlexWaygood Jun 14, 2024
07e5a80
Reduce clones and make them more explicit
AlexWaygood Jun 15, 2024
514e9b6
Use interior mutability to avoid cloning when unnecessary
AlexWaygood Jun 15, 2024
1f68832
Do not implement `FileSystem`
AlexWaygood Jun 16, 2024
035ea53
Simplify normalization logic
AlexWaygood Jun 17, 2024
5c33610
Update crates/ruff_db/src/vendored.rs
AlexWaygood Jun 17, 2024
ecf97a9
Get rid of unneeded methods
AlexWaygood Jun 17, 2024
be8f952
Return `Result` from `read()`
AlexWaygood Jun 17, 2024
f4f97d4
Replace a wrapper type with a type alias
AlexWaygood Jun 17, 2024
df248ce
fix tests
AlexWaygood Jun 17, 2024
f05bcb2
Bring back some notion of vendored-stub-file "metadata"
AlexWaygood Jun 17, 2024
89ebc91
Merge remote-tracking branch 'upstream/main' into vendored-filesystem
AlexWaygood Jun 18, 2024
13c2802
add some newlines
AlexWaygood Jun 18, 2024
5d511bb
More docs etc.
AlexWaygood Jun 18, 2024
3127ff6
See if bumping `zstd-sys` changes anything
AlexWaygood Jun 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions crates/ruff_db/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,11 @@ camino = { workspace = true }
countme = { workspace = true }
dashmap = { workspace = true }
filetime = { workspace = true }
itertools = { workspace = true }
salsa = { workspace = true }
tracing = { workspace = true }
rustc-hash = { workspace = true }
zip = { workspace = true }

[dev-dependencies]
once_cell = { workspace = true }
72 changes: 1 addition & 71 deletions crates/ruff_db/src/file_system.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ use std::ops::Deref;
use std::path::{Path, StripPrefixError};

use camino::{Utf8Path, Utf8PathBuf};
use filetime::FileTime;

use crate::metadata::FileRevision;
pub use memory::MemoryFileSystem;
pub use os::OsFileSystem;

Expand Down Expand Up @@ -514,55 +514,6 @@ impl Metadata {
}
}

/// A number representing the revision of a file.
///
/// Two revisions that don't compare equal signify that the file has been modified.
/// Revisions aren't guaranteed to be monotonically increasing or in any specific order.
///
/// Possible revisions are:
/// * The last modification time of the file.
/// * The hash of the file's content.
/// * The revision as it comes from an external system, for example the LSP.
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
pub struct FileRevision(u128);

impl FileRevision {
pub fn new(value: u128) -> Self {
Self(value)
}

pub const fn zero() -> Self {
Self(0)
}

#[must_use]
pub fn as_u128(self) -> u128 {
self.0
}
}

impl From<u128> for FileRevision {
fn from(value: u128) -> Self {
FileRevision(value)
}
}

impl From<u64> for FileRevision {
fn from(value: u64) -> Self {
FileRevision(u128::from(value))
}
}

impl From<FileTime> for FileRevision {
fn from(value: FileTime) -> Self {
let seconds = value.seconds() as u128;
let seconds = seconds << 64;
let nanos = u128::from(value.nanoseconds());

FileRevision(seconds | nanos)
}
}

#[derive(Copy, Clone, Eq, PartialEq, Debug, Hash)]
pub enum FileType {
File,
Expand All @@ -583,24 +534,3 @@ impl FileType {
matches!(self, FileType::Symlink)
}
}

#[cfg(test)]
mod tests {
use filetime::FileTime;

use crate::file_system::FileRevision;

#[test]
fn revision_from_file_time() {
let file_time = FileTime::now();
let revision = FileRevision::from(file_time);

let revision = revision.as_u128();

let nano = revision & 0xFFFF_FFFF_FFFF_FFFF;
let seconds = revision >> 64;

assert_eq!(file_time.nanoseconds(), nano as u32);
assert_eq!(file_time.seconds(), seconds as i64);
}
}
2 changes: 2 additions & 0 deletions crates/ruff_db/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,10 @@ use crate::source::{line_index, source_text};
use crate::vfs::{Vfs, VfsFile};

pub mod file_system;
mod metadata;
pub mod parsed;
pub mod source;
pub mod vendored;
pub mod vfs;

pub(crate) type FxDashMap<K, V> = dashmap::DashMap<K, V, BuildHasherDefault<FxHasher>>;
Expand Down
69 changes: 69 additions & 0 deletions crates/ruff_db/src/metadata.rs
AlexWaygood marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
/// A number representing the revision of a file.
///
/// Two revisions that don't compare equal signify that the file has been modified.
/// Revisions aren't guaranteed to be monotonically increasing or in any specific order.
///
/// Possible revisions are:
/// * The last modification time of the file.
/// * The hash of the file's content.
/// * The revision as it comes from an external system, for example the LSP.
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
pub struct FileRevision(u128);

impl FileRevision {
pub fn new(value: u128) -> Self {
Self(value)
}

pub const fn zero() -> Self {
Self(0)
}

#[must_use]
pub fn as_u128(self) -> u128 {
self.0
}
}

impl From<u128> for FileRevision {
fn from(value: u128) -> Self {
FileRevision(value)
}
}

impl From<u64> for FileRevision {
fn from(value: u64) -> Self {
FileRevision(u128::from(value))
}
}

impl From<filetime::FileTime> for FileRevision {
fn from(value: filetime::FileTime) -> Self {
let seconds = value.seconds() as u128;
let seconds = seconds << 64;
let nanos = u128::from(value.nanoseconds());

FileRevision(seconds | nanos)
}
}

#[cfg(test)]
mod tests {
use filetime::FileTime;

use super::*;

#[test]
fn revision_from_file_time() {
let file_time = FileTime::now();
let revision = FileRevision::from(file_time);

let revision = revision.as_u128();

let nano = revision & 0xFFFF_FFFF_FFFF_FFFF;
let seconds = revision >> 64;

assert_eq!(file_time.nanoseconds(), nano as u32);
assert_eq!(file_time.seconds(), seconds as i64);
}
}
2 changes: 1 addition & 1 deletion crates/ruff_db/src/parsed.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ mod tests {
use crate::file_system::FileSystemPath;
use crate::parsed::parsed_module;
use crate::tests::TestDb;
use crate::vfs::VendoredPath;
use crate::vendored::VendoredPath;
use crate::vfs::{system_path_to_file, vendored_path_to_file};

#[test]
Expand Down
Loading
Loading