Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MRepo refactor #3118

Merged
merged 48 commits into from
Jan 17, 2024
Merged
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
0ba552d
Rename Enum members in MRepo
AntoinePrv Jan 9, 2024
cdeffca
use solv::Pool in MRepo
AntoinePrv Jan 9, 2024
ffba7e3
Free MRepo add_pip_as_python_dependency
AntoinePrv Jan 9, 2024
9b3358a
Free MRepo repodata reading
AntoinePrv Jan 9, 2024
5d20c09
Free MRepo read_solv
AntoinePrv Jan 9, 2024
186b8ef
Free MRepo write_solv
AntoinePrv Jan 9, 2024
cbc92c0
Remove MRepo constructor
AntoinePrv Jan 9, 2024
7cf1fcc
Remove MPool capture from MRepo
AntoinePrv Jan 9, 2024
b5f0ecd
Add missing pip dependency
AntoinePrv Jan 9, 2024
771ff90
Refactor solv file serialization
AntoinePrv Jan 9, 2024
85f0cc4
No lock on non-existant files
AntoinePrv Jan 10, 2024
5083c3b
Cleanup dead code
AntoinePrv Jan 10, 2024
490c6b2
write_solve create-directories
AntoinePrv Jan 10, 2024
2b102c7
Clean MRepo member functions
AntoinePrv Jan 10, 2024
5aeaad0
Decouple MRepo and Pool
AntoinePrv Jan 10, 2024
dbeb782
Finetune Clang-format
AntoinePrv Jan 10, 2024
b0724c9
Add filename location to error
AntoinePrv Jan 12, 2024
90ba415
Do not fail on solv file write
AntoinePrv Jan 12, 2024
a303fc2
Load new pool loader for repodata and .solv files
AntoinePrv Jan 10, 2024
8a06d74
Remove repo name in favor of URL
AntoinePrv Jan 11, 2024
1cb44b2
Refactor MRepo index loading
AntoinePrv Jan 12, 2024
7a31b50
Fix SubDirData.writable_solv_cache
AntoinePrv Jan 15, 2024
994a4d2
Fix solv file loading
AntoinePrv Jan 15, 2024
fa8a8ac
Lock repodata.json while reading metadata
AntoinePrv Jan 15, 2024
7c5491a
Refactor expected in load_subdir_in_pool
AntoinePrv Jan 15, 2024
9634201
use expected in solver helpers
AntoinePrv Jan 15, 2024
1f9df16
Refactor helpers functions and error messages in Repo
AntoinePrv Jan 15, 2024
454ec89
Add MPool::create_repo_from_packages
AntoinePrv Jan 15, 2024
f443a6d
Use load_installed_packages_in_pool
AntoinePrv Jan 15, 2024
033694e
Remove MRepo ctor
AntoinePrv Jan 16, 2024
d1bdd43
Fix rebase
AntoinePrv Jan 16, 2024
563e098
Refactor MRepo::priority
AntoinePrv Jan 16, 2024
6701292
Update Taskfile realpath to work on MacOS
AntoinePrv Jan 16, 2024
e1048a1
Fix virtual packages being removed
AntoinePrv Jan 16, 2024
010d98e
Move MRepo::priority to MPool
AntoinePrv Jan 16, 2024
3b39bff
Rename MRepo > solver::libsolv::RepoInfo
AntoinePrv Jan 17, 2024
362fd20
Move RepoInfo parameters for its own library
AntoinePrv Jan 17, 2024
30803fd
Move make_package_info to helpers
AntoinePrv Jan 17, 2024
c001153
Add libmambapy.solver.libsolv
AntoinePrv Jan 17, 2024
a862362
Add solver::libsolv::Priorities op
AntoinePrv Jan 17, 2024
106a0ac
Bind libsolv parameters
AntoinePrv Jan 17, 2024
777ab9e
Bind RepodataOrigin
AntoinePrv Jan 17, 2024
f66ce40
Merge libsolv/serialization and libsolv/parameters
AntoinePrv Jan 17, 2024
97b633f
Bind new MPool methods
AntoinePrv Jan 17, 2024
30c216e
Add RepoInfo op
AntoinePrv Jan 17, 2024
49127cc
Add RepoInfo changes to log
AntoinePrv Jan 17, 2024
61ebb7d
Rename attribute
AntoinePrv Jan 17, 2024
2fcfbc7
Remove RepoInfo::repo
AntoinePrv Jan 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .clang-format
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ AllowShortLoopsOnASingleLine: 'false'
AlwaysBreakTemplateDeclarations: 'true'
BinPackArguments: false
BinPackParameters: false
BreakAfterAttributes: Leave
BreakBeforeBinaryOperators: All
BreakBeforeBraces: Allman
BreakBeforeTernaryOperators: 'true'
Expand Down
6 changes: 2 additions & 4 deletions Taskfile.dist.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,9 @@ vars:
CMAKE_PRESET: 'mamba-unix-shared-debug-dev'
CACHE_DIR: '{{.BUILD_DIR}}/pkgs'
DOCS_DIR: '{{.BUILD_DIR}}/docs'
DOCS_DOXYGEN_XML_DIR:
sh: realpath -m '{{.DOCS_DIR}}/doxygen-xml'
DOCS_DOXYGEN_XML_DIR: '{{.PWD}}/{{.DOCS_DIR}}/doxygen-xml'
MAMBA_NAME: 'mamba' # Depend on preset...
TEST_MAMBA_EXE:
sh: 'realpath -m {{.CMAKE_BUILD_DIR}}/micromamba/{{.MAMBA_NAME}}'
TEST_MAMBA_EXE: '{{.PWD}}/{{.CMAKE_BUILD_DIR}}/micromamba/{{.MAMBA_NAME}}'
CPU_PERCENTAGE: 75
CPU_TOTAL:
sh: >-
Expand Down
25 changes: 16 additions & 9 deletions libmamba/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,7 @@ set(
# Filesystem library
${LIBMAMBA_SOURCE_DIR}/fs/filesystem.cpp
# C++ utility library
${LIBMAMBA_SOURCE_DIR}/util/cfile.cpp
${LIBMAMBA_SOURCE_DIR}/util/cryptography.cpp
${LIBMAMBA_SOURCE_DIR}/util/encoding.cpp
${LIBMAMBA_SOURCE_DIR}/util/environment.cpp
Expand All @@ -161,13 +162,6 @@ set(
${LIBMAMBA_SOURCE_DIR}/util/string.cpp
${LIBMAMBA_SOURCE_DIR}/util/url.cpp
${LIBMAMBA_SOURCE_DIR}/util/url_manip.cpp
# C++ wrapping of libsolv
${LIBMAMBA_SOURCE_DIR}/solv-cpp/pool.cpp
${LIBMAMBA_SOURCE_DIR}/solv-cpp/queue.cpp
${LIBMAMBA_SOURCE_DIR}/solv-cpp/repo.cpp
${LIBMAMBA_SOURCE_DIR}/solv-cpp/solvable.cpp
${LIBMAMBA_SOURCE_DIR}/solv-cpp/solver.cpp
${LIBMAMBA_SOURCE_DIR}/solv-cpp/transaction.cpp
# Implementation of version and matching specs
${LIBMAMBA_SOURCE_DIR}/specs/archive.cpp
${LIBMAMBA_SOURCE_DIR}/specs/authentication_info.cpp
Expand All @@ -182,6 +176,17 @@ set(
${LIBMAMBA_SOURCE_DIR}/specs/unresolved_channel.cpp
${LIBMAMBA_SOURCE_DIR}/specs/version.cpp
${LIBMAMBA_SOURCE_DIR}/specs/version_spec.cpp
# C++ wrapping of libsolv
${LIBMAMBA_SOURCE_DIR}/solv-cpp/pool.cpp
${LIBMAMBA_SOURCE_DIR}/solv-cpp/queue.cpp
${LIBMAMBA_SOURCE_DIR}/solv-cpp/repo.cpp
${LIBMAMBA_SOURCE_DIR}/solv-cpp/solvable.cpp
${LIBMAMBA_SOURCE_DIR}/solv-cpp/solver.cpp
${LIBMAMBA_SOURCE_DIR}/solv-cpp/transaction.cpp
# Solver interface
${LIBMAMBA_SOURCE_DIR}/solver/libsolv/helpers.cpp
${LIBMAMBA_SOURCE_DIR}/solver/libsolv/parameters.cpp
${LIBMAMBA_SOURCE_DIR}/solver/libsolv/repo_info.cpp
# Artifacts validation
${LIBMAMBA_SOURCE_DIR}/validation/tools.cpp
${LIBMAMBA_SOURCE_DIR}/validation/errors.cpp
Expand Down Expand Up @@ -220,7 +225,6 @@ set(
${LIBMAMBA_SOURCE_DIR}/core/package_fetcher.cpp
${LIBMAMBA_SOURCE_DIR}/core/package_paths.cpp
${LIBMAMBA_SOURCE_DIR}/core/query.cpp
${LIBMAMBA_SOURCE_DIR}/core/repo.cpp
${LIBMAMBA_SOURCE_DIR}/core/run.cpp
${LIBMAMBA_SOURCE_DIR}/core/shell_init.cpp
${LIBMAMBA_SOURCE_DIR}/core/solver.cpp
Expand Down Expand Up @@ -262,6 +266,7 @@ set(
# Utility library
${LIBMAMBA_INCLUDE_DIR}/mamba/util/build.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/util/cast.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/util/cfile.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/util/compare.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/util/conditional.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/util/cryptography.hpp
Expand Down Expand Up @@ -300,6 +305,9 @@ set(
${LIBMAMBA_INCLUDE_DIR}/mamba/specs/unresolved_channel.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/specs/version.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/specs/version_spec.hpp
# Solver Interface
${LIBMAMBA_INCLUDE_DIR}/mamba/solver/libsolv/parameters.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/solver/libsolv/repo_info.hpp
# Artifacts validation
${LIBMAMBA_INCLUDE_DIR}/mamba/validation/tools.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/validation/errors.hpp
Expand Down Expand Up @@ -332,7 +340,6 @@ set(
${LIBMAMBA_INCLUDE_DIR}/mamba/core/progress_bar.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/core/pinning.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/core/query.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/core/repo.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/core/run.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/core/shell_init.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/core/solution.hpp
Expand Down
2 changes: 1 addition & 1 deletion libmamba/include/mamba/api/install.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@

#include "mamba/core/package_cache.hpp"
#include "mamba/core/pool.hpp"
#include "mamba/core/repo.hpp"
#include "mamba/fs/filesystem.hpp"
#include "mamba/solver/libsolv/repo_info.hpp"
#include "mamba/specs/match_spec.hpp"
#include "mamba/specs/package_info.hpp"

Expand Down
99 changes: 98 additions & 1 deletion libmamba/include/mamba/core/pool.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,22 @@

#include <solv/pooltypes.h>

#include "mamba/core/error_handling.hpp"
#include "mamba/solver/libsolv/parameters.hpp"
#include "mamba/solver/libsolv/repo_info.hpp"
#include "mamba/specs/package_info.hpp"

namespace mamba
{
class ChannelContext;
class Context;
class PrefixData;
class MSubdirData;

namespace fs
{
class u8path;
}

namespace solv
{
Expand Down Expand Up @@ -61,6 +71,45 @@ namespace mamba
solv::ObjPool& pool();
const solv::ObjPool& pool() const;

auto add_repo_from_repodata_json(
const fs::u8path& path,
std::string_view url,
solver::libsolv::PipAsPythonDependency add = solver::libsolv::PipAsPythonDependency::No,
solver::libsolv::RepodataParser parser = solver::libsolv::RepodataParser::Mamba
) -> expected_t<solver::libsolv::RepoInfo>;

auto add_repo_from_native_serialization(
const fs::u8path& path,
const solver::libsolv::RepodataOrigin& expected,
solver::libsolv::PipAsPythonDependency add = solver::libsolv::PipAsPythonDependency::No
) -> expected_t<solver::libsolv::RepoInfo>;

template <typename Iter>
auto add_repo_from_packages(
Iter first_package,
Iter last_package,
std::string_view name = "",
solver::libsolv::PipAsPythonDependency add = solver::libsolv::PipAsPythonDependency::No
) -> solver::libsolv::RepoInfo;

template <typename Range>
auto add_repo_from_packages(
const Range& packages,
std::string_view name = "",
solver::libsolv::PipAsPythonDependency add = solver::libsolv::PipAsPythonDependency::No
) -> solver::libsolv::RepoInfo;

auto native_serialize_repo(
const solver::libsolv::RepoInfo& repo,
const fs::u8path& path,
const solver::libsolv::RepodataOrigin& metadata
) -> expected_t<solver::libsolv::RepoInfo>;

void set_installed_repo(const solver::libsolv::RepoInfo& repo);

void
set_repo_priority(const solver::libsolv::RepoInfo& repo, solver::libsolv::Priorities priorities);

void remove_repo(::Id repo_id, bool reuse_ids);
Comment on lines +74 to 113
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jaimergp What do you think of the new API here?


ChannelContext& channel_context() const;
Expand All @@ -84,7 +133,55 @@ namespace mamba
* - Facilitate (potential) future investigation of parallel solves.
*/
std::shared_ptr<MPoolData> m_data;

auto add_repo_from_packages_impl_pre(std::string_view name) -> solver::libsolv::RepoInfo;
void add_repo_from_packages_impl_loop(
const solver::libsolv::RepoInfo& repo,
const specs::PackageInfo& pkg
);
void add_repo_from_packages_impl_post(
const solver::libsolv::RepoInfo& repo,
solver::libsolv::PipAsPythonDependency add
);
};
} // namespace mamba

// TODO machinery functions in separate files
auto load_subdir_in_pool(const Context& ctx, MPool& pool, const MSubdirData& subdir)
-> expected_t<solver::libsolv::RepoInfo>;

auto load_installed_packages_in_pool(const Context& ctx, MPool& pool, const PrefixData& prefix)
-> solver::libsolv::RepoInfo;

/********************
* Implementation *
********************/

template <typename Iter>
auto MPool::add_repo_from_packages(
Iter first_package,
Iter last_package,
std::string_view name,
solver::libsolv::PipAsPythonDependency add
) -> solver::libsolv::RepoInfo
{
auto repo = add_repo_from_packages_impl_pre(name);
for (; first_package != last_package; ++first_package)
{
add_repo_from_packages_impl_loop(repo, *first_package);
}
add_repo_from_packages_impl_post(repo, add);
return repo;
}

template <typename Range>
auto MPool::add_repo_from_packages(
const Range& packages,
std::string_view name,
solver::libsolv::PipAsPythonDependency add
) -> solver::libsolv::RepoInfo
{
return add_repo_from_packages(packages.begin(), packages.end(), name, add);
}
}

#endif // MAMBA_POOL_HPP
138 changes: 0 additions & 138 deletions libmamba/include/mamba/core/repo.hpp

This file was deleted.

Loading
Loading