Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch the "hooks" branch up with "develop" with a big ol' merge #4927

Merged
merged 527 commits into from
Jan 6, 2025

Conversation

ximinez
Copy link
Collaborator

@ximinez ximinez commented Feb 23, 2024

High Level Overview of Change

Note that this is not fully functional code! It is being merged into the hooks branch, which is still "in-development".

This branch merges develop into hooks, and makes some fixes / updates to get it partially working.

Results by OS:

  1. It builds on Linux, but a unit test run doesn't finish. Some relatively simple unit tests fail. e.g. AccountSet.
  2. It doesn't build on Windows / MSVC, but it gets a lot further than the original PR did. Notably, it is able to download and set up the wasmedge dependency using Conan. All of the errors appear to be limited to applyHook.cpp, and all appear to be due to parsing / preprocessing errors with the massively complex macros in macros.h
  3. It builds with MacOS clang, but fails to start: dyld[6586]: Library not loaded: /opt/homebrew/opt/llvm/lib/libLLVM.dylib Referenced from: <68C35517-5F51-3521-A8EE-2CE9D9882E28> /Users/ed/.conan/data/wasmedge/0.9.0/_/_/package/72521513b9fee5d76bcfd97cf0256cfa56cb3897/lib/libwasmedge_c.dylib

I have made no significant effort to fix any remaining issues, and I don't intend to, though I will be happy to accept contributions. This is not intended to be working code, just to get the branch caught up so that someone else can get it working.

Context of Change

This is a follow up to #4225, which also only affected that branch.

Type of Change

  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Tests (you added tests for code that already exists, or your new feature included in this PR)

Future Tasks

Significant work and debugging needs to be done to get this "ready for prime time".

Note that this is not fully functional code! It is being merged into the hooks branch, which is still "in-development".

intelliot and others added 30 commits October 16, 2023 17:59
It might be possible for the server code to indirect through certain
`end()` iterators. While a debug build would catch this problem with
`assert()`s, a release build would crash. If there are problems in this
area in the future, it is best to get a definitive indication of the
nature of the error regardless of whether it's a debug or release build.
To accomplish this, these `assert`s are converted into `LogicError`s
that will produce a reasonable error message when they fire.
Implement native support for W3C DIDs.

Add a new ledger object: `DID`.

Add two new transactions:
1. `DIDSet`: create or update the `DID` object.
2. `DIDDelete`: delete the `DID` object.

This meets the requirements specified in the DID v1.0 specification
currently recommended by the W3C Credentials Community Group.

The DID format for the XRP Ledger conforms to W3C DID standards.
The objects can be created and owned by any XRPL account holder.
The transactions can be integrated by any service, wallet, or application.
Add a new RPC / WS call for `server_definitions`, which returns an
SDK-compatible `definitions.json` (binary enum definitions) generated by
the server. This enables clients/libraries to dynamically work with new
fields and features, such as ones that may become available on side
chains. Clients query `server_definitions` on a node from the network
they want to work with, and immediately know how to speak that node's
binary "language", even if new features are added to it in the future
(as long as there are no new serialized types that the software doesn't
know how to serialize/deserialize).

Example:

```js
> {"command": "server_definitions"}
< {
    "result": {
        "FIELDS": [
            [
                "Generic",
                {
                    "isSerialized": false,
                    "isSigningField": false,
                    "isVLEncoded": false,
                    "nth": 0,
                    "type": "Unknown"
                }
            ],
            [
                "Invalid",
                {
                    "isSerialized": false,
                    "isSigningField": false,
                    "isVLEncoded": false,
                    "nth": -1,
                    "type": "Unknown"
                }
            ],
            [
                "ObjectEndMarker",
                {
                    "isSerialized": false,
                    "isSigningField": true,
                    "isVLEncoded": false,
                    "nth": 1,
                    "type": "STObject"
                }
            ],
        ...
```

Close XRPLF#3657

---------

Co-authored-by: Richard Holland <richard.holland@starstone.co.nz>
The Network ID logic should not be applied to pseudo-transactions.

This allows amendments to enable on a network with an ID > 1024.

Context:
- NetworkID: XRPLF#4370
- Pseudo-transactions: https://xrpl.org/pseudo-transaction-types.html

Fix XRPLF#4736

---------

Co-authored-by: RichardAH <richard.holland@starstone.co.nz>
The unity build speeds up compilation by bundling multiple source files
into one larger file. This reduces Windows CI build time by up to 50%.

As described in XRPLF#4596, the automatic Windows builds take a very long
time. Unity builds are significantly faster - currently about 45 min,
much closer to the typical MacOS (35-40 minutes) and nix (~30 minutes)
run times.

This is intended as a stopgap solution until a more resourced and
reliable runner is available.

No C++ code was changed. This only affects CI.
…F#4733)

Using the "Amount" field in Payment transactions can cause incorrect
interpretation. There continue to be problems from the use of this
field. "Amount" is rarely the correct field to use; instead,
"delivered_amount" (or "DeliveredAmount") should be used.

Rename the "Amount" field to "DeliverMax", a less misleading name. With
api_version: 2, remove the "Amount" field from Payment transactions.

- Input: "DeliverMax" in `tx_json` is an alias for "Amount"
  - sign
  - submit (in sign-and-submit mode)
  - submit_multisigned
  - sign_for
- Output: Add "DeliverMax" where transactions are provided by the API
  - ledger
  - tx
  - tx_history
  - account_tx
  - transaction_entry
  - subscribe (transactions stream)
- Output: Remove "Amount" from API version 2

Fix XRPLF#3484

Fix XRPLF#3902
Update the documentation to describe network health with more nuance as
well as context about related factors.
Remove `tx_history` and `ledger_header` methods from API version 2.

Update `RPC::Handler` to allow for methods (or method implementations)
to be API version specific. This partially resolves XRPLF#4727. We can now
store multiple handlers with the same name, as long as they belong to
different (non-overlapping) API versions. This necessarily impacts the
handler lookup algorithm and its complexity; however, there is no
performance loss on x86_64 architecture, and only minimal performance
loss on arm64 (around 10ns). This design change gives us extra
flexibility evolving the API in the future, including other parts of
XRPLF#4727.

In API version 2, `tx_history` and `ledger_header` are no longer
recognised; if they are called, `rippled` will return error
`unknownCmd`

Resolve XRPLF#3638

Resolve XRPLF#3539
Remove dependency on `<ranges>` header, since it is not implemented by
all compilers which we want to support.

This code change only affects unit tests.

Resolve XRPLF#4787
Introduce the `fixFillOrKill` amendment.

Fix an edge case occurring when an offer with `tfFillOrKill` set (but
without `tfSell` set) fails to cross an offer with a better rate. If
`tfFillOrKill` is set, then the owner must receive the full TakerPays.
Without this amendment, an offer fails if the entire `TakerGets` is not
spent. With this amendment, when `tfSell` is not set, the entire
`TakerGets` does not have to be spent.

For details about OfferCreate, see: https://xrpl.org/offercreate.html

Fix XRPLF#4684

---------

Co-authored-by: Scott Schurr <scott@ripple.com>
With clang 15, an unused-but-set-variable warning was emitted:

PostgresDatabase.cpp:178:14: warning: variable 'expNumResults' set but
not used [-Wunused-but-set-variable]
    uint32_t expNumResults = 1;
The command line API still uses `apiMaximumSupportedVersion`.
The unit test RPCs use `apiMinimumSupportedVersion` if unspecified.

Context:
- XRPLF#4568
- XRPLF#4552
* Remove include <ranges>

* Formatting fix

* Output for subscriptions

* Output from sign, submit etc.

* Output from ledger

* Output from account_tx

* Output from transaction_entry

* Output from tx

* Store close_time_iso in API v2 output

* Add small APIv2 unit test for subscribe

* Add unit test for transaction_entry

* Add unit test for tx

* Remove inLedger from API version 2

* Set ledger_hash and ledger_index

* Move isValidated from RPCHelpers to LedgerMaster

* Store closeTime in LedgerFill

* Time formatting fix

* additional tests for Subscribe unit tests

* Improved comments

* Rename mInLedger to mLedgerIndex

* Minor fixes

* Set ledger_hash on closed ledger, even if not validated

* Update API-CHANGELOG.md

* Add ledger_hash, ledger_index to transaction_entry

* Fix validated and close_time_iso in account_tx

* Fix typos

* Improve getJson for Transaction and STTx

* Minor improvements

* Replace class enum JsonOptions with struct

We may consider turning this into a general-purpose template and using it elsewhere

* simplify the extraction of transactionID from Transaction object

* Remove obsolete comments

* Unconditionally set validated in account_tx output

* Minor improvements

* Minor fixes

---------

Co-authored-by: Chenna Keshava <ckeshavabs@gmail.com>
* Promote API version 2 to supported

* Switch command line to API version 1

* Fix LedgerRequestRPC test

* Remove obsolete tx_account method

This method is not implemented, the only parts which are removed are related to command-line parsing

* Fix RPCCall test

* Reduce diff size, small test improvements

* Minor fixes

* Support for the mold linker

* [fold] handle case where both mold and gold are installed

* [fold] Use first non-default linker

* Fix TransactionEntry_test

* Fix AccountTx_test

---------

Co-authored-by: seelabs <scott.determan@yahoo.com>
Co-authored-by: manoj <mdoshi@ripple.com>
* Fix binary output from tx method

* Formatting fix

* Minor test improvement

* Minor test improvements
…uitous proposal changes (XRPLF#4760)

* Optimize the calculation of close time to avoid
impasse and minimize gratuitous proposal changes.

* git apply clang-format.patch

* Review (Howard) fixes.

* Review fix for impasse discovered by John.

* Review fixes (comments) from John.

* Scott S review fixes. Also clang-format.
* Support for the mold linker (XRPLF#4807)

* Promote API version 2 to supported (XRPLF#4803)

* Promote API version 2 to be supported

* Switch the command line to API version 1

* Fix LedgerRequestRPC test

* Remove obsolete tx_account method

This method is not implemented, the only parts which are removed are related to command-line parsing

* Fix RPCCall test

* Reduce diff size, small test improvements

* Minor fixes

* Support for the mold linker

* Fix TransactionEntry_test

* Fix AccountTx_test

---------

Co-authored-by: seelabs <scott.determan@yahoo.com>

* Update Linux smoketest distros (XRPLF#4813)

* Fix 2.0 regression in tx method with binary output (XRPLF#4812)

* Fix binary output from tx method

* Formatting fix

* Minor test improvement

* Minor test improvements

* Optimize calculation of close time to avoid impasse and minimize gratuitous proposal changes (XRPLF#4760)

* Optimize the calculation of close time to avoid
impasse and minimize gratuitous proposal changes.

* git apply clang-format.patch

* Scott S review fixes. Also clang-format.

* Set version to 2.0.0-rc2

---------

Co-authored-by: manoj <mdoshi@ripple.com>
Co-authored-by: Scott Determan <scott.determan@yahoo.com>
Co-authored-by: Bronek Kozicki <brok@incorrekt.com>
Co-authored-by: Michael Legleux <legleux@users.noreply.github.com>
Co-authored-by: Mark Travis <mtrippled@users.noreply.github.com>
* Promote API version 2 to supported

* Switch command line to API version 1

* Fix LedgerRequestRPC test

* Remove obsolete tx_account method

This method is not implemented, the only parts which are removed are related to command-line parsing

* Fix RPCCall test

* Reduce diff size, small test improvements

* Minor fixes

* Support for the mold linker

* [fold] handle case where both mold and gold are installed

* [fold] Use first non-default linker

* Fix TransactionEntry_test

* Fix AccountTx_test

---------

Co-authored-by: seelabs <scott.determan@yahoo.com>
Co-authored-by: manoj <mdoshi@ripple.com>
Copy link
Collaborator

@vlntb vlntb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went through the last 20 synchronizations with develop to check that merges didn't produce any obvious errors. Looks good to me.

intelliot and others added 8 commits November 25, 2024 13:16
`STNumber` lets objects and transactions contain multiple fields for
quantities of XRP, IOU, or MPT without duplicating information about the
"issue" (represented by `STIssue`). It is a straightforward serialization of
the `Number` type that uniformly represents those quantities.

---------

Co-authored-by: John Freeman <jfreeman08@gmail.com>
Co-authored-by: Howard Hinnant <howard.hinnant@gmail.com>
Move tfClawTwoAssets check to preflight and return
error temINVALID_FLAG

---------

Co-authored-by: yinyiqian1 <yqian@ripple.com>
* Copy Antithesis SDK version 0.4.0 to directory external/
* Add build option `voidstar` to enable instrumentation with Antithesis SDK
* Define instrumentation macros ASSERT and UNREACHABLE in terms of regular C assert
* Replace asserts with named ASSERT or UNREACHABLE
* Add UNREACHABLE to LogicError
* Document instrumentation macros in CONTRIBUTING.md
---------
Co-authored-by: John Freeman <jfreeman08@gmail.com>
Refactors LedgerEntry to make it easier to read and understand.
Adds two CMake functions:

* add_module(library subdirectory): Declares an OBJECT "library" (a CMake abstraction for a collection of object files) with sources from the given subdirectory of the given library, representing a module. Isolates the module's headers by creating a subdirectory in the build directory, e.g. .build/tmp123, that contains just a symlink, e.g. .build/tmp123/basics, to the module's header directory, e.g. include/xrpl/basics, in the source directory, and putting .build/tmp123 (but not include/xrpl) on the include path of the module sources. This prevents the module sources from including headers not explicitly linked to the module in CMake with target_link_libraries.
* target_link_modules(library scope modules...): Links the library target to each of the module targets, and removes their sources from its source list (so they are not compiled and linked twice).

Uses these functions to separate and explicitly link modules in libxrpl:

    Level 01: beast
    Level 02: basics
    Level 03: json, crypto
    Level 04: protocol
    Level 05: resource, server
Bronek and others added 5 commits December 16, 2024 17:48
* Rename ASSERT to XRPL_ASSERT
* Upgrade to Anthithesis SDK 0.4.4, and use new 0.4.4 features
  * automatic cast to bool, like assert
* Add instrumentation workflow to verify build with instrumentation enabled
Replace Issue in STIssue with Asset. STIssue with MPTIssue is only used in MPT tests.
Will be used in Vault and in transactions with STIssue fields once MPT is integrated into DEX.
For example, Expected<std::uint32_t, Json::Value>, will now build even though there is animplicit conversion from unsigned int to Json::Value.
Fix `error C2039: 'set_difference': is not a member of 'std'`
* upstream/develop: (518 commits)
  Set version to 2.4.0-b1
  fix: Add header for set_difference (5197)
  fix: allow overlapping types in `Expected` (5218)
  Add MPTIssue to STIssue (5200)
  Antithesis instrumentation improvements (5213)
  Enforce levelization in libxrpl with CMake (5111)
  refactor: clean up `LedgerEntry.cpp` (5199)
  test: Add more test cases for Base58 parser (5174)
  test: Check for some unlikely null dereferences in tests (5004)
  Add Antithesis intrumentation (5042)
  Set version to 2.3.0
  refactor(AMMClawback): move tfClawTwoAssets check (5201)
  Add a new serialized type: STNumber (5121)
  fix: check for valid ammID field in amm_info RPC (5188)
  Set version to 2.3.0-rc2
  fix: include `index` in `server_definitions` RPC (5190)
  Fix ledger_entry crash on invalid credentials request (5189)
  Set version to 2.3.0-rc1
  Replace Uint192 with Hash192 in server_definitions response (5177)
  Fix potential deadlock (5124)
  ...
@ximinez ximinez force-pushed the hooks-merge branch 2 times, most recently from e9dc32c to 3aa4bfe Compare December 20, 2024 18:55
@ximinez
Copy link
Collaborator Author

ximinez commented Dec 21, 2024

If there are no objections, I'm going to merge this PR as-is sometime after I get back from PTO on 1/6/2025.

As a reminder, the target of this merge is the "in-development" hooks branch, so the fact that it's currently mostly broken isn't really an issue.

@ximinez ximinez force-pushed the hooks-merge branch 2 times, most recently from 8e27671 to 5883d75 Compare January 6, 2025 23:06
@ximinez ximinez merged commit 5883d75 into XRPLF:hooks Jan 6, 2025
7 of 20 checks passed
@ximinez ximinez deleted the hooks-merge branch January 6, 2025 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.