Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory optimizations for large address queries #392

Merged
merged 20 commits into from
Jan 27, 2016

Commits on Jan 11, 2016

  1. Address Service: Start to use streams for memory optimization with la…

    …rge queries
    Braydon Fuller committed Jan 11, 2016
    Configuration menu
    Copy the full SHA
    cab25cf View commit details
    Browse the repository at this point in the history
  2. Address Service: Start to cache getAddressSummary based on range of…

    … block heights
    Braydon Fuller committed Jan 11, 2016
    Configuration menu
    Copy the full SHA
    40eb4f5 View commit details
    Browse the repository at this point in the history
  3. Address Service: Limit the length of outputs that can be queried at a…

    … time
    Braydon Fuller committed Jan 11, 2016
    Configuration menu
    Copy the full SHA
    cef2f76 View commit details
    Browse the repository at this point in the history
  4. Address Service: Use streams to combine inputs and outputs

    Braydon Fuller committed Jan 11, 2016
    Configuration menu
    Copy the full SHA
    8298e38 View commit details
    Browse the repository at this point in the history
  5. Address Service: Use address summary cache for pagination

    Braydon Fuller committed Jan 11, 2016
    Configuration menu
    Copy the full SHA
    5c4f3c4 View commit details
    Browse the repository at this point in the history
  6. Address Service: Restored multi-address history queries

    - Restored functionality to be able to query the history of multiple addresses in one query
    - Sorted mempool transactions by timestamp in txid lists
    Braydon Fuller committed Jan 11, 2016
    Configuration menu
    Copy the full SHA
    8d2f69c View commit details
    Browse the repository at this point in the history
  7. Address Service: Fixed HASH_TYPES_MAP naming issue

    Braydon Fuller committed Jan 11, 2016
    Configuration menu
    Copy the full SHA
    188ff28 View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2016

  1. Address Service: Fixed many bugs from tests

    - Refactored getAddressSummary and added several tests
    - Fixed bugs revealed from the integration regtests
    - Updated many unit tests
    Braydon Fuller committed Jan 13, 2016
    Configuration menu
    Copy the full SHA
    4fcec87 View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2016

  1. Address Service: Removed caching and added max query limits

    Querying addresses that have millions of transactions is supported however
    takes hundreds of seconds to fully calculate the balance. Creating a cache of
    previous results wasn't currently working because the `isSpent` query is always
    based on the current bitcoind tip. Thus the balance of the outputs would be included
    however wouldn't be removed when spent as the output wouldn't be checked again
    when querying for blocks past the last checkpoint. Including the satoshis in the
    inputs address index would make it possible to subtract the spent amount,
    however this degrades optimizations elsewhere. The syncing times or querying
    for addresses with 10,000 transactions per address.
    
    It may preferrable to have an additional address service that handles high-volume
    addresses be on an opt-in basis so that a custom running client could select
    high volume addresses to create optimizations for querying balances and history.
    The strategies for creating indexes differs on these use cases.
    Braydon Fuller committed Jan 14, 2016
    Configuration menu
    Copy the full SHA
    ead6c2f View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2016

  1. Address Service: Updated tests and fixed various bugs

    Braydon Fuller committed Jan 15, 2016
    Configuration menu
    Copy the full SHA
    e79c00d View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2016

  1. Address Service: More tests for history

    Braydon Fuller committed Jan 18, 2016
    Configuration menu
    Copy the full SHA
    3d9b6d5 View commit details
    Browse the repository at this point in the history
  2. Address Service: Added test for history `getAddressDetailsForTransact…

    …ion`
    Braydon Fuller committed Jan 18, 2016
    Configuration menu
    Copy the full SHA
    687400e View commit details
    Browse the repository at this point in the history
  3. Address Service: Removed event listeners prior to stopping

    Braydon Fuller committed Jan 18, 2016
    Configuration menu
    Copy the full SHA
    62934b4 View commit details
    Browse the repository at this point in the history
  4. Address Service: Removed nolonger used constant for cache

    Braydon Fuller committed Jan 18, 2016
    Configuration menu
    Copy the full SHA
    a166b6a View commit details
    Browse the repository at this point in the history
  5. Address Service: Sort mempool txids

    Braydon Fuller committed Jan 18, 2016
    Configuration menu
    Copy the full SHA
    d4f2df5 View commit details
    Browse the repository at this point in the history
  6. Address Service: Include default callback earlier

    Braydon Fuller committed Jan 18, 2016
    Configuration menu
    Copy the full SHA
    e498e0f View commit details
    Browse the repository at this point in the history
  7. Address Service: Sort after unconfirmed and confirmed

    Braydon Fuller committed Jan 18, 2016
    Configuration menu
    Copy the full SHA
    4502903 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2016

  1. Address Service: Bump maximum number of addresses default

    Braydon Fuller committed Jan 19, 2016
    Configuration menu
    Copy the full SHA
    39f8355 View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2016

  1. Address Service: Fixed test for max address limit

    Braydon Fuller committed Jan 26, 2016
    Configuration menu
    Copy the full SHA
    a2acc0c View commit details
    Browse the repository at this point in the history
  2. Address Service: End stream without pausing first

    There was an issue where streams would still be held open if "pause" was
    called before "end", this would lead to http requests from the insight-api
    not being returned with an error status as soon as possible but would
    instead stay open.
    Braydon Fuller committed Jan 26, 2016
    Configuration menu
    Copy the full SHA
    3d7fb6f View commit details
    Browse the repository at this point in the history