Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data Reader #2386

Closed
wants to merge 5 commits into from
Closed

Data Reader #2386

wants to merge 5 commits into from

Commits on May 19, 2015

  1. Added BlockingQueue for inter-thread communication.

    @cypof
    cypof committed May 19, 2015
    Configuration menu
    Copy the full SHA
    6108b25 View commit details
    Browse the repository at this point in the history

  2. Thread-local Caffe

    @cypof
    cypof committed May 19, 2015
    Configuration menu
    Copy the full SHA
    b8b5a52 View commit details
    Browse the repository at this point in the history

  3. Changed the way threads are started and stopped 

    - Interrupt the thread before waiting on join
- Provide a method for looping threads to exit on demand
- CHECK if start and stop succeed instead of returning an error
    @cypof
    cypof committed May 19, 2015
    Configuration menu
    Copy the full SHA
    59bb3ad View commit details
    Browse the repository at this point in the history

  4. Persistent prefetch thread

    @cypof
    cypof committed May 19, 2015
    Configuration menu
    Copy the full SHA
    01cbda5 View commit details
    Browse the repository at this point in the history

  5. Added DataReader for parallel training with one DB session 

    - Makes sure each solver accesses a different subset of the data
- Sequential reading of DB for performance
- Prefetches a configurable amount of data to host memory
- Distributes data to solvers in round-robin way for determinism
    @cypof
    cypof committed May 19, 2015
    Configuration menu
    Copy the full SHA
    0bd8238 View commit details
    Browse the repository at this point in the history