Code

Use mmap for external memory.#9282

Code

Merged

trivialfis merged 53 commits intodmlc:masterdmlc/xgboost:masterfrom trivialfis:ext-mmaptrivialfis/xgboost:ext-mmapCopy head branch name to clipboardJun 19, 2023

+588-264

Commits

Commits on Jun 11, 2023

• use mmap for external memory.

  trivialfis

  committed
  b1573d4

  Copy full SHA for b1573d4

• reduce size.

  trivialfis

  committed
  23a89b0

  Copy full SHA for 23a89b0

• cleanup.

  trivialfis

  committed
  fa5d460

  Copy full SHA for fa5d460

• abstract into a dmlc stream.

  trivialfis

  committed
  9ebd4ef

  Copy full SHA for 9ebd4ef

• cleanup.

  trivialfis

  committed
  3832fd5

  Copy full SHA for 3832fd5

• Cleanup.

  trivialfis

  committed
  18c3544

  Copy full SHA for 18c3544

• macos.

  trivialfis

  committed
  a04dc39

  Copy full SHA for a04dc39

• debug.

  trivialfis

  committed
  1e0405e

  Copy full SHA for 1e0405e

• Fix.

  trivialfis

  committed
  ba358af

  Copy full SHA for ba358af

• cleanup.

  trivialfis

  committed
  a6202d0

  Copy full SHA for a6202d0

Commits on Jun 12, 2023

• Cleanup.

  trivialfis

  committed
  117fb97

  Copy full SHA for 117fb97

• Skip python test.

  trivialfis

  committed
  9ee1643

  Copy full SHA for 9ee1643

• lint.

  trivialfis

  committed
  da00b6d

  Copy full SHA for da00b6d

• Add test.

  trivialfis

  committed
  05ce49b

  Copy full SHA for 05ce49b

• rename.

  trivialfis

  committed
  ed635d3

  Copy full SHA for ed635d3

• cleanup.

  trivialfis

  committed
  9b5c686

  Copy full SHA for 9b5c686

• Remove page in grad-based sampling.

  trivialfis

  committed
  1b0dab2

  Copy full SHA for 1b0dab2

• remove page in uniform sampling.

  trivialfis

  committed
  f383f76

  Copy full SHA for f383f76

• remove in no sampling.

  trivialfis

  committed
  68b838d

  Copy full SHA for 68b838d

• GPU initialization.

  trivialfis

  committed
  4b5d38f

  Copy full SHA for 4b5d38f

• use ctx.

  trivialfis

  committed
  4521f04

  Copy full SHA for 4521f04

• comment.

  trivialfis

  committed
  39ed218

  Copy full SHA for 39ed218

• lint.

  trivialfis

  committed
  a736125

  Copy full SHA for a736125

• doc.

  trivialfis

  committed
  a61a079

  Copy full SHA for a61a079

Commits on Jun 13, 2023

• windows mmap

  trivialfis

  committed
  4989269

  Copy full SHA for 4989269

• Merge branch 'ext-mmap-win' into ext-mmap

  trivialfis

  committed
  9068cc8

  Copy full SHA for 9068cc8

• compile

  trivialfis

  committed
  341c8fb

  Copy full SHA for 341c8fb

• Pad the file for windows.

  trivialfis

  committed
  2660c66

  Copy full SHA for 2660c66

• Avoid padding the data.

  trivialfis

  committed
  58c0d99

  Copy full SHA for 58c0d99

• Cleanup.

  trivialfis

  committed
  5baf5ca

  Copy full SHA for 5baf5ca

• debug.

  trivialfis

  committed
  925245c

  Copy full SHA for 925245c

• Fix.

  trivialfis

  committed
  bcf4cdb

  Copy full SHA for bcf4cdb

• Cleanup.

  trivialfis

  committed
  c195db1

  Copy full SHA for c195db1

• Cleanup.

  trivialfis

  committed
  9bbecf5

  Copy full SHA for 9bbecf5

• cleanup.

  trivialfis

  committed
  d3987e8

  Copy full SHA for d3987e8

• GPU compilation.

  trivialfis

  committed
  788f2b6

  Copy full SHA for 788f2b6

• lint.

  trivialfis

  committed
  e88f561

  Copy full SHA for e88f561

• log time.

  trivialfis

  committed
  6a02601

  Copy full SHA for 6a02601

• improve the tests.

  trivialfis

  committed
  9dd5812

  Copy full SHA for 9dd5812

Commits on Jun 14, 2023

• Timer.

  trivialfis

  committed
  94b8a0d

  Copy full SHA for 94b8a0d

• fix win leak

  trivialfis

  committed
  a4e11d3

  Copy full SHA for a4e11d3

• mingw

  trivialfis

  committed
  8cdbb87

  Copy full SHA for 8cdbb87

• reduce page number.

  trivialfis

  committed
  22ae3f6

  Copy full SHA for 22ae3f6

• Merge remote-tracking branch 'jiamingy/ext-mmap' into ext-mmap

  trivialfis

  committed
  3d76acc

  Copy full SHA for 3d76acc

• polishing.

  trivialfis

  committed
  c8726c3

  Copy full SHA for c8726c3

• Improve test.

  trivialfis

  committed
  b5b57a0

  Copy full SHA for b5b57a0

• fix.

  trivialfis

  committed
  076a788

  Copy full SHA for 076a788

• Forbid pointer to bool cast.

  trivialfis

  committed
  8b993ff

  Copy full SHA for 8b993ff

Commits on Jun 15, 2023

• cleanup.

  trivialfis

  committed
  6169fdc

  Copy full SHA for 6169fdc

• read-only.

  trivialfis

  committed
  914a186

  Copy full SHA for 914a186

• read-only.

  trivialfis

  committed
  f3e39ac

  Copy full SHA for f3e39ac

• fix.

  trivialfis

  committed
  de4f71c

  Copy full SHA for de4f71c

• Don't blame.

  trivialfis

  committed
  6552242

  Copy full SHA for 6552242