Skip to content

Commit

Permalink
feat: Benne implementation & python binding
Browse files Browse the repository at this point in the history 
Squashed commit of the following:

commit decb6ccdc71a4d7ca6ee6bf3ae78326d24f75ae9
Author: tuidan <1009575361@qq.com>
Date:   Sat Sep 9 02:36:02 2023 -0400

    add xtick for outlier distance

commit 5e4a6bc669ba72e2cc48eaf3ad0503faa32f9fbc
Author: wzru <wangshaun@outlook.com>
Date:   Tue Sep 5 16:15:39 2023 +0800

    fix: bench data 182246

commit 1afa88d06681085af878d20a83c62d5ba42ddf19
Author: wzru <wangshaun@outlook.com>
Date:   Tue Sep 5 15:53:26 2023 +0800

    fix: bench data 182246

commit 1f9e334bc758e1bdd59bb72effeb5001c0f11f56
Author: wzru <wangshaun@outlook.com>
Date:   Tue Sep 5 11:48:16 2023 +0800

    fix: draw.py

commit 1986e7b454f7f573599475b05ed67365bf786f5d
Author: tuidan <1009575361@qq.com>
Date:   Sat Sep 2 03:58:05 2023 -0400

    add breakdown script

commit ad27a2a04c2d9878f7548d9a5fb910b649be0a51
Author: wzru <wangshaun@outlook.com>
Date:   Sun Aug 27 14:02:29 2023 +0800

    fix: bench data 182246

commit aa2cd572b4437332042eb9e865b311447a02a38d
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Wed Aug 23 01:41:26 2023 -0400

    Update Benne.cpp

commit b583c3937bdc0c1e52f5bc8fabb2c46af2b81ea1
Author: tuidan <1009575361@qq.com>
Date:   Tue Aug 22 13:46:04 2023 -0400

    update Benne

commit 552ba4ef8aae01ca4eb763d8ee85dc288a20d2e8
Merge: dd7a64b c7c02c3
Author: wzru <wangshaun@outlook.com>
Date:   Tue Aug 22 00:04:43 2023 +0800

    Merge branch 'Benne' of https://github.com/intellistream/Sesame into Benne

commit dd7a64b119802df64d9d340476bbea5711a8b48d
Author: wzru <wangshaun@outlook.com>
Date:   Tue Aug 22 00:04:20 2023 +0800

    fix: bench data 182246

commit c7c02c3dcd1f10bb9dd7aee42f44de8b5b2f5402
Author: tuidan <1009575361@qq.com>
Date:   Sun Aug 20 16:19:50 2023 -0400

    "update draw for benne

    "

commit beff9d19a158748f143841f136108ce50a4ee3cc
Author: tuidan <1009575361@qq.com>
Date:   Sun Aug 20 15:45:17 2023 -0400

    update draw script for benne

commit 0ddd295d4252147dce10cfd58fbe6b4ad8d8563b
Author: wzru <wangshaun@outlook.com>
Date:   Mon Aug 21 02:45:12 2023 +0800

    fix: bench data 182246

commit 53d31e3c3ff74e1287449708407b9a5297d850e7
Author: wzru <wangshaun@outlook.com>
Date:   Sat Aug 19 00:34:02 2023 +0800

    fix: bench data 182246

commit 38b1828c7ad8e35aec9ec8ba8f46ebc06c924fa9
Author: wzru <wangshaun@outlook.com>
Date:   Fri Aug 18 23:37:47 2023 +0800

    fix: migration sink

commit 24c8f5a642538475964497e54af0917a8da4d354
Author: wzru <wangshaun@outlook.com>
Date:   Wed Aug 16 22:58:47 2023 +0800

    fix: bench data 182246

commit ccaac0859d94b362c21d508c577cb7bc74507347
Author: wzru <wangshaun@outlook.com>
Date:   Wed Aug 16 18:24:23 2023 +0800

    fix(benne): scripts & output csv names

commit 4e5e95af30a8f508db5f31a01644e47055ddb03a
Author: wzru <wangshaun@outlook.com>
Date:   Wed Aug 16 16:48:40 2023 +0800

    chore(script): mv former scripts into sigmod23

commit c4d0616d36e90e0306b11c9b3cb6eaf5198a0f7b
Merge: 96a0388 42ae4c2
Author: tuidan <1009575361@qq.com>
Date:   Wed Aug 16 02:21:32 2023 -0400

    Merge branch 'Benne' of github.com:intellistream/Sesame into Benne

commit 42ae4c2260c0a2aa01bfedcd94c1136f1f316104
Author: tuidan <1009575361@qq.com>
Date:   Wed Aug 16 02:17:26 2023 -0400

    update README

commit 96a0388d06bdd902257c85a9435b5c6e9bd71f69
Merge: e3882ab 5c61c39
Author: tuidan <1009575361@qq.com>
Date:   Wed Aug 16 01:59:05 2023 -0400

    Merge branch 'Benne' of https://github.com/intellistream/Sesame into Benne

commit e3882ab5e8b8ef70099db23040667bc7844321c4
Author: tuidan <1009575361@qq.com>
Date:   Wed Aug 16 01:57:55 2023 -0400

    modify result format

commit 5c61c394a8d966a8ec1f3e156c0efcf38d24632b
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Wed Aug 16 01:39:59 2023 -0400

    Update README.md

commit e3e7ff626a90f2aed271ecb9e5faaabd399ca82a
Author: tuidan <1009575361@qq.com>
Date:   Wed Aug 16 01:39:15 2023 -0400

    modify experimental result's format

commit 34f2ffc7ea35ed194cc96d530caae270720fd259
Author: wzru <wangshaun@outlook.com>
Date:   Sun Aug 13 23:22:45 2023 +0800

    fix: bench data 182246

commit 01b89d36b7a63c8127c2e8a31536788bc99db041
Author: wzru <wangshaun@outlook.com>
Date:   Sun Aug 13 19:55:01 2023 +0800

    feat: bench data 182246

commit 9dd5974407e64bd9bd84b4813572bfa7158f95ad
Author: WangZhengru <wangshaun@outlook.com>
Date:   Thu Aug 10 00:25:36 2023 +0800

    feat: benne acc w/o migration & correct timer

commit 638eb94baa63694fc0eb81990e70c233e878997c
Author: WangZhengru <wangshaun@outlook.com>
Date:   Wed Aug 9 23:47:50 2023 +0800

    refactor: PerfRes

commit fae728a38924162969e5fb10e6cc65879c405114
Author: WangZhengru <wangshaun@outlook.com>
Date:   Wed Aug 9 22:55:51 2023 +0800

    refactor: BenchRes

commit b251e494d33daace95c3b855c3a4c0086285129d
Author: WangZhengru <wangshaun@outlook.com>
Date:   Wed Aug 9 21:28:01 2023 +0800

    refactor: RunBenchmark

commit fa3af64d9afbbe3304cad182d3b7281a07d4e131
Author: WangZhengru <wangshaun@outlook.com>
Date:   Wed Aug 9 18:05:26 2023 +0800

    refactor: runBenchmark

commit 62e258c161179a47c1444fa5fa6468dbda3c6f7c
Author: WangZhengru <wangshaun@outlook.com>
Date:   Wed Aug 9 16:48:08 2023 +0800

    chore: clean param

commit b758916a65075617dbd08180071ab59be54965fa
Author: WangZhengru <wangshaun@outlook.com>
Date:   Wed Aug 9 00:03:46 2023 +0800

    feat: python binding

commit ba3d5a7aa0b902d5d329b0a113a78307b560f8c0
Author: WangZhengru <wangshaun@outlook.com>
Date:   Sat Aug 5 16:14:09 2023 +0800

    chore(dataset): rm test datasets

    Former-commit-id: 52052f4eb96a5f4a0cc388cd84d658b91317d4ed

commit ad10a48aba2b3a8aa972521cf15628b0e3620a70
Author: WangZhengru <wangshaun@outlook.com>
Date:   Sat Aug 5 16:09:58 2023 +0800

    chore(dataset): rm dataset files

    Former-commit-id: 928f896729d8489130d86acbcf193a4d324fc217

commit 613555b557da6a2fb34df7beedd98ba869c941a1
Author: wzru <wangshaun@outlook.com>
Date:   Wed Aug 2 01:34:30 2023 +0800

    feat: bench data 2532

    Former-commit-id: ee70e3751c6de182473da5c29b3a24e1d59624a0

commit d1098a508bdd6087fea3b340015896a5c43d2e32
Author: wzru <wangshaun@outlook.com>
Date:   Mon Jul 31 00:41:56 2023 +0800

    feat: update script

    Former-commit-id: 014ec151ecbccd7a0f5900384df8e9fa694291e0

commit 7ccd7173e52f08b7e43365482973cc5e06429b21
Author: WangZhengru <wangshaun@outlook.com>
Date:   Sun Jul 30 23:28:14 2023 +0800

    feat: change_log, first_algo, mig_us, det_us

    Former-commit-id: 5471ca9818f183f68903dc915c06cb1272828c12

commit 3622cd0ce346da6fb217b5597a614985fb48e998
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Sat Jul 22 22:06:13 2023 -0400

    Update Benne.cpp

    Former-commit-id: e0b8e5628ff07327c602a3d385e48f493c8c38f7

commit 95cdb0a375653b3d63f3e6fee76a0fde595d8598
Author: wzru <wangshaun@outlook.com>
Date:   Thu Jul 20 02:18:25 2023 +0800

    feat: fine-tune

    Former-commit-id: ca5ad03bde0eacdcd12d8ee855ba6e0aec121d9c

commit af10163cb3e4af09d2c5299568d73ad793d3b15a
Author: wzru <wangshaun@outlook.com>
Date:   Wed Jul 19 21:39:50 2023 +0800

    feat: fine-tune

    Former-commit-id: 04ba2b4c8b08b337f4fc729b64c756624cfc8d1b

commit 170f76dec4c1a9672bbcf91040d4737ab5ea9345
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Tue Jul 18 11:43:08 2023 -0400

    Update Benne.cpp

    Former-commit-id: fa75cedc34f781b9310cc0ad3e6ab5d414fffb41

commit 1e3789c54091e2592d291ddf07d6169e51ce2187
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Tue Jul 18 11:40:44 2023 -0400

    Update Benne.cpp

    Former-commit-id: f1a061e55da11e938bf2d047733421ca7cb78747

commit 9cc98e933a241e4cd51b042431fcd59ef1131f1e
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Tue Jul 18 11:22:44 2023 -0400

    Update Benne.cpp

    Former-commit-id: 9065c4e4b573421f4c276ae7e445693fbc966099

commit e2bd0f7f2986746025e4b9d6715b2927f7f1275c
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Tue Jul 18 11:21:17 2023 -0400

    Update Benne.cpp

    Former-commit-id: 44c4d8e9d91f50e174e8185096a2f87f79983812

commit 02b64363bc75d10cbae779057d604c30454a8216
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Tue Jul 18 11:20:48 2023 -0400

    Update Benne.cpp

    Former-commit-id: b3342eb94634ff46825ac313c9f31ca93792dc2d

commit 7a4bf34a2973f6fc99797eb0eae10f53201442bc
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Mon Jul 17 10:23:31 2023 -0400

    Update Benne.cpp

    Former-commit-id: d78548d8de57f21b22a638e4b2e702a4d3d3fc08

commit 29e78e7c2c91ee181244a059fe1dcf5f13ecda54
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Mon Jul 17 10:21:36 2023 -0400

    Update Benne.cpp

    Former-commit-id: acb378e09ba59607d1f2ccebbbaccd67633bf679

commit feff372815481dfc7d6f821e01fc4d16458f4fa4
Author: wzru <wangshaun@outlook.com>
Date:   Mon Jul 17 20:44:36 2023 +0800

    feat: fine-tune

    Former-commit-id: d471bc52df4aa4f8db2a5f0085d11c7d851920cb

commit 30dba42b9404c1dbd6376e362779f31c05038e5c
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Mon Jul 17 08:32:34 2023 -0400

    Update Benne.cpp

    Former-commit-id: 0d64e000403eb7f4c9639322cf1af9a5fd1bf027

commit 2609ee175ed082b09a8de9f09e87ef3f6f418cf1
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Mon Jul 17 07:55:15 2023 -0400

    Update purity.csv

    Former-commit-id: 493ca35624a6c809132ec015ae94f937db87b9a8

commit 11ce4d56c0f09b1814bf2029dca5e1eceed7ddc7
Author: wzru <wangshaun@outlook.com>
Date:   Thu Jul 13 22:43:53 2023 +0800

    feat: fine-tune

    Former-commit-id: 992ace864df1351db42b60514c30df2a36896c13

commit 306e03ca48bc7806d95741ca13602ef37480c122
Author: wzru <wangshaun@outlook.com>
Date:   Thu Jul 13 21:37:55 2023 +0800

    fix: DBStream

    Former-commit-id: fc6cfbf620ae83046c8a674e29a87ed6fb7bf3f5

commit 70852794a3ae1fd03510d1c6e8747ecf73c35a63
Author: wzru <wangshaun@outlook.com>
Date:   Thu Jul 13 15:48:46 2023 +0800

    feat: fine-tune

    Former-commit-id: 37b217daa9cc3a1c710599fa0f63ff3e218eacd3

commit d4520fcc75295ea31516e95bfb91ced314f5b85d
Author: wzru <wangshaun@outlook.com>
Date:   Tue Jul 11 00:14:36 2023 +0800

    feat: fine-tune

    Former-commit-id: 680a56cfb0271a63bd8b62ab9468a2aa9853b148

commit c6213d4e6afeff278fc432b8ba372013609ccb4e
Author: WangZhengru <wangshaun@outlook.com>
Date:   Mon Jul 3 00:17:34 2023 +0800

    feat: add benne.sh

    Former-commit-id: 9d06dc7a51e4c19c74ad0f35bac9811c636ac3ec

commit 01549bd9d6da6ef8301d3397d05706b8fff5c0bd
Author: WangZhengru <wangshaun@outlook.com>
Date:   Sun Jul 2 12:56:58 2023 +0800

    feat: update README

    Former-commit-id: ff80487cfd94ef221c823a58334426a13a6aa4cb

commit 03b4165627814cb3792dc46ec4eaa44a4ac5423a
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Sat Jul 1 17:17:43 2023 -0400

    Update README.md

    Former-commit-id: a9e57c1eb5324e0d9f6f4839d152e564ff4a6748

commit 268b2ff79d152be4d398b37ed9f4cfe921e00803
Author: tuidan <1009575361@qq.com>
Date:   Tue Jun 27 12:39:10 2023 -0400

    fix: update selection

    Former-commit-id: cffbd8a934d4677dc784d2bf890effeb8e6d228a

commit 02a8ce492fc1de563ca87533fe0d6538f2f8ae54
Author: tuidan <1009575361@qq.com>
Date:   Tue Jun 27 12:21:08 2023 -0400

    feat: sink center for algo change

    Former-commit-id: b306ba84a4993236a6fbacaea108adbf2375eb43

commit ea2aaa547a7456ce73f36915219730051ca0d536
Author: tuidan <1009575361@qq.com>
Date:   Tue Jun 27 12:20:44 2023 -0400

    feat: sink center for algo change

    Former-commit-id: 8e4f02b028d18a058e572ca7e9dc88de6ceffcec

commit 7311de3c098c23fba7edb70e38d7674e154dff96
Author: tuidan <1009575361@qq.com>
Date:   Tue Jun 27 04:41:11 2023 -0400

    feat: update benne efficiency

    Former-commit-id: f1913c08b1fea9f5705855d1dff7d4ac0dd331f4

commit 3f6d115bd4c378803a55682fb4fb6c52149fd045
Author: tuidan <1009575361@qq.com>
Date:   Tue Jun 27 03:59:23 2023 -0400

    feat: finish G16

    Former-commit-id: 9bde0a42b1f5d097a0ce74a97271aa3d8ce17cad

commit b55edc619274ee056fdd390e10b453b2cb7fa04c
Author: tuidan <1009575361@qq.com>
Date:   Mon Jun 26 03:33:34 2023 -0400

    feat: upadte V16

    Former-commit-id: f0438c4437c89423faa1c3d05f98db899fd45a13

commit f41ec65c06cdff550084a3e143c53cff8b866f30
Author: WangZhengru <wangshaun@outlook.com>
Date:   Sun Jun 25 21:59:25 2023 +0800

    fix: ds_changed

    Former-commit-id: 4e9d376f47d3dc2b91512d9322b9dc244062ce2e

commit 505a3c9c92504dbaca17750d24c788e506bfd156
Author: WangZhengru <wangshaun@outlook.com>
Date:   Sun Jun 25 17:59:34 2023 +0800

    fix: incre works

    Former-commit-id: ca75111f2886ae3bb305e2624f6de4afafc36f89

commit 4d7992e7f430724a8a37b99869402631b57e56e4
Author: WangZhengru <wangshaun@outlook.com>
Date:   Sun Jun 25 17:41:54 2023 +0800

    feat: oneshot works

    Former-commit-id: 86d3344a5ff15075dadbd2222dcf4af7bf13b8a6

commit 72d4ac08451416b17d0fc6081e2b28346088939f
Merge: 3bc0aec 0eedd44
Author: WangZhengru <wangshaun@outlook.com>
Date:   Sun Jun 25 15:26:44 2023 +0800

    Merge branch 'Benne' of github.com:intellistream/Sesame into Benne

    Former-commit-id: 5f1e01bf9d9916f5e1695a192c76b83be3b52247

commit 3bc0aecad03ed16dbb1f34cdc37c02a6892f993e
Author: WangZhengru <wangshaun@outlook.com>
Date:   Sun Jun 25 15:26:35 2023 +0800

    feat: Benne accuracy works

    Former-commit-id: 5acb6fba27f5323ebda2d1da69557a83b08a23c1

commit 0eedd440f3e5112155905e88685c9e79a3db1541
Author: tuidan <1009575361@qq.com>
Date:   Sun Jun 25 02:00:53 2023 -0400

    feat: update selection

    Former-commit-id: 6985dbb5d686ae3bd12f9fbdf1f182d4db61190e

commit f9acb9cef5b6fd390eea556c2507923a0f518ae8
Author: WangZhengru <wangshaun@outlook.com>
Date:   Sat Jun 24 23:29:15 2023 +0800

    fix: works on balance

    Former-commit-id: bf44a0331bd39a6e2f1196aea77419101ee40b98

commit 2ca053659a8ef9c415606c2cf28b4ae87ce88cf8
Author: WangZhengru <wangshaun@outlook.com>
Date:   Sat Jun 24 23:19:25 2023 +0800

    fix: compiling

    Former-commit-id: 1f89895dcc8633f39585071821109ce236ba87c7

commit db2386f398bfee81be0d7cd2a308ee45cb269dfe
Author: tuidan <1009575361@qq.com>
Date:   Sat Jun 24 03:03:58 2023 -0400

    feat: add dispersion calculation

    Former-commit-id: 0717b8cabf69d3b5b72d1cbc9a6dc3c9be8f7203

commit a45c42e0aff1aab5b19d97487adafc2673c163df
Author: tuidan <1009575361@qq.com>
Date:   Sat Jun 24 03:03:39 2023 -0400

    feat: add dispersion calculation

    Former-commit-id: e66cf536ec6e222a116212e84a25898ee1681596

commit 32d9c4b8614164d4bf2b594e8c7a9cbb71231d9a
Author: tuidan <1009575361@qq.com>
Date:   Fri Jun 23 22:27:10 2023 -0400

    feat: modify benne

    Former-commit-id: 4e2046730620c15dd8bd61bc4176764faebb9d94

commit dd9defe04ff520e543cf60a44692e2b5f5e9436f
Author: tuidan <1009575361@qq.com>
Date:   Fri Jun 23 21:28:12 2023 -0400

    feat: initial code of Benne

    Former-commit-id: 5e8390f2fb42e98ee1d807de9dad46907ca70bf9

commit be7f08c8274fd5db614df316daefb97b38b72ecd
Author: tuidan <1009575361@qq.com>
Date:   Fri Jun 23 02:43:52 2023 -0400

    feat: init benne

    Former-commit-id: 507458fb5ab1672736c87c75446d636f3506a61e

commit 00039a052393cebb8c5897965c6508e7b98b6015
Author: Xin Wang <40883104+tuidan@users.noreply.github.com>
Date:   Sat Jun 17 19:14:58 2023 -0400

    feat: modify README (#175)

    Former-commit-id: f3570c24bb92c954d4f53e7b8149ace6de333fe0

commit 120490c2211750f976cb10820001a04bf7a38b44
Author: Shuhao Zhang (Tony) <tonyzhang19900609@gmail.com>
Date:   Sat May 6 19:44:35 2023 +0800

    Update README.md

    Former-commit-id: 1c0ebc9fc69248d76356a5795bb7261494cb70ea

commit 47edcd7437acfba3f3a3e3207f2d588c6d91f73e
Author: Shuhao Zhang (Tony) <tonyzhang19900609@gmail.com>
Date:   Sat May 6 19:42:15 2023 +0800

    Update README.md

    Former-commit-id: d7d5f8cb92503a8fc5b0224a5a80eb85b878576f

commit a8c87955ba2c24ebe5cafcd9892582f80e188f8e
Author: WangZhengru <wangshaun@outlook.com>
Date:   Wed Apr 12 00:00:06 2023 +0800

    feat(sigmod23): general stream clustering algorithm & benchmark (#174)

    * fix: fix DStream param

    * feat: add eds.sh

    * feat: add flag run_eval

    * feat: if StreamKM++ run offline

    * fix: fix opt-insects.sh

    * fix: use std::queue

    * fix: use boost::lockfree::spsc_queue

    * fix: fix DStream clock()

    * feat: insects best param

    * fix: add boost to CI

    * fix: no groupBy when run_eval==false

    * fix: fix output result size

    * fix: fix output result size

    * fix: no eval when results.size==0

    * feat: add run_pur & run_cmm flags

    * fix: fix arr_rate wait

    * feat: add datasets scripts

    * feat: optmize DBStream offline

    * fix: fix CluStream timer

    * fix: fix DenStream timer

    * fix: fix DStream seconds()

    * fix: fix SystemTest.Birch

    * feat: update eds.sh

    * DBStream offline problems fixed (#158)

    * DenStream outlier cluster problem fixed

    * feat: optmize DBStream offline

    * Fix offline problem of DBStream

    Co-authored-by: s1926539 <45706792+GabrielWuNR@users.noreply.github.com>
    Co-authored-by: WangZhengru <harryshaunwang@outlook.com>

    * Revert "DBStream offline fixed (#158)" (#159)

    This reverts commit 9c9ab2c8e4253477ecfd17b87a6f6918d41b1898 [formerly 06c5fab29a05e31edfbc10761f68b801e5c5bac8].

    * change Landmark from [20000,20000,10000]to [0,0,0] for G1,G3 and G8 on KDD

    * add V10

    * feat: add V10 (#160)

    * fix: fix V10 algo_name

    * feat: modify et_s to be online timer

    * fix: fix spsc_queue cap

    * fix: fix et.size()

    * feat: optimize G5

    * feat: optimize EDMStream

    * feat: add progress_bar for online

    * feat: add draw scripts (#161)

    * add V10

    * add python_script

    * modify python_script

    * feat: optimize param for performance

    * feat: optimize cf tree param for eds

    * modify V1，V2，V3

    * feat: update insects.sh

    * feat: update insects.sh

    * feat: finish draw scripts

    * feat: add win_timer

    * feat: add window model timer (#163)

    * dstream add win_timer

    * DBstream add win_timer

    * DenStream add win_timer

    * EDMStream add win_timer

    Co-authored-by: tuidan <1009575361@qq.com>

    * chore: git mv scripts

    * fix: fix SLKMeans win_timer

    * fix: modify /home/shaun to $HOME

    * feat:add draw function for eds varying arrival rate

    * feat: update scripts & raw data

    * fix: update G9 on kdd purity

    * feat: add INSECTS

    * chore: use create_symlink instead of cp datasets

    * chore: use create_symlink instead of cp datasets

    * chore: use static link

    * update draw.py

    * update draw.py

    * feat: Birch use generic

    * Update cmake.yml

    * Update cmake.yml

    * Update cmake.yml

    * fix: fix V9 seconds()

    * feat: DenStream faster on EDS/ODS

    * add kmeansK for offline clustering parameter

    * fix bugs about kmeansk

    * fix bugs about kmeansk

    * chore: add KMeans K

    * feat: add -march=native

    * fix: fix CMakeLists.txt

    * feat: fine-tune experiment

    - https://app.asana.com/0/1202552003774755/1202992694077623/f

    * fix: TimeMeter::printTime cout

    * fix: Birch uses G6

    * fix: fix gtest

    * fix: update bench/*.sh

    * fix: fix G1 bugs (#164)

    * fix: fix DesignTest.V6/V7/V8

    * fix: add k for StreamKMeans

    * fix: fix StreamKM KMeans

    * fix: fix sensor.sh

    * feat: update latest benchmark data

    * fix: update latest benchmark data

    * fix: fix benchmark data

    - G5 on CoverType
    - CluStream on Sensor

    * feat: add cmm-ods

    * feat: add purity-ods

    * feat: increase arr rate on EDS

    * feat: fine-tune G4/G7/G8

    - speed up G4/G7 on INSECTS
    - rollback G8 on CoverType/INSECTS

    * fix: update latest benchmark data

    * feat: add throughput data for EDS/ODS

    * feat: add extra experiments & run_offline flag

    - landmark
    - sliding
    - outlier_cap
    - run_offline

    * feat: add cluster_size & outlier_size for generic

    - add damped window experiments

    * feat: add dim sum time

    * feat: add G1-G10 perf on FCT/INSECTS/KDD99/sensor

    * feat: add .clang-format

    * feat: timer & buffer options for outlier detection

    * fix: landmark & timer

    * feat: DataSink output result to file

    * fix: test

    * fix: test

    * fix: GenericTest.V5

    * feat: run_cmm=false

    * fix: GenericTest.V5

    * feat: add bench_id

    * fix: run_cmm=true

    * fix: cluster_size & outlier_size

    * feat: no outliers in eval

    * feat: add scripts for G11/G12/G13

    * chore(ticat): remove marktime bench_begin

    * feat(eval): NMI metric (#166)

    * add nmi

    * add nmi

    * fix: fix NMI & real-world algo RunOffline

    * modify DBSCAN

    * feat: add KDD99_F

    * fix: nmi & outlier_cap

    * modify DBSCAN

    * feat: add G14 enum

    * revert(offline): put outliers to eval

    * feat: add warning when benchmark in debug mode

    * fix: G1 & Birch tests

    * fix: G5/G6/G12 tests

    * chore: use KDD_F

    * chore: sql scripts

    * fix bug of DBSCAN

    * Update DStream

    * feat: speed up DStream

    * fix: tests

    * feat: add landmark experiments

    * fix: GenericTest.V5

    * chore: mv class MeyersonSketch into KMeansSummary

    * fix: SLKMeans use index as timestamp

    * fix: fine-tune param after outlier modification

    * fix: fix meta="$meta"

    * fix: scripts

    * feat(ds): MeyersonSketch

    * fix: scripts

    * chore: mv original CoresetTree into LandmarkWindow

    * feat: slow CoresetTree

    * fix: CoresetTree with buckets

    * perf: speed up Coreset Tree

    * Modify EDMStream & G10

    * Modify G10

    * feat: timer check on outliers

    * feat: add G14/G15 scripts

    * perf: speed up DStream

    * perf: speed up some scripts

    * Modify G9 and DStream

    * fix: scripts

    * update parameter tuning

    * feat: new exp data 230114

    * perf: speed up V9

    * fix: V9 gap

    * fix: V9

    * fix: G1 & G9 params

    * fix: GenericTest.V5

    * fix: scripts

    * fix: bench data 112926

    * fix: Birch & V6

    * fix: bench data 112926

    * fix: scripts

    * fix: birch

    * fix: DStream

    * fix: V9

    * update: script/readme

    * update: script/readme

    * fix: V9 and DStream

    * fix: V9 and DStream

    * feat: bench data 30855

    * feat: add mock data

    * fix: V9 & V10 timer

    * fix: V9 & V10 bench data

    * fix: scripts

    * feat: bench data 22208

    * fix: bench data 22208

    * feat: extra.dstream

    * fix: extra.dstream data

    * feat: extra.step & output num_grids/gap

    * feat: step-eds cluster_size & outlier_size

    * fix: OutlierDetection-B

    * feat: bench data 235808

    * fix: bench data 235808

    * feat: change v10 from OD-B to OD

    * feat: change v9 from OD-B to OD

    * fix: G10 & G15 bench data

    * fix: G10 & G15 bench data

    * fix: G4 use outlier buffer

    * fix: bench data 235808

    * fix: bench data 235808

    * fix: bench data 235808

    * fix: bench data 235808

    * fix: bench data 235808

    * fix: sliding with outlier buffer

    * fix: DStream on sensor segmentation fault (#168)

    * fix: scripts

    * feat: update README.md

    * fix: DStream make_pair (#169)

    * Fix DSTREAM problems

    * fix DStream make_pair problem

    * fix: DStream on sensor (#171)

    * fix DStream make_pair problem 2.0

    * Now all fixed

    * Now all fixed

    * problem in adjust new labels, forgot brackets

    * fix: bench data 164201

    * fix: G3 G4 G5

    * fix: G3 & G4 covertype

    * fix: change absolute window for DenStream

    * feat: bench data 230318

    * feat: upload ODS.txt

    * feat: fine-tune & ODS

    * feat: bench data 2441

    * revert: ODS -> EDS_O

    * fine-tune temp save

    * feat: fine-tune

    * feat: bench data 002931

    * feat: bench data 083610

    * feat: fine-tune

    * chore: uncomment 0-7 algo

    * feat: bench data 221644

    * fix: fine-tune

    * fix: fine-tune CluStream & DStream

    * feat: bench data 011452

    * fix: fine-tune

    * feat: bench data 204913

    * fixed bugs in for loop (#172)

    * feat: bench data 103646

    * fix: DStream on KDD/sensor/EDS (#173)

    * feat: info ignore in release mode

    * feat: bench data 120331

    * fix: DStream on FCT qps

    * fix: modify draw script

    * fix: modify draw script

    * fix: extra.offline

    * feat: REAME

    * fix: fine-tune

    * feat: bench data 000029

    * feat: add citation

    * feat: add citation

    * fix: extra.damped

    * fix: extra.damped data

    * fix: update draw script

    * chore(benchmark): remove benchmark/benchmark

    * chore: remove git submodules

    * fix: tests.yml

    * chore(prof): remove gprof files

    * ci: add clang-format-check.yml

    * chore(benchmark): remove benchmark/test

    * fix: tests.yml

    * style: clang-format

    * fix: ci

    * chore: remove ConceptDrift

    * feat: remove V1-V8

    * chore: remove scripts/tune and scripts/mock

    ---------

    Co-authored-by: NRGabrielF <104135042+NRGabrielF@users.noreply.github.com>
    Co-authored-by: s1926539 <45706792+GabrielWuNR@users.noreply.github.com>
    Co-authored-by: tuidan <40883104+tuidan@users.noreply.github.com>
    Co-authored-by: tuidan <1009575361@qq.com>
    Co-authored-by: Shuhao Zhang (Tony) <tonyzhang19900609@gmail.com>
    Co-authored-by: PierreWoL <118140290+PierreWoL@users.noreply.github.com>
    Former-commit-id: f0e048e81fa43c7ac4151155a5642eb5994cd8b8

commit 398c75a51575fa2903373b93be5f908beec2ae61
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Thu Mar 3 13:48:36 2022 +0800

    Synthetic (#82)

    * Modify: turn Grid Width from int  to double

    * add parameter of DStream in utils

    * benchmark modify

    * benchmark modify --add bash

    * benchmark modify --add bash

    * just for the accuracy test

    * benchmark modify --add bash

    * benchmark modify --add bash

    * benchmark modify --add bash

    * Comment out SESAME_DEBUG and SESAME_INFO
    Reduce the overhead cost of this category
    Output directly via std::cout

    * experiment code all complete

    * forget local dataset

    * forgot this one

    * timer modify for CDF

    * timer modify for CDF

    * timer reduce

    * fix some double issue

    * 1.26

    * 1.27

    * fix window issue of DBStream and DensSTREAM

    * 1.28

    * All bugs and low performance issues has been eliminated, Figure deadline 1.31 12 p.m.

    * All bugs and low performance issues has been eliminated, Figure deadline 1.31 12 p.m.

    * All bugs and low performance issues has been eliminated

    * benchmark modify for parameter in DStream

    * benchmark modify for parameter in DStream

    * add cumulative time

    * 2.3

    * 2.6

    * 2.6CMM

    * 2.7

    * 2.7

    * 2.7 diamond

    * 2.8

    * 2.8

    * timeter

    * 2.8

    * timeter

    * timeter

    * 2.8

    * 3.3 Commit

    * Update Benchmark.cpp

    Co-authored-by: s1926539 <zhenyu.wun@gmail.com>
    Former-commit-id: efbd40084c591059af851f71bdafd96ab021f524

commit 92313a0ded0ee3c2faddfc89f03b404a73b3d681
Author: Shuhao Zhang (Tony) <tonyzhang19900609@gmail.com>
Date:   Wed Mar 2 11:29:05 2022 +0800

    Create LICENSE

    Former-commit-id: fed405975898f49655da4d6ca903f73637d14a0c

commit 8ed54e95e2f3c786546cca794c05534613fec2a6
Author: Shuhao Zhang (Tony) <tonyzhang19900609@gmail.com>
Date:   Wed Mar 2 10:58:27 2022 +0800

    Update README.md

    Former-commit-id: 1376e860b696af053b9399bf6c2fea6d72c1871a

commit 9b23fb60015ca36f241754ac704b5ed7f868f223
Author: Shuhao Zhang (Tony) <tonyzhang19900609@gmail.com>
Date:   Fri Feb 11 19:23:30 2022 +0800

    Update README.md

    Former-commit-id: 77097d95c288fd6458842ff63f3048a04dd6b751

commit fd8670c161ebe220f6f89abbf08fad9f15cd955b
Author: Shuhao Zhang (Tony) <tonyzhang19900609@gmail.com>
Date:   Sun Feb 6 11:13:28 2022 +0800

    Benchmark modify wzy (#80)

    Note, there are still two bugs in clustream and streamKM. The "out of boundary" bug.

    Former-commit-id: bc91b6bc2ea121dcf74e6a13ab129ee89d96c1e8

commit d1127d281e31fc7c50b27b976f825de452cd83fd
Author: Gabriel WU <45706792+GabrielWuNR@users.noreply.github.com>
Date:   Mon Dec 13 18:14:42 2021 +0800

    Dstream finish (#79)

    * ReWrite DStream based on MOA

    * ReWrite DStream based on MOA

    * ReWrite DStream based on MOA, left Debug after refactor all algorithms

    * Evaluation wx (#70)

    * add purity

    * add preparations for CMM, still have problem about driver

    * CMM done, but need to remove hard code

    * still need to change voteMap function

    * purity wx (#71)

    * add purity

    * add preparations for CMM, still have problem about driver

    * CMM done, but need to remove hard code

    * still need to change voteMap function

    * fix bug of purity

    * Update README.md

    * Add Outlier Detection Part (#72)

    * Update README.md

    * pull request from DataSource-yzh to main (#74)

    * Updated DataSource

    * Removed hard-code in DataSource::load

    * Modified timer and removed hard-coded queue initialization

    Co-authored-by: Zhonghao Yang <zhonghao_yang@zhonghaosMBP.dhcp.sutd.edu.sg>
    Co-authored-by: Zhonghao Yang <zhonghao_yang@mymail.sutd.edu.sg>

    * Fix timer (#75)

    * Construct Time measurement

    * Constructing Time measurement

    * Re write timer hpp

    * Construct Time measurement

    * Constructing Time measurement

    * Re write timer hpp

    * Finish timer, need final debug ... X_X

    * A LITTLE BUG IN OUTLIER DETECTION PART, NEED TO ADJUST ACCUMULATE TIMER

    * turn accumulate timer into more simple one

    * turn accumulate timer into more simple one

    * fix all

    * Subtract connection based offline refinement. which can be used later in refactor part (#76)

    * modify DStream

    * FU DStream, no bugs, performance is terrible

    * FU DStream, no bugs, performance now OK

    * ReWrite DStream based on MOA, left Debug after refactor all algorithms

    * FU DStream, no bugs, performance now OK

    * ReWrite DStream based on MOA, left Debug after refactor all algorithms

    * delete most of the print

    * ReWrite DStream based on MOA, left Debug after refactor all algorithms

    * delete most of the print

    * ReWrite DStream based on MOA, left Debug after refactor all algorithms

    * delete most of the print

    * ReWrite DStream based on MOA

    * ReWrite DStream based on MOA

    * ReWrite DStream based on MOA, left Debug after refactor all algorithms

    * modify DStream

    * FU DStream, no bugs, performance is terrible

    * FU DStream, no bugs, performance now OK

    * ReWrite DStream based on MOA, left Debug after refactor all algorithms

    * FU DStream, no bugs, performance now OK

    * recover test cmake

    Co-authored-by: tuidan <40883104+tuidan@users.noreply.github.com>
    Co-authored-by: Tony <tonyzhang19900609@gmail.com>
    Co-authored-by: Zhonghao Yang <62000831+zhonghao-yang@users.noreply.github.com>
    Co-authored-by: Zhonghao Yang <zhonghao_yang@zhonghaosMBP.dhcp.sutd.edu.sg>
    Co-authored-by: Zhonghao Yang <zhonghao_yang@mymail.sutd.edu.sg>
    Former-commit-id: 040fcee8952688db6ff2d846cfcde9b608c45c50

commit f78344577c1d5608c3771dcb61994f02d9736ac4
Author: Gabriel WU <45706792+GabrielWuNR@users.noreply.github.com>
Date:   Wed Nov 24 08:39:59 2021 +0800

    Subtract connection based offline refinement. which can be used later in refactor part (#76)

    Former-commit-id: a73099182fcfead115da2e6f8baa0655d131c9f3

commit 5f33b8536c20c758b0cc13ad44013cdebced313b
Author: Gabriel WU <45706792+GabrielWuNR@users.noreply.github.com>
Date:   Fri Nov 12 02:43:09 2021 +0800

    Fix timer (#75)

    * Construct Time measurement

    * Constructing Time measurement

    * Re write timer hpp

    * Construct Time measurement

    * Constructing Time measurement

    * Re write timer hpp

    * Finish timer, need final debug ... X_X

    * A LITTLE BUG IN OUTLIER DETECTION PART, NEED TO ADJUST ACCUMULATE TIMER

    * turn accumulate timer into more simple one

    * turn accumulate timer into more simple one

    * fix all

    Former-commit-id: 4f1d8fd60b961a0987845fac26c5b6eb4024f50f

commit edfe943ff1964c730b111c639fa9a648cb589af6
Author: Zhonghao Yang <62000831+zhonghao-yang@users.noreply.github.com>
Date:   Wed Nov 3 22:02:48 2021 +0800

    pull request from DataSource-yzh to main (#74)

    * Updated DataSource

    * Removed hard-code in DataSource::load

    * Modified timer and removed hard-coded queue initialization

    Co-authored-by: Zhonghao Yang <zhonghao_yang@zhonghaosMBP.dhcp.sutd.edu.sg>
    Co-authored-by: Zhonghao Yang <zhonghao_yang@mymail.sutd.edu.sg>
    Former-commit-id: 6dd6f062d722d5b47eefbfb11ffdef1b7463617c

commit 2f19050a0978aae696268eff3f739b242350f037
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Wed Nov 3 17:13:53 2021 +0800

    Update README.md

    Former-commit-id: 780584c8a9f937aeb14218830bd846106222ebd0

commit 54be31bab7cb83cfa37d9ed60bf6ca94b463f6bd
Author: Gabriel WU <45706792+GabrielWuNR@users.noreply.github.com>
Date:   Tue Nov 2 18:22:35 2021 +0800

    Add Outlier Detection Part (#72)

    Former-commit-id: 1b4f4e1bfa2b3401daaec13b6608c6855077a66e

commit f14c98105d7f749a868c829fad456ea1f3cde7ef
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Mon Nov 1 10:35:53 2021 +0800

    Update README.md

    Former-commit-id: ae50f31d3e11a38e2e152d52e67cfca3c615fe2a

commit e4eaf6458193ad75ac7f2d7120c0c01f352ffb6f
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Thu Oct 28 13:06:56 2021 +0800

    purity wx (#71)

    * add purity

    * add preparations for CMM, still have problem about driver

    * CMM done, but need to remove hard code

    * still need to change voteMap function

    * fix bug of purity

    Former-commit-id: d4b3492ef25021fbe3889bfa5678f681b653c3c0

commit 4eb5c67f440d5167c6487c53857f05e4b3b097c0
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Wed Oct 27 16:11:18 2021 +0800

    Evaluation wx (#70)

    * add purity

    * add preparations for CMM, still have problem about driver

    * CMM done, but need to remove hard code

    * still need to change voteMap function

    Former-commit-id: 16049490519df43a79853562fe245497ed0560ad

commit 1799b606015d9cd25d2763b6f13f9fc68f90351c
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Wed Sep 22 11:19:39 2021 +0800

    EDM (#64)

    Former-commit-id: ecaa42e7e1e497e682861295d06cc5239065203a

commit 0bf242a0b15c69197d92e09a57af4c965d48788e
Author: Gabriel WU <45706792+GabrielWuNR@users.noreply.github.com>
Date:   Tue Sep 21 19:54:46 2021 +0800

    finish DBStream (#66)

    Former-commit-id: 3d882f843e0e0fea4d9a4f22bd01d4bcfcf48acb

commit 21f6705102e5d94be3b2547bca4bd4b02a50a24b
Author: Gabriel WU <45706792+GabrielWuNR@users.noreply.github.com>
Date:   Thu Sep 2 21:43:31 2021 +0800

    Fix Time interval bugs in DenStream and CluStream (#62)

    Former-commit-id: 3da316b293d2cdc539173d59114adf96c9b44120

commit f7a01b2e970372783318418bbe32d870c5791514
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Thu Sep 2 21:09:42 2021 +0800

    Add time measurement. Fix a bug in engine (critical). (#59)

    * #56
    add time measurement

    fix a bug in engine.

    observe a bug in denstream

    * #56
    add time measurement

    fix a bug in engine.

    observe a bug in denstream

    update tests accordingly.

    Former-commit-id: 124966d6c59fe0ac4d14869c5bca953b7b9c24da

commit 3004e1ba449e461c943f4d834ef53535466cdec2
Merge: 1d9a355 e95a680
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Wed Sep 1 17:21:31 2021 +0800

    Merge pull request #57 from intellistream/CodeReformat#56

    Former-commit-id: 9d09f037aa36803c7a05553f96653018c76fab86

commit e95a680c969f73b87f9a5e70fc915e0616992d77
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Wed Sep 1 17:16:44 2021 +0800
    code reformat

    Former-commit-id: e776a1829434caf3a299264018bad781ac1fbdb3

commit 1d9a355971f42cbeb523c1b5d3cbf4bf848091e0
Author: Gabriel WU <45706792+GabrielWuNR@users.noreply.github.com>
Date:   Thu Aug 26 20:37:38 2021 +0800

    Finish debug Denstream (#53)

    * fix bugs in OfflineClustering.hpp

    * still bugs in Merge part

    * Finish debug

    Former-commit-id: b7ecbb9573f240516960e19484470d6944f34b59

commit bc9c43e0f56e6dbcc4bb156a83063e873b6e843c
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Wed Aug 25 16:59:00 2021 +0800

    refactor param passing#49 (#50)

    Former-commit-id: bd18e5d8ef9153dfb3e54c74d09a7ca7c0828f4a

commit 7be4f80b5d77ed212abf12468071b96163c61697
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Wed Aug 25 16:24:36 2021 +0800

    Birch wx (#48)

    * modify data sink part in StreamKM

    * fix bug for birch, complete task #10, modify store functions

    Former-commit-id: 511978befd88d548e341e86d8c03a49feea8d7b4

commit 318512175646ca239f918fdc9b0bc15f13715446
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Sun Aug 22 00:38:32 2021 +0800

    Refactor Barriers (#45)

    Clean a race condition bug in Sink.

    Former-commit-id: b9b253c1e6803300eedd953df02c38d5f315249f

commit 79ae5302adca8d023df576b2bb0ff6765e81fb97
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Sat Aug 21 23:43:16 2021 +0800

    Revert "Mac version" (#44)

    Former-commit-id: b3d1d2033870d6561ebe7ecfc7cc02ce9edaf087

commit 5ce9b8d403658da5cf4b614a9516124fb3beffa7
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Sat Aug 21 22:42:51 2021 +0800

    Mac version (#43)

    Former-commit-id: 5eebeb3d0cd07789e9063d2f3cf1296f49c6adf5

commit 4e6fde22725b6f3ce0a8230a329cc09971e33b51
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Thu Aug 19 14:42:21 2021 +0800

    Fix a strange gtest compilation error.

    Add copyright.

    Cleanup unused codes.

    Former-commit-id: 08b4b63782ea4b11309dcb9b4247d3c0b66abf64

commit b94d0cacb770d103a09f6ab1c371f5b1e1f7ca3a
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Thu Aug 19 14:42:21 2021 +0800

    Fix a strange gtest compilation error.

    Former-commit-id: d4a8b76f3ebe62a6ea98f04ff09743302246b135

commit 8be45367f7e13e7e8547bd3955b9b844aaa14f83
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Wed Aug 18 20:15:35 2021 +0800

    Engine Refactor (#34)

    Close issue: #30

    Co-authored-by: s1926539 <zhenyu.wun@gmail.com>
    Former-commit-id: 03625d2c9707cc63bf46020aa0bf4500858fb6e1

commit c326162f7589f521fb2e719db763f84ceec88910
Author: Gabriel WU <45706792+GabrielWuNR@users.noreply.github.com>
Date:   Tue Aug 17 20:31:50 2021 +0800

    CluStream-wzy (#33)

    * Clustream debug finish -- need to pull request

    * delete some TODOs

    Former-commit-id: 24929910e4c60c051b1691df14e15a8b08f93351

commit 11a0759510966fa0844d32984a76fc916f00ce30
Author: shuhaozhangtony <shuhao_zhang@sutd.edu.sg>
Date:   Mon Aug 9 20:08:59 2021 +0800

    fix cmake issues.

    Former-commit-id: eed168286b16057f8f3a1de7d380cd21db0cfaff

commit 4a0e67d0c48557202ba89759d0ed3e50f6c808fc
Merge: c02390c 4ef59fc
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Mon Aug 9 19:39:41 2021 +0800

    Merge pull request #22 from intellistream/addGTests

    add Gtests.

    Former-commit-id: ddab91d11ceb6bd2c45a0e591b351be2074495f5

commit 4ef59fc6f97c6cc05a352171712310e02a0887b4
Author: shuhaozhangtony <shuhao_zhang@sutd.edu.sg>
Date:   Mon Aug 9 19:38:05 2021 +0800

    fix cmake issues.

    Former-commit-id: 34bcda3a5d1195c037120b30e09adea3e2aad701

commit 64bb6dd2dde7c337f3c6263afcec1c7301768578
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Mon Aug 9 19:12:47 2021 +0800

    Update cmake.yml

    Former-commit-id: 1f4a2038b82827c5f21f0fff87205c6143e46e0d

commit 1f45c6bf60b5b9ec19615cdf369b9d2bcf473113
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Mon Aug 9 19:11:02 2021 +0800

    Update cmake.yml

    Former-commit-id: 337d231eac9c56291e6987278fac537a7036e79b

commit 981be14b5c94e0c65b6217d77877ac4294cf4367
Author: shuhaozhangtony <shuhao_zhang@sutd.edu.sg>
Date:   Mon Aug 9 19:10:50 2021 +0800

    fix data path issues.
    fix default parameters.

    Former-commit-id: 9e7cb84e7f9c096749e0a988885441d3f3571ba9

commit 9b0dd1e371b64c63b76863b2f4c739a90a1366bc
Author: shuhaozhangtony <shuhao_zhang@sutd.edu.sg>
Date:   Mon Aug 9 18:44:56 2021 +0800

    fix data path issues.
    fix default parameters.

    Former-commit-id: c48e255a17fba24c89d0accbe462ff408a7da204

commit 60edb19b173e9ef008d38de3262a984280bea699
Author: shuhaozhangtony <shuhao_zhang@sutd.edu.sg>
Date:   Sun Aug 8 01:39:19 2021 +0800

    add Gtests.

    Former-commit-id: a50d41b613b208fb05439d8c8f5292b52f53a572

commit c02390cc971459fc8dd7cf01856de72e099661bd
Merge: 5e1b101 a359727
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Sun Aug 8 00:42:16 2021 +0800

    Merge pull request #21 from intellistream/revert-20-wx

    change a function name

    Former-commit-id: 26860c799b11b2002dcbed80fea890b308074986

commit a359727706f1ad321247ee61edf9482fb00e8715
Author: tuidan <1009575361@qq.com>
Date:   Sun Aug 8 00:38:54 2021 +0800

    change the function name (buildTimeWindow -> runOnlineClustering)

    Former-commit-id: 05c36b9edcc6ce626ceb504e5a21ebe26af7da49

commit e426789a824e83bc774a049717a6c95b5384f270
Author: tuidan <1009575361@qq.com>
Date:   Sat Aug 7 22:50:41 2021 +0800

    reply to reviews of ShuhaoZhangTony #20

    Former-commit-id: 9647419d06077265a813a781cd7daacf5758b0ee

commit 2ec50c06522f66c4f723f04e06dc722d1d89c7f3
Author: tuidan <1009575361@qq.com>
Date:   Sat Aug 7 15:27:41 2021 +0800

    Add Algorithm_parameters #16

    Former-commit-id: 629f78e6bea15fe23799994d9a9c00a84bed9fc0

commit 38d96c4fd3c77c44ec1bd7479f0b6f17d9e30cd5
Author: tuidan <1009575361@qq.com>
Date:   Fri Aug 6 16:34:12 2021 +0800

    fix bug01 done

    Former-commit-id: 00d0b7073938ee3fc8c478cb5aaac60836aaeafd

commit 041643542f7914516f47d40028bd91e376e843d4
Author: tuidan <1009575361@qq.com>
Date:   Fri Aug 6 04:19:55 2021 +0800

    fix bug

    Former-commit-id: 352ec18d890f85b148127e6d2f2b7a356753d049

commit 61dd2e430570c2bdbc0270206fd545977d1181ae
Author: Tony <shuhao_zhang@sutd.edu.sg>
Date:   Sat Aug 7 23:33:21 2021 +0800

    Revert "Recent changes of the framework structure"

    Former-commit-id: 55a66c909b9d31e6e3bed1a4106333b7c88d3894

commit 5e1b1011bb2514c07d8d4a7e28b93e003ff2b3f5
Merge: 245e075 832d9ce
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Sat Aug 7 23:24:29 2021 +0800

    Merge pull request #20 from intellistream/wx

    Recent changes of the framework structure

    Former-commit-id: 4885eb2b7dd145925c90ebd9d172e3b54b0bba02

commit 832d9cea9b6753d22dfb904057cd48e2413d2c8f
Author: tuidan <1009575361@qq.com>
Date:   Sat Aug 7 22:50:41 2021 +0800

    reply to reviews of ShuhaoZhangTony #20

    Former-commit-id: 3dbd69bf2bbdb231505055e32c978be79cefcfa4

commit 1ecb1435974c9106ab6729b299b5e7e6d71f7668
Author: tuidan <1009575361@qq.com>
Date:   Sat Aug 7 15:27:41 2021 +0800

    Add Algorithm_parameters #16

    Former-commit-id: 8350a99a7e848e99bc533c2d87da879e05933076

commit 245e0758e36be3b919b64d039b3a7207c05b1ade
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Sat Aug 7 11:13:52 2021 +0800

    Update README.md

    Former-commit-id: 0b93b82a4eeaffbfcd8f3343bbf2929e06722308

commit 366f2f567ab20b6cb11a697ecaebfc6d87611957
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Sat Aug 7 11:10:51 2021 +0800

    Update cmake.yml

    Former-commit-id: 1e5a62f71b2b92b1c942aa9c11b2c1b90366017b

commit 7261aed22d16383475c442567123cdaffbb6b470
Author: Tony <tonyzhang19900609@gmail.com>
Date:   Sat Aug 7 11:07:38 2021 +0800

    Create cmake.yml

    add workflows.

    Former-commit-id: 6470d10fcefe0db5b6de7d6d0984c78f0401db45

commit 2c9e0622d4c67aafe1f594128718bf027abc6103
Author: tuidan <1009575361@qq.com>
Date:   Fri Aug 6 16:34:12 2021 +0800

    fix bug01 done

    Former-commit-id: bbe58443e8036e0980c96e242cf05baefaa32df1

commit 74a0daa357f7ae80cac6d2276ae0fd59e137023b
Author: tuidan <1009575361@qq.com>
Date:   Fri Aug 6 04:19:55 2021 +0800

    fix bug

    Former-commit-id: a06205fa1a3669c4da084db9be5a0e4116cebf5c

commit 842688fa85a39251be39ef729f276aa9318dbe69
Author: tuidan <1009575361@qq.com>
Date:   Sat Jul 31 17:48:45 2021 +0800

    last commit version of master

    Former-commit-id: ea843a0141835956ac294bc4ae50f4d63cadff1c

commit fa969d907a18e32aabcdf1c87dd1376748bf0546
Author: tuidan <1009575361@qq.com>
Date:   Wed Jul 28 23:03:40 2021 +0800

    add some descriptions and comments

    Former-commit-id: 643f0df01644d652b3cc6163c75785226b7f6c95

commit ea6e2202b8f9423ed4ada95ec5bb91cbcc44430f
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Tue Jul 27 17:08:29 2021 +0800

    Refactor Done.

    Former-commit-id: b684b375445b4b332d7fb65260c6a37fb58570a6

commit f05c6a93f93eefcc092e9fb8cbc348e27b2915c2
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Mon Jul 26 23:09:01 2021 +0800

    Half-way done.. Still a lot to do.

    Former-commit-id: a6418d7c0f07b0b61682f4350ef0f0c9b522eec6

commit ce6ea9282dd479e08fbfe5e40319f6a9d49ee107
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Mon Jul 26 17:47:11 2021 +0800

    remove tmp

    Former-commit-id: 32cb727bb2704fbb33d9f03f7f7cdd738a9993dd

commit cae0490a0b8ae05387477c0d296b8037c9c58d81
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Mon Jul 26 17:45:27 2021 +0800

    Add SSE

    Former-commit-id: 63eed8767d91334cac94bd843b2bdc2eae651d1e

commit 569359860d43fcc0035adbc39ea616504b36de43
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Mon Jul 26 16:45:07 2021 +0800

    re-package

    Former-commit-id: bb14fb329db9e6e6af3a6f4a6fd1d9fa90fa9a03

commit b2945f9b271104c26efc3e65d2f54aa9462d7a6c
Author: tuidan <1009575361@qq.com>
Date:   Fri Jul 23 18:01:49 2021 +0800

    change structure of the run() function

    Former-commit-id: 6a7c0055cf43b34b3614380276f198f919d91289

commit b7fa553a26aaf67fc04ad2cdfd768083bed498a8
Author: tuidan <1009575361@qq.com>
Date:   Fri Jul 23 17:48:15 2021 +0800

    change structure of the run() function

    Former-commit-id: 84cd4b9e76272d29b7cacd4274178c6b48d6d2d1

commit 58fac566a90e4163ca7ed99aa82b66c316fdc62c
Author: tuidan <1009575361@qq.com>
Date:   Thu Jul 22 02:43:12 2021 +0800

    fix bug and it could run
    -p 15120 -c 10 -d 54 -s 100 -S 10

    Former-commit-id: 3baf836948742aafad6cb17c493494376794f897

commit 34811137194816df9891b52b5e8907f5208780d5
Author: tuidan <1009575361@qq.com>
Date:   Wed Jul 21 02:14:53 2021 +0800

    change print formate and include class into SESAME namespace

    Former-commit-id: a39f9d26ebadabd22910851a1b0aa4b0d7a25fbd

commit a2977ca7b78e58c0bddf11d93f0bddd80b223209
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Tue Jul 20 18:32:05 2021 +0800

    able to run but..

    Former-commit-id: 3a6da437ee2076c09c169f9a906f11c408b1d8e1

commit b8cebd7d6ab860f7cb505c6b2d5e3edc44f5d763
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Tue Jul 20 17:00:46 2021 +0800

    able to run but..

    Former-commit-id: 142fa4cde59aeb6f60adf4822b320c8be621227d

commit 83a7d251d60d7e5f52dbec12b181b8d5d40a100c
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Tue Jul 20 16:39:09 2021 +0800

    able to run but..

    Former-commit-id: e4b04927bec450d5fb1552008ff5a08d43f08904

commit 1d60faaad041118157f367caf3c66000afa96544
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Tue Jul 20 16:14:38 2021 +0800

    able to run but..

    Former-commit-id: 4a1e45e24ff2bad5aa107a12a5894aa1c35c2d4e

commit 803f0b108e04b6c31843160348ead96d1eba2d97
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Mon Jul 19 19:18:20 2021 +0800

    add Point

    Former-commit-id: e3f6582ebf6056ab105807f7c3b4af8409252251

commit fe0e549ece6e382af4d778b3d977b9385a13e3b9
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Mon Jul 19 18:57:45 2021 +0800

    Major Structure Update.
    Able to run.

    Former-commit-id: 771248920bcb46a444d8268356f1508fefdf2aa0

commit c660b056ac94980be731e2267588f106fce0b0a4
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Mon Jul 19 18:48:25 2021 +0800

    Major Structure Update.
    Able to run.

    Former-commit-id: ab9b76582f5a90553e0db3a449fce51c3a50dfe6

commit 8171bca53a48abe461e234440bbc607f67d3e0d2
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Mon Jul 19 18:26:08 2021 +0800

    Major Structure Update.
    Able to run.

    Former-commit-id: 577d9a5761a1ee388b2aad2220f6e32b64707e8d

commit a89819325e6cc6027615cf5ca896025bcfc6facf
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Mon Jul 19 16:27:02 2021 +0800

    Major Structure Update.

    Former-commit-id: 0c02951266b8644db4f2bf238df6b2f59571b3ce

commit 3f5c899d6a32aa40bf3cb41dd276255704b3f562
Author: tuidan <1009575361@qq.com>
Date:   Mon Jul 12 15:44:43 2021 +0800

    modify StreamKM++(completely without debug)

    Former-commit-id: d63b524972e0285032e11eee031f6b0d70ec64eb

commit af47e30bb476b296e792471e2c5003efef2b954b
Merge: 227f66a 71098e4
Author: tuidan <1009575361@qq.com>
Date:   Mon Jul 12 15:42:00 2021 +0800

    Merge branch 'wx'

    Former-commit-id: b60405ee36ff48717f311f5815fe3df52b59881a

commit 71098e4f3d82db03f13c2a24dfa989509b2add2a
Author: tuidan <1009575361@qq.com>
Date:   Mon Jul 12 15:36:26 2021 +0800

    modify StreamKM++(completely without debug)

    Former-commit-id: 490809cd37a19062abb8072617438c66965508ba

commit 78557cc00a412b245c62dd5626358101cbcfd12e
Author: tuidan <1009575361@qq.com>
Date:   Sat Jul 10 21:25:47 2021 +0800

    modify StreamKM++(completely without debug)

    Former-commit-id: 206cf4431c37ff44bfd5a8d29f318d91b59d9276

commit 17a40ef128513c2df432cc2fe4a8bb34f5174713
Author: tuidan <1009575361@qq.com>
Date:   Sat Jul 10 12:18:36 2021 +0800

    modify StreamKM++(partially)

    Former-commit-id: 6ec189fc7764df305d5aeca9ea0859e80ef5dfa8

commit ec92f451d754504fd08cfe4e4c27d8d5396d72d5
Author: tuidan <1009575361@qq.com>
Date:   Sat Jul 10 03:12:54 2021 +0800

    modify StreamKM++(partially)

    Former-commit-id: fa511284e8e1de51ea64817571be719209de97ae

commit 868dfe6fc00768a3665bc84423c4a1474c740a31
Author: tuidan <1009575361@qq.com>
Date:   Thu Jul 8 23:13:54 2021 +0800

    modify benchmark

    Former-commit-id: 805296be489774e9fecad5ac5c5fd9bec53c77e1

commit a32d10d75a59a80a5a1b0716a0ebd18cea71caa8
Author: tuidan <1009575361@qq.com>
Date:   Tue Jul 6 15:26:57 2021 +0800

    add ifndef in StreamKM++ and add Kmeans functions

    Former-commit-id: 859bbfa20eed724e1f4ae9f9ffb374573aba0c74

commit fb6dd77f974516005800f70e177270dcf759a4b8
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Sun Jul 4 14:02:31 2021 +0800

    Update README.md

    Former-commit-id: 067fc44f4d096741890c5e509437a98a3c7d2697

commit 3fb249351f970a2d3ba91fdbfa5546284f9ac637
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Sun Jul 4 13:57:58 2021 +0800

    Update README.md

    Former-commit-id: 3120eaa4c5bb8198dc24cd06f8f6ee365853d21e

commit f67fba7ecbe98bc58657dc61469e5a8107f7a6ab
Author: tuidan <40883104+tuidan@users.noreply.github.com>
Date:   Sun Jul 4 13:55:43 2021 +0800

    Update README.md

    Former-commit-id: 82f0165fb7b210d4b3c59036a1122db01cb72469

commit 1bcf1af96d797d40e71e24c1412d7b26bb0f850b
Author: tuidan <1009575361@qq.com>
Date:   Sun Jul 4 13:51:04 2021 +0800

    update README.md

    Former-commit-id: b435e8dec347858e93762ad3c5a3ede901783081

commit 07db87ca91bc8f5b341683bad8d4a6daff887afb
Author: tuidan <1009575361@qq.com>
Date:   Sun Jul 4 13:34:42 2021 +0800

    update README.md

    Former-commit-id: 3c4aad88426e8e9e9a5c04029dd5e1c75d6352ed

commit 0307e4c7a9046cffe7ab56d5eb4a656d3ba648d1
Author: tuidan <1009575361@qq.com>
Date:   Sun Jul 4 13:32:22 2021 +0800

    update README.md

    Former-commit-id: 981734a3135da045b82a838901420866cce76717

commit 5100767d330a451bade01bcf43bb50cbcba13d45
Author: tuidan <1009575361@qq.com>
Date:   Sat Jul 3 22:20:35 2021 +0800

    update README.md

    Former-commit-id: f068a02f140a13ffff3ced5b1d46595d84562fca

commit b67e69664889a984e78020219e69605fc06bdb78
Author: tuidan <1009575361@qq.com>
Date:   Sat Jul 3 21:24:47 2021 +0800

    update the StreamKM++ version1.0

    Former-commit-id: 0b4bcd498749ad678e90efeb5523da36a74d2ec3

commit c4b00bc9d6df5ae44cf109501b8c6e223778a19e
Author: tuidan <1009575361@qq.com>
Date:   Sat Jul 3 21:21:18 2021 +0800

    update the StreamKM++ version1.0

    Former-commit-id: 799b12f56dc1dab4e307cd4d189c453b8ed80af1

commit 367ea1af32f2d922785a7a2727dad8a42ef60be9
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Thu Jul 1 20:08:08 2021 +0800

    src structure update.

    Former-commit-id: 4dc971faeb1cbdfbaad75161fb7efe8ae2a22264

commit 227f66a7bf12bf28b9ad8cb25cbdf8f0ef2770d3
Author: Shuhao Zhang <tonyzhang19900609@gmail.com>
Date:   Thu Jul 1 20:08:08 2021 +0800

    src structure update.

    Former-commit-id: 2045da33914874d28ebe45455b59d685937204aa

commit b5aee7705a4b2657de51c407c86206103f040810
Author: tuidan <1009575361@qq.com>
Date:   Thu Jul 1 09:22:25 2021 +0800

    code clean version2.0

    Former-commit-id: da2b9f609a5858e8567ba9da5ca818ae50b13c90

commit 84658f6be039fa36e557ca1fbd8e0c50b50da026
Author: tuidan <1009575361@qq.com>
Date:   Thu Jul 1 09:21:24 2021 +0800

    code clean version2.0

    Former-commit-id: 52710406898c35f0749c69d2acbce484c9cb47e4

commit d4e611b93b411d0775b747cecee49867f515b038
Author: tuidan <1009575361@qq.com>
Date:   Thu Jul 1 09:17:31 2021 +0800

    code clean version2.0

    Former-commit-id: 855c52ed2480de2b02bf22f35f21a373d13ef7e9

commit 0f5f9085fc9f76fb50dc768beee297816c7ab8f1
Author: tuidan <1009575361@qq.com>
Date:   Thu Jul 1 08:23:16 2021 +0800

    code clean version2.0

    Former-commit-id: 1c7b5c11dd6686e90cc91c2d20c9372a7301e455

commit 2b869e63b4b39e6739f356f02a0aae631610760d
Merge: 0ebebe6 56792c5
Author: tuidan <1009575361@qq.com>
Date:   Thu Jul 1 08:08:01 2021 +0800

    code clean version2.0

    Former-commit-id: 6d2b9dd2e33711e77e5bb2da94e383965c31047e

commit 0ebebe609d62f847ad1d4d7cc25d7289b4a4605f
Author: tuidan <1009575361@qq.com>
Date:   Thu Jul 1 08:05:13 2021 +0800

    code clean version2.0

    Former-commit-id: b58f3ced9a420059f1ba3f56e9dfdd241c861933

commit 9b0f998f6cf477e52c1c053f05c79e792e52cade
Author: tuidan <199906300553wx>
Date:   Thu Jul 1 00:03:54 2021 +0800

    code clean version2.0

    Former-commit-id: 5abdd5375683fba055f78d10e6d92931d74f2265

commit 56792c5332e2f31b3eeac56ab43a7c68f39aa670
Author: 1009575361@qq.com <199906300553wx>
Date:   Thu Jul 1 00:03:54 2021 +0800

    code clean version2.0

    Former-commit-id: 5cb317c8b96543a9eae0fee7399398d6562101bf

commit 37830cb86adc03fc5f2682f13829bac30077309c
Author: 1009575361@qq.com <199906300553wx>
Date:   Wed Jun 30 23:59:34 2021 +0800

    code clean version2.0

    Former-commit-id: b49653f71ee85981a7a1f03f764aa88a6e702edf

commit f43658b25e1519c581021613251431d498ff21dc
Author: 1009575361@qq.com <199906300553wx>
Date:   Wed Jun 30 23:53:24 2021 +0800

    code clean version2.0

    Former-commit-id: dfb898ef2e8cff96b186d355a8ac8342857e9e73

commit 773b38c502aa5f4583b6c19a090ea7b726b87f8b
Author: 1009575361@qq.com <199906300553wx>
Date:   Wed Jun 30 23:48:27 2021 +0800

    code clean version2.0

    Former-commit-id: a457e3831ed9673771d52528ae2de24dd8f6fc80

commit 3e071b9c01df34a61d5555ac827222ecde088abc
Author: s1926539 <45706792+s1926539@users.noreply.github.com>
Date:   Wed Jun 30 12:39:22 2021 +0800

    Upload current coding Clustream

    Finish online part...Still verfiying

    Former-commit-id: b2d11d7f791c8098f8a74535e131d05afb0f9059

commit 05c7b52c3d3d6ca3145fcfa1ebd1f1bed7295780
Author: s1926539 <45706792+s1926539@users.noreply.github.com>
Date:   Fri Jun 25 15:54:13 2021 +0800

    Upload kdd 99

    Upload kdd 99 subset for algorithms training

    Former-commit-id: cc352523dab135053ea0bee013b5dc99b61c146b
  • Loading branch information
@wzru
wzru committed Sep 9, 2023
1 parent 93800bc commit 50c92b6
Show file tree
Hide file tree
Showing 162 changed files with 7,914 additions and 994 deletions.
9 changes: 8 additions & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,5 +26,12 @@
/cmake-*
/debug
.vscode/

/scripts/benne/jpg/
/scripts/benne/pdf/
*.log

datasets/

*.egg-info/
*.out
results.txt
7 changes: 6 additions & 1 deletion CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
cmake_minimum_required(VERSION 3.14)
project(SESAME CXX)

if (NOT CMAKE_BUILD_TYPE OR CMAKE_BUILD_TYPE STREQUAL "")
set(CMAKE_BUILD_TYPE "Release" CACHE STRING "" FORCE)
endif()

# Custom CMake find instructions and macros
set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake;${CMAKE_MODULE_PATH}")
include(cmake/macros.cmake)
Expand All @@ -19,7 +23,7 @@ message(STATUS "sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc
message(STATUS "sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 11")

# Set Optimization Flags
set(CMAKE_CXX_FLAGS "-g -std=c++20 -Wall -Werror=return-type -fconcepts-diagnostics-depth=2 -lpthread -fopenmp -static -march=native")
set(CMAKE_CXX_FLAGS "-g -std=c++20 -Wall -Werror=return-type -fconcepts-diagnostics-depth=2 -lpthread -fopenmp -march=native")
set(CMAKE_CXX_FLAGS_DEBUG "-O0 -DNO_RACE_CHECK -DSESAME_DEBUG_MODE=1 -DDEBUG")
set(CMAKE_CXX_FLAGS_RELEASE "-Wno-ignored-qualifiers -Wno-sign-compare -O3 -DNDEBUG")

Expand Down Expand Up @@ -58,6 +62,7 @@ if (ENABLE_UNIT_TESTS)
endif ()

find_package(OpenMP)
find_package(gflags REQUIRED)

# Print all used include directories
message(STATUS "INCLUDE_DIRS:")
Expand Down
2 changes: 1 addition & 1 deletion benchmark/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ if (BUILD_BENCHMARKS)
target_link_libraries(benchmark sesame gflags)
if (NOT EXISTS ${CMAKE_CURRENT_BINARY_DIR}/datasets)
add_custom_target(bench_datasets ALL
COMMAND ${CMAKE_COMMAND} -E create_symlink ${CMAKE_CURRENT_SOURCE_DIR}/datasets ${CMAKE_CURRENT_BINARY_DIR}/datasets)
COMMAND ${CMAKE_COMMAND} -E create_symlink ${CMAKE_CURRENT_SOURCE_DIR}/../datasets ${CMAKE_CURRENT_BINARY_DIR}/datasets)
add_dependencies(benchmark bench_datasets)
endif (NOT EXISTS ${CMAKE_CURRENT_BINARY_DIR}/datasets)
message("---Done building benchmarks")
Expand Down
141 changes: 65 additions & 76 deletions benchmark/src/Benchmark.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@

#include <filesystem>

using namespace std::filesystem;
using namespace std;
using namespace std::filesystem;
using namespace SESAME;

DEFINE_int32(algo, 0, "Algorithm to use");
Expand Down Expand Up @@ -58,20 +58,27 @@ DEFINE_int32(time_window, 50, "Time window");
DEFINE_int32(num_online_clusters, 80, "Number of online clusters");
// SL-KMeans
DEFINE_double(delta_grid, 0.2, "The delta parameter used int the grid for guessing the optimum.");
DEFINE_int32(num_samples, 100, "Number of samples");
DEFINE_int32(num_samples, 10, "Number of samples");
// Generic
DEFINE_int32(landmark, 1000, "Landmark");
DEFINE_int32(sliding, 1000, "Sliding");
DEFINE_int32(landmark, 10000, "Landmark");
DEFINE_int32(sliding, 500, "Sliding");
DEFINE_double(outlier_distance_threshold, 1000, "Outlier distance threshold");
DEFINE_int32(outlier_cap, 100, "Outlier cap");
DEFINE_double(outlier_density_threshold, 100, "Outlier density threshold");
DEFINE_double(neighbor_distance, 200, "Neighbor distance");
DEFINE_int32(k, 0, "KMeans K");
DEFINE_int32(k, 7, "KMeans K");
DEFINE_int32(arr_rate, 0, "Arrival rate");
DEFINE_bool(run_offline, true, "Whether run offline clustering");
DEFINE_bool(run_eval, true, "Whether run evaluation");
DEFINE_bool(run_cmm, true, "Whether run CMM evaluation");
DEFINE_bool(run_cmm, false, "Whether run CMM evaluation");
DEFINE_bool(run_pur, true, "Whether run Purity evaluation");
// Benne
DEFINE_int32(obj, 0, "Objective: 0(balance), 1(accuracy), 2(efficiency)");
DEFINE_int32(queue_size_threshold, 10000, "Benne queue size threshold");
DEFINE_int32(dim_threshold, 30, "Benne dimension threshold");
DEFINE_double(variance_threshold, 100.0, "Benne variance threshold");
DEFINE_int32(outliers_num_threshold, 200, "Benne outliers threshold");
DEFINE_double(outliers_dist_threshold, 50.0, "Benne outliers distance threshold");

int main(int argc, char **argv)
{
Expand All @@ -83,81 +90,63 @@ int main(int argc, char **argv)
<< "# #\n"
<< "############################################################"
"\033[0m\n";
sleep(2);
sleep(1);
#endif
// Parse parameters.
param_t param;
gflags::ParseCommandLineFlags(&argc, &argv, true);
param_t param;
param.algo = (AlgoType)FLAGS_algo;
BenchmarkUtils::defaultParam(param);
// gen from gen_flags.js
param.algo = (AlgoType)FLAGS_algo;
param.input_file = FLAGS_input_file;
param.num_points = FLAGS_num_points;
param.dim = FLAGS_dim;
param.num_clusters = FLAGS_num_clusters;
param.max_in_nodes = FLAGS_max_in_nodes;
param.max_leaf_nodes = FLAGS_max_leaf_nodes;
param.distance_threshold = FLAGS_distance_threshold;
param.seed = FLAGS_seed;
param.coreset_size = FLAGS_coreset_size;
param.radius = FLAGS_radius;
param.delta = FLAGS_delta;
param.beta = FLAGS_beta;
param.buf_size = FLAGS_buf_size;
param.alpha = FLAGS_alpha;
param.lambda = FLAGS_lambda;
param.clean_interval = FLAGS_clean_interval;
param.min_weight = FLAGS_min_weight;
param.base = FLAGS_base;
param.cm = FLAGS_cm;
param.cl = FLAGS_cl;
param.grid_width = FLAGS_grid_width;
param.min_points = FLAGS_min_points;
param.epsilon = FLAGS_epsilon;
param.mu = FLAGS_mu;
param.num_last_arr = FLAGS_num_last_arr;
param.time_window = FLAGS_time_window;
param.num_online_clusters = FLAGS_num_online_clusters;
param.delta_grid = FLAGS_delta_grid;
param.num_samples = FLAGS_num_samples;
param.landmark = FLAGS_landmark;
param.sliding = FLAGS_sliding;
param.outlier_distance_threshold = FLAGS_outlier_distance_threshold;
param.outlier_cap = FLAGS_outlier_cap;
param.outlier_density_threshold = FLAGS_outlier_density_threshold;
param.neighbor_distance = FLAGS_neighbor_distance;
param.k = FLAGS_k;
param.arr_rate = FLAGS_arr_rate;
param.run_offline = FLAGS_run_offline;
param.run_eval = FLAGS_run_eval;
param.run_cmm = FLAGS_run_cmm;
param.run_pur = FLAGS_run_pur;
param.algo = (AlgoType)FLAGS_algo;
param.input_file = FLAGS_input_file;
param.num_points = FLAGS_num_points;
param.dim = FLAGS_dim;
param.num_clusters = FLAGS_num_clusters;
param.max_in_nodes = FLAGS_max_in_nodes;
param.max_leaf_nodes = FLAGS_max_leaf_nodes;
param.distance_threshold = FLAGS_distance_threshold;
param.seed = FLAGS_seed;
param.coreset_size = FLAGS_coreset_size;
param.radius = FLAGS_radius;
param.delta = FLAGS_delta;
param.beta = FLAGS_beta;
param.buf_size = FLAGS_buf_size;
param.alpha = FLAGS_alpha;
param.lambda = FLAGS_lambda;
param.clean_interval = FLAGS_clean_interval;
param.min_weight = FLAGS_min_weight;
param.base = FLAGS_base;
param.cm = FLAGS_cm;
param.cl = FLAGS_cl;
param.grid_width = FLAGS_grid_width;
param.min_points = FLAGS_min_points;
param.epsilon = FLAGS_epsilon;
param.mu = FLAGS_mu;
param.num_last_arr = FLAGS_num_last_arr;
param.time_window = FLAGS_time_window;
param.num_online_clusters = FLAGS_num_online_clusters;
param.delta_grid = FLAGS_delta_grid;
param.num_samples = FLAGS_num_samples;
param.landmark = FLAGS_landmark;
param.sliding = FLAGS_sliding;
param.outlier_distance_threshold = FLAGS_outlier_distance_threshold;
param.outlier_cap = FLAGS_outlier_cap;
param.outlier_density_threshold = FLAGS_outlier_density_threshold;
param.neighbor_distance = FLAGS_neighbor_distance;
param.k = FLAGS_k;
param.arr_rate = FLAGS_arr_rate;
param.run_offline = FLAGS_run_offline;
param.run_eval = FLAGS_run_eval;
param.run_cmm = FLAGS_run_cmm;
param.run_pur = FLAGS_run_pur;
param.obj = (BenneObj)FLAGS_obj;
param.benne_threshold.dim = FLAGS_dim_threshold;
param.benne_threshold.queue_size = FLAGS_queue_size_threshold;
param.benne_threshold.variance = FLAGS_variance_threshold;
param.benne_threshold.outliers_num = FLAGS_outliers_num_threshold;
param.benne_threshold.outliers_dist = FLAGS_outliers_dist_threshold;

param.output_file = "results.txt";
param.fast_source = true;
param.store = false;

std::vector<SESAME::PointPtr> input;
std::vector<SESAME::PointPtr> results;

// Create Spout.
SESAME::DataSourcePtr sourcePtr = GenericFactory::New<DataSource>(param);

// Directly load data from file. TODO: configure it to load from external
// sensors, e.g., HTTP.
BenchmarkUtils::loadData(param, sourcePtr);

// Create Sink.
SESAME::DataSinkPtr sinkPtr = GenericFactory::New<DataSink>(param);

// Create Algorithm.
SESAME::AlgorithmPtr algoPtr = SESAME::AlgorithmFactory::create(param);

param.Print();

// Run algorithm producing results.
auto res = BenchmarkUtils::runBenchmark(param, sourcePtr, sinkPtr, algoPtr);

res->Print();
}
RunBenchmark(param);
}
35 changes: 19 additions & 16 deletions include/Algorithm/Algorithm.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

#include "Algorithm/DataStructure/Point.hpp"
#include "Algorithm/Param.hpp"
#include "Evaluation/Evaluation.hpp"
#include "Sinks/DataSink.hpp"
#include "Timer/Timer.hpp"

Expand All @@ -35,31 +36,33 @@ class Algorithm
virtual void Init() = 0;
virtual void RunOnline(SESAME::PointPtr input) = 0;
virtual void RunOffline(SESAME::DataSinkPtr ptr) = 0;
void Insert(SESAME::PointPtr input){};
virtual void OutputOnline(std::vector<PointPtr> &centers){};
void Store(std::string output_file, int dim, std::vector<PointPtr> results);
Timer win_timer, ds_timer, out_timer, ref_timer, sum_timer, lat_timer, on_timer;
param_t param;
int cnt = 0;
std::vector<int64> et;
void PrintPerf()
PerfRes GetPerf()
{
cout << "win_us: " << win_timer.sum / 1000 << endl;
cout << "ds_us: " << ds_timer.sum / 1000 << endl;
cout << "out_us: " << out_timer.sum / 1000 << endl;
cout << "ref_us: " << ref_timer.sum / 1000 << endl;
auto sum = win_timer.sum + ds_timer.sum + out_timer.sum + ref_timer.sum;
cout << "sum_us: " << sum / 1000 << endl;
assert(param.num_points);
PerfRes res;
res.win_us = win_timer.sum / 1000;
res.ds_us = ds_timer.sum / 1000;
res.out_us = out_timer.sum / 1000;
res.ref_us = ref_timer.sum / 1000;
res.sum_us = sum_timer.sum / 1000;
if (et.size() == 5)
{
cout << "on_20: " << et[0] / 1e6 << endl;
cout << "on_40: " << et[1] / 1e6 << endl;
cout << "on_60: " << et[2] / 1e6 << endl;
cout << "on_80: " << et[3] / 1e6 << endl;
cout << "on_100: " << et[4] / 1e6 << endl;
res.on_20 = et[0] / 1e6;
res.on_40 = et[1] / 1e6;
res.on_60 = et[2] / 1e6;
res.on_80 = et[3] / 1e6;
res.on_100 = et[4] / 1e6;
}
cout << "lat_us: " << lat_timer.sum / 1e3 / param.num_points << endl;
cout << "et_s: " << on_timer.sum / 1e9 << endl;
cout << "qps: " << param.num_points * 1e9 / sum_timer.sum << endl;
res.lat_us = lat_timer.sum / 1e3 / param.num_points;
res.et_s = on_timer.sum / 1e9;
res.qps = param.num_points * 1e9 / sum_timer.sum;
return res;
}
void Count()
{
Expand Down
96 changes: 96 additions & 0 deletions include/Algorithm/Benne.hpp
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
//
// Created by tuidan on 2021/8/24.
//

#ifndef SESAME_INCLUDE_ALGORITHM_BENNE_HPP_
#define SESAME_INCLUDE_ALGORITHM_BENNE_HPP_

#include "Algorithm/Algorithm.hpp"
#include "Algorithm/DesignAspect/Generic.hpp"
#include "Algorithm/OfflineRefinement/KMeans.hpp"
#include "Algorithm/WindowModel/LandmarkWindow.hpp"
#include "Sinks/DataSink.hpp"
#include "Utils/BenchmarkUtils.hpp"

#include <utility>

namespace SESAME
{
struct characteristics
{
bool frequentDrift = false;
bool manyOutliers = false;
bool highDimension = false;
};

enum windowSelection
{
landmark = 0,
sliding = 1,
damped = 2
};
enum dataSelection
{
MCs = 0,
CFT = 1,
CoreT = 2,
DPT = 3,
Grids = 4,
AMS = 5
};
enum outlierSelection
{
OD = 0,
NoOD = 1,
ODB = 2,
ODT = 3,
ODBT = 4
};
enum refineSelection
{
Incre = 0,
OneShot = 1,
NoRefine = 2
};
class Benne : public Algorithm
{
public:
std::vector<PointPtr> queue_;
std::vector<PointPtr> materialized_centers;
// std::vector<PointPtr> centers;
BenneThreshold T;
bool ds_changed = false;
AlgorithmPtr algo;
BenneObj obj;
characteristics chara;
windowSelection windowSel;
dataSelection dataSel;
outlierSelection outlierSel;
refineSelection refineSel;
KMeans kmeans;
int first_algo;
size_t change_count = 0;
std::vector<std::pair<int, int>> change_log;
Timer mig_timer, det_timer;

using MicroClusters = ClusteringFeaturesList;
static constexpr int INCRE_REF_CNT = 50000;

Benne(param_t &cmd_params);

~Benne();

void Init() override;

void RunOnline(PointPtr input) override;

void RunOffline(DataSinkPtr sinkPtr) override;

private:
void Train(const PointPtr &point);
int Infer(const SESAME::PointPtr &input);
void UpdateAlgo(int, int);
};

} // namespace SESAME
#endif // SESAME_INCLUDE_ALGORITHM_BENNE_HPP_
2 changes: 1 addition & 1 deletion include/Algorithm/Birch.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
namespace SESAME
{

class BirchParameter : public AlgorithmParameters
class BirchParameter : public SesameParam
{
public:
int max_in_nodes; // B
Expand Down
Loading

0 comments on commit 50c92b6

Please sign in to comment.