Skip to content
This repository has been archived by the owner on May 15, 2019. It is now read-only.

Spot naming #73

Open
wants to merge 7 commits into
base: tmp_spot
Choose a base branch
from
Open

Spot naming #73

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e8670dc
spot_naming
NathanSegerlind Sep 27, 2016
070110d
spot_naming
NathanSegerlind Sep 28, 2016
f7d5f24
spot_naming
NathanSegerlind Sep 29, 2016
14bb7a5
Merge branch 'dev' into spot_naming
NathanSegerlind Sep 29, 2016
707bdc2
spot_naming
NathanSegerlind Sep 29, 2016
f26816b
spot_naming
NathanSegerlind Sep 29, 2016
1692d21
spot_naming
NathanSegerlind Sep 29, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "oni-lda-c"]
path = oni-lda-c
url = https://github.com/OpenNetworkInsight/oni-lda-c
2 changes: 1 addition & 1 deletion ml_ops.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ hdfs dfs -rm -R -f ${HDFS_SCORED_CONNECTS}

${MPI_PREP_CMD}

time spark-submit --class "org.opennetworkinsight.SuspiciousConnects" \
time spark-submit --class "org.spot.SuspiciousConnects" \
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we was missing this one?

--master yarn-client \
--driver-memory ${SPK_DRIVER_MEM} \
--conf spark.driver.maxResultSize=${SPK_DRIVER_MAX_RESULTS} \
Expand Down
12 changes: 12 additions & 0 deletions upload.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/bin/bash

USER=$1
CLUSTER=$2
DIRNAME=$3

# any optional arguments you wish to pass directly to rsync go in position 4

# exclude the hidden files, target, lib, project, src... except the assembled jar
rsync $4 -v -a --include='target' --include='target/scala-2.10' --include='target/scala-2.10/*.jar' \
--include='spot-lda-c' --include='spot-lda-c/*' --include='*.py' --include='*.sh' --include 'top-1m.csv' \
--exclude='*' . $USER@$CLUSTER:~/$DIRNAME
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this file be part of the project?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on one hand:

people would almost certainly use it and appreciate it

on the other:

there is a process for adding stuff to the product and we didn't follow it
there's no documentation for it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I'm taking it out