Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to run this project in Docker #9

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
df9cdd5
Added Dockerfile in order to run this project easier and faster, expa…
Jul 31, 2020
8ed6c12
Enhanced description on how to run it and what is required
Jul 31, 2020
f974456
Update market_scanner.py
NihilistPenguin Jul 31, 2020
7574904
Add files via upload
NihilistPenguin Jul 31, 2020
6f5cb6c
Update grapher.py
NihilistPenguin Jul 31, 2020
ae0a979
Add files via upload
NihilistPenguin Jul 31, 2020
2f4fd42
Update README.md
NihilistPenguin Jul 31, 2020
29c29bc
Update README.md
NihilistPenguin Jul 31, 2020
4c1770b
Update README.md
NihilistPenguin Jul 31, 2020
cbf1b4e
Update grapher.py
NihilistPenguin Jul 31, 2020
f9e20da
Update grapher.py
NihilistPenguin Jul 31, 2020
eb859ec
Update market_scanner.py
NihilistPenguin Jul 31, 2020
2fd7709
add figures directory with demo images
Jul 31, 2020
1d9ddc8
add figures directory with sample images
Jul 31, 2020
24eca53
Delete .gitignore
NihilistPenguin Jul 31, 2020
e845186
Update README.md
Jul 31, 2020
18b5427
Update README.md
Jul 31, 2020
65c26cf
Update README.md
Jul 31, 2020
9fb4250
Merge pull request #16 from tcosculluela/master
Jul 31, 2020
21933ae
Revert "Parallelization"
SamPom100 Jul 31, 2020
f2b9e36
Merge pull request #20 from SamPom100/revert-16-master
Jul 31, 2020
4893e10
Merge pull request #18 from NihilistPenguin/master
Jul 31, 2020
bd6508b
Revert "Add various functionality "
SamPom100 Jul 31, 2020
1032b62
Merge pull request #21 from SamPom100/revert-18-master
Jul 31, 2020
df21531
As per PR comments removed duplicate code and made it clearer about r…
Jul 31, 2020
439401a
Merge upstream
Jul 31, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
FROM python:3
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using smaller image, there's no need to have all of python to run this.

python:3-slim-buster is around 48MB compressed.
python:3 is around 320MB compressed.


RUN pip install --upgrade pip && \
pip install --no-cache-dir nibabel pydicom matplotlib pillow && \
pip install --no-cache-dir med2image

# As per https://docs.docker.com/develop/develop-images/dockerfile_best-practices/

COPY requirements.txt /tmp
RUN pip install --requirement /tmp/requirements.txt
COPY . /tmp/

RUN mkdir /opt/wsb/
COPY grapher.py market_scanner.py stocklist.py /opt/wsb/

RUN mkdir /opt/wsb/data
COPY data /opt/wsb/data
44 changes: 35 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,50 @@

This scans every ticker on the market, gets their last 5 months of volume history, and alerts you when a stock's volume exceeds 10 standard deviations from the mean within the last 3 days. (these numbers are all adjustable). Helps find anomalies in the stock market

## [Website Link!](http://165.22.228.6/ ):

## How to run this:
[credit!](https://www.reddit.com/r/wallstreetbets/comments/i10mif/i_made_a_website_for_that_scanner_made_by_that/)

-download your favorite Python IDE. (I use VSCode)

-get my script from GitHub
## Requirements

-open the script in your IDE and install all required dependancies by typing pip install -r requirements.txt into the IDE's terminal. You can get to the the terminal on VSC by pressing CMD and ` at the same time.
Docker - install Docker https://docs.docker.com/get-docker/

-run the market_scanner.py and it will print out results into the terminal
-you can also graph any ticker's volume in grapher.py
Alternatively,

1) download your favorite Python IDE. (For example, the project creator uses VSCode)

2) Clone project from GitHub

3) Open project in your IDE and install all required dependancies by typing pip install -r requirements.txt into the IDE's terminal. You can get to the the terminal on VSC by pressing CMD and ` at the same time.

## How to run
Build docker image by running commands below, from the same directory as Dockerfile.
<pre>
docker build --tag NAME:VERSION . ie docker build --tag wsb:1.0 .
</pre>
You will be presented with a message
<pre>
Successfully built ID
Successfully tagged wsb:1.0
</pre>
Then run
<pre>
docker run -ti ID bash
Once inside
python /opt/wsb/market_scanner.py
</pre>

Alternatively,
1) run the market_scanner.py and it will print out results into the terminal
2) you can also graph any ticker's volume in grapher.py

## Controlling the Script
-Line 17 controls the amount of months of historical volume the script gets
Line 17 controls the amount of months of historical volume the script gets

-Line 75 controls the amount of days before today that it will alert you
Line 75 controls the amount of days before today that it will alert you

-Line 84 controls the number of standard deviations away from the mean volume
Line 84 controls the number of standard deviations away from the mean volume



Expand Down
18 changes: 6 additions & 12 deletions market_scanner.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,6 @@
from stocklist import NasdaqController
from tqdm import tqdm

from joblib import Parallel, delayed
import multiprocessing


class mainObj:
def getData(self, ticker):
Expand Down Expand Up @@ -65,21 +62,18 @@ def days_between(self, d1, d2):
d2 = datetime.datetime.strptime(d2, "%Y-%m-%d")
return abs((d2 - d1).days)

def parallel_wrapper(self,x, cutoff, currentDate):
d = (self.find_anomalies_two(self.getData(x), cutoff))
if d['Dates']:
for i in range(len(d['Dates'])):
if self.days_between(str(currentDate)[:-9], str(d['Dates'][i])) <= 3:
self.customPrint(d, x)

def main_func(self, cutoff):
StocksController = NasdaqController(True)
list_of_tickers = StocksController.getList()
currentDate = datetime.datetime.strptime(
date.today().strftime("%Y-%m-%d"), "%Y-%m-%d")
start_time = time.time()

Parallel(n_jobs=multiprocessing.cpu_count())(delayed(self.parallel_wrapper)(x, cutoff, currentDate) for x in tqdm(list_of_tickers) )
for x in tqdm(list_of_tickers):
d = (self.find_anomalies_two(self.getData(x), cutoff))
if d['Dates']:
for i in range(len(d['Dates'])):
if self.days_between(str(currentDate)[:-9], str(d['Dates'][i])) <= 3:
self.customPrint(d, x)

print("\n\n\n\n--- this took %s seconds to run ---" %
(time.time() - start_time))
Expand Down
1 change: 0 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,3 @@ numpy==1.18.4
mplcursors==0.3
python_dateutil==2.8.1
tqdm==4.48.0
joblib
3 changes: 1 addition & 2 deletions stocklist.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,7 @@ def __init__(self, update=True):

line = line.strip().split("|")

#line[6] and line[4] is for ETFs. Let's skip those to make this faster.
if line[0] == "" or line[1] == "" or (filename == 'nasdaqlisted' and line[6] == 'Y') or (filename == 'otherlisted' and line[4] == 'Y'):
if line[0] == "" or line[1] == "":
continue

all_listed.write(line[0] + ",")
Expand Down