Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing periodic fetching, refactoring backend code #176

Merged
merged 27 commits into from
Apr 27, 2018
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
6a52ad6
changes that not quite fix the transaction conflict errors
Qqwy Apr 24, 2018
5f081b0
Fetching code with tasks that works
Qqwy Apr 25, 2018
7c34ff7
Refactoring
Qqwy Apr 25, 2018
63bae52
Refactoring into services
Qqwy Apr 25, 2018
dcadd2a
Refactoring done?
Qqwy Apr 25, 2018
013b877
move impl.fetch to flask_monitoring_dashboard_client module
Qqwy Apr 25, 2018
e5eecd5
Moving 'impl' dependencies to separate packages
Qqwy Apr 25, 2018
c391005
Merge branch 'development' into backend/periodic_fetching
Qqwy Apr 25, 2018
ad8f05a
cleaner task name
Qqwy Apr 25, 2018
295bbe9
Extracting pydash_database and pydash_logger
Qqwy Apr 25, 2018
c13c708
Removing sync() statement now that we explicitly begin transactions
Qqwy Apr 25, 2018
c16c51e
WIP setting up pytest.
Qqwy Apr 25, 2018
2d331a0
Fixed gitignore
Qqwy Apr 25, 2018
1086e28
TESTING environment variable during tests
Qqwy Apr 25, 2018
0837cba
endpoint call test
Qqwy Apr 25, 2018
552d67c
Tests for the EndpointCall entity.
Qqwy Apr 25, 2018
d9f40b9
some tests in the tasks scheduler
Qqwy Apr 25, 2018
7f4ca8f
Tests for the Tasks scheduler
Qqwy Apr 25, 2018
a59e1c3
dashboard tests
Qqwy Apr 25, 2018
4f42b57
make dasboard tests time independent
Qqwy Apr 26, 2018
fbb6ca1
Unit test for the User
Qqwy Apr 26, 2018
cf12f70
user testing
Qqwy Apr 26, 2018
63f834d
finished User Repository unit tests
Qqwy Apr 26, 2018
63c8600
Ensures test isolation by flushing the in-memory database after each
Qqwy Apr 26, 2018
992cfca
Fixes problems with build_and_run script
Qqwy Apr 26, 2018
facbd5e
maybe fix the 'transaction not started' errors?
Qqwy Apr 26, 2018
1669bf6
Fixes react serving
Qqwy Apr 26, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -113,3 +113,4 @@ ENV/
.vscode
#Directories
/pydash/logs
/pydash/.pytest_cache
11 changes: 11 additions & 0 deletions Pipfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]

[requires]
python_version = "3.6"
3 changes: 2 additions & 1 deletion build_and_run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@ RunFlask()
{
PydashPrint "Finally: Starting flask webservice. Close with Ctrl+C"
cd pydash
pipenv run "flask run --no-reload"
pipenv run flask run --no-reload
cd ..
}

BuildFrontend
Expand Down
13 changes: 4 additions & 9 deletions pydash/Pipfile
Original file line number Diff line number Diff line change
@@ -1,12 +1,9 @@
[[source]]

url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"


[packages]

Flask = "*"
Flask-WTF = "*"
flask-login = "*"
Expand All @@ -18,16 +15,14 @@ requests = "*"
Flask-Cors = "*"
pqdict = "*"


[dev-packages]


pytest = "*"
pytest-cov = "*"
pytest-xdist = "*"
pytest-env = "*"

[requires]

python_version = "3.6"


[pipenv]

keep_outdated = true
135 changes: 133 additions & 2 deletions pydash/Pipfile.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions pydash/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import pytest
import pydash_database
import pydash_app.user.repository
import pydash_app.dashboard.repository
@pytest.fixture(autouse=True)
def clean_in_memory_database(*_):
pydash_app.user.repository.clear_all()
pydash_app.dashboard.repository.clear_all()

Original file line number Diff line number Diff line change
@@ -1,14 +1,22 @@
"""
Performs the remote requests to the flask-monitoring-dashboard.

The method names in this module 1:1 reflect the names of the flask-monitoring-dashboard API
(but without the word 'JSON' in them, because conversion from JSON to Python dictionaries/lists
is one of the thing this module handles for you.)
"""

import requests
import jwt
import json

import pydash_app.impl.logger as pylog
import pydash_logger

DETAILS_ENDPOINT = 0
RULES_ENDPOINT = 1
DATA_ENDPOINT = 2

logger = pylog.Logger(__name__)
logger = pydash_logger.Logger(__name__)

def get_details(dashboard_url):
"""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,26 +19,31 @@
The scheduler will be started by calling the `start()` function. It will stop scheduling and tear down the spawned processes when calling the `stop()` function.
This function will also (in most cases) be automatically called when the main process finishes execution.


Example code with default scheduler:


>>> import pydash_app.impl.periodic_tasks as pt
>>> import periodic_tasks as pt
>>> import datetime
>>> pt.start_default_scheduler()
>>> pt.add_periodic_task('foo', datetime.timedelta(seconds=3), pt.foo)
>>> pt.add_periodic_task('bar', datetime.timedelta(seconds=5), pt.bar)
>>> pt.add_background_task('baz', pt.baz)
>>> pt.add_periodic_task('bar', datetime.timedelta(seconds=1), pt.bar) # overrides previous `bar` task with new settings
>>> pt.remove_task('foo')
>>> pt.default_task_scheduler.stop()


Example code with custom scheduler:

>>> import pydash_app.impl.periodic_tasks as pt
>>> import periodic_tasks as pt
>>> ts = pt.TaskScheduler()
>>> import datetime
>>> import datetime, time
>>> ts.start()
>>> ts.add_periodic_task('foo', datetime.timedelta(seconds=1), pt.foo)
>>> ts.add_periodic_task('bar', datetime.timedelta(seconds=5), pt.bar)
>>> ts.add_periodic_task('foo', datetime.timedelta(milliseconds=1), pt.foo)
>>> ts.add_periodic_task('bar', datetime.timedelta(milliseconds=5), pt.bar)
>>> time.sleep(2)
>>> ts.stop()
"""

from .task_scheduler import TaskScheduler
Expand Down
Loading