Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* implemented course vectorizers under searches/utils
* implemented course vectorizers under searches/utils
* implemented relevance scoring
* implemented relevance scoring
* add get_relevant_search() method returning course objs
* add get_relevant_search() method returning course objs
* implemented stemming / ngram
* implemented stemming / ngram
* test case added
* stemming/acronym/weighting implemented
* evaluation functions implemented
* evaluation functions implemented
* objectified search utility functions
* objectified search utility functions
* Course clustering with LDA
* Course clustering with LDA
* optimized advance search
* optimized advance search
* refactoring codes
* refactoring codes
* rebased from staging
* rebased from staging
* satisfied linting
* satisfied linting
* ingested/digested jhu and modified search flow
* temp
* rebased from staging origin, still need vectorized search migration
* implemented course vectorizers under searches/utils
* implemented course vectorizers under searches/utils
* implemented relevance scoring
* implemented relevance scoring
* add get_relevant_search() method returning course objs
* add get_relevant_search() method returning course objs
* implemented stemming / ngram
* implemented stemming / ngram
* test case added
* stemming/acronym/weighting implemented
* stemming/acronym/weighting implemented
* evaluation functions implemented
* evaluation functions implemented
* objectified search utility functions
* objectified search utility functions
* Course clustering with LDA
* Course clustering with LDA
* optimized advance search
* optimized advance search
* refactoring codes
* refactoring codes
* rebased from staging
* rebased from staging
* satisfied linting
* satisfied linting
* ingested/digested jhu and modified search flow
* temp
* rebased from staging origin, still need vectorized search migration
* do not store count vectorizer obj
* removed pickle file
* revised unit test
* revised unit test
* prevented from crashing when ingesting
* rebased from master
* removed non-using part of the code for now
* modified requirements and cv loader
* updated upon review
* fixed to render searcher obj at app initialization
* resolved migration issue
* rebased and print exception
* modified scoring / strip() query
* resolved dependency issue
* handle duplicates
* changed to use static english dictionary count vectorizer for search.
* added docstrings
* modify license description, formatting and searches.rst description
* merge migrations
* fix tests for vectorized search
* expand on documentation
* cleanup webpack
* cleanup scripts
* fix update_semesterfield importing in migrations
* cleanup root
* update package.json
* provide cause of internal errors by setting debug to true
* fix tests
* remove documentation legacy
* move dict out of root
* umich parser pep8 compliance
* salisbury pep8
* JHU pep8
JHU course evals access db directly and are not pep8 compliant.
80 char limit is not enforced.
* UMD pep8 compliance and refactor
* gw instructor list limit
* vandy pep8 compliance and refactor, UNCOMPLETE
* Validator pep8 compliance
Almost full compliance. Still debating on whether 79 char line limit is worth
making 99 char limit.
* Style compliance for Queens parser
* style complianace for digestor, uncomplete
* working on refactor
* Conitnuing with refactor and pep8 compliance
:plane: to Israel!
* working on dictionary term/semester filtering
* fixed parenthesis syntax error in ingestor
* removed ingest_offerings and changed to ingest_meeting
* removed ingest_offering
* Ingestor refactor
Made detailed docstrings,
created internal _get function and removed getchain
Changed ALL_KEYS to _ALL_KEYS
* Cleaned ingestor cleandict and renamed to clean
+ general refactor of validator but not complete yet
* method reordering in gw parser
* add pep8 newline
* Various refactors
- filter years and terms command line arg and filtering
* spaces instead of tabs in data pipeline json schemas
* removed unneccessary import
* fixed issue created by new years and terms filtering scheme
* style update for commands argparser file
* working on conforming jhu evals to data pipeline
* fixed pep8 line overflow error in hacky way
* changed wording in comment
* Added eval to ingestion command
NOTE: digest command is temporarily broken.
- removed skip_shallow_duplicates from ingest cmd args
-- NOTE: this could very well cause unforseen issues with peoplesoft courses hiding/overriden classes!
- created --type cmd arg for ingestion
-- had to hack in a way to use new_{course, textbook, eval}_parsers, to be changed on deprecation of school_mappers.py
- changed hide_progress_bar to progress_bar (logic flip)
* hopkins evals to pipeline
* Added Vandy/Hopkins evals to data pipeline
NOTE: digestion command is still broken due to argparse errors
- Added evals to ingestor + validator
- tweaked ingestion cmd args some more
- pep8 compliance
* removed extra lines from supervisord config file
* tabs to spaces in logger
* created new exceptions modules, moved some functionality to utils module
* function out complex resolutions
* moved and documented external_utils to utils
* removed invalid reference to internal_utils
* some library reference refactors
* Created JSONWriter in logger but not yet used in code base
* JHU ingestion starts without error
* Refactor Tracker
- Seperated Viewer and Tracker by file and made better distinction.
- used @Property to clean up Tracker class
- Factored out Counter and TimeDistribution into their own Viewer classes
- Create Bus-like protocol so Viewers can filter Tracker broadcasts
* track status and better docstrings
* cleaned up validatoin schema loading
* load_schemas once only in Validator
* Refactor and cleanup validator
* changed parser_library to library
* The big move 🏠 🚚💨 🏠
- moved all parsing functionality into its own module
- created global variable to hold parsing directory in settings.py
- updated school mappers
- BROKEN: uoft was not moved and must be dealt with later, digestion command is still untested
* delete deprecated/moved/redone files b/c git is dumb
* moved makeschool to parsing module
* Trimmed names of files and parsers ✂️
- changed <school}_<pytpe>.py to <ptype>.py
- updated school_mappers to handle new format
- no longer need each type of parser in school directory if unused
- some pep8 things
* tabs to spaces in parsing commands
* Refactored pipeline commands
- use format strings to handle defaults in args
- changed format of progress bar and cleaned up
- started refactor of digestor
* ❗❗ remove parsing files from scripts again... 😡
* Added vectorizer back to digest command
- would like to move this to its own command in due course.
* partial refactor of digestor + bugfix in digest diff
* uncomment .eslintrc.js
* added dummy VALID_SCHOOLS to school_mappers to pass tests
* added uoft to VALID_SCHOOLS even though parser is not active in order to pass tests
* Sort'em 🔤
This should help with conflicts and duplications in these files
* License and registration please 👮
Added Open GPL3 license to parsing directory files with # comment style.
* Fix npm run watch
* add/update testing/contrib docs (#979)
* added everything back (#980)
* Remove client secrets from core code (#981)
* move hashids to local_settings.py
* move all secrets to sensitive.py
* add encrypted secrets as environment variables for travis
* Correct global vars indentation in travis.yml
* use environment variables via travisci web client instead
* handle secret key before imports
* provide dev credentials
* use SemesterlyTest Facebook App credentials
* add docs for secrets
* uncomment out try catch for error printing on get_secret
* Parsing/docs edit (#983)
* Added DataUpdate object to parsing models
- Created DataUpdate object to track last updated status per schools per semester
- Modified SchoolList view and commented out assertNone in associated test
- Used JSONStreamWriter to write ingestor and save meta data for digestor
to read and be able to update DataUpdate objects per semester
- General cleanup as well
- Created Hoarder as tracker viewer to accumulate term, dept, instr data through parse
* removed Updates completely and updated tests to use DataUpdate
* doc and docstring edits- Loading branch information
1 parent
1d8fe40
commit 6d0a62b