Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/sdk v4 bug fixes - Fixing some bugs #5401

Merged
merged 48 commits into from
Sep 5, 2023

Conversation

deeleeramone
Copy link
Contributor

@deeleeramone deeleeramone commented Sep 3, 2023

This PR fixes some provider-related items and rebuilds the static packages:

  • AlphaVantage

    • Rename EOD -> historical
    • Fix validation errors
  • CBOE

    • Add missing router commands
      • index_search()
      • index_snapshots()
    • Index symbols can accept ticker with or without ^
    • Add multi-ticker support for stocks.load()
  • YFinance

    • Use yf.download() as historical prices method for all assets, wrapped as a helper function.
    • Fix intraday prices: stocks, index, crypto, forex
    • Allow index to be called by any field from available_indices()
    • Crypto symbols no longer need a - in the symbol
    • Index doesn't force ^ into the symbol
    • Turns off all yFinance price adjustments.

the-praxs and others added 30 commits August 15, 2023 01:18
* boilerplate for av provider

* adding stock eod for av

* ruff n black

* changing available functions for better ux

* docstring

* move debug_mode out of system settings (#5366)

* move debug_mode out of system settings

* Update router.py

* update readme with debug mode

* dev_mode hub service + openbb_ namespace to Env

* update readme

* copy env to avoid problems w global state

* Linting

* rebuild python interface + fix py38

* Add @classproperty to fetcher for nested return types (#5371)

* add classproperty to fetcher and cpi.py

* type var bound

* validate model in registry map

* better exception msg

* fix fred mypy and add data_type

* typo

* specific model validation

* specific model validation

* reformat error msg

* better

* msg

* add black & ruff to core deps

* supress linter errors if not debug mode

* pylint + regen fixedincome

* py38 compat

* fix bug getting provider for placeholder cmds + ruff

* Revert "fix bug getting provider for placeholder cmds + ruff"

This reverts commit 1328c0b.

* ruff

* avoid bug for placeholder and no provider

* expose available providers in provider interface

* poetry toml to match ci ruff version

* first we ruff and then we black

* add back noqa: E501 to docstrings inside package

* use singleton for provider interface: same functionality (#5372)

* use singleton for provider interface: same functionality

* ruff

* docstring typo

* update readme with tuna reference

* move to correct readme

* Merge

* Some error handling (#5357)

* Removed prints (#5367)

* Removed prints

* FIxed typing

* FIxed typing

* Fixed typing

* Fixed typing

* Fixed typing

* Fixed typing

* Fixed typing

* Fix

* Reverted stuff

* Feature/move package (#5374)

* move package to openbb_sdk

* avoid test writing files

* black

* revert toml changes, will do it in another PR

* run build in a separate process

* change error msg

* rename func

* change msg

* rename func + msg

* change param name

* update readme

* lazy load some deps

* Update README.md

* modules is better

* doc

* ruff

* pylint

* fix autocomplete (#5383)

* Core tests (#5375)

* makeing the function easier - @montezdesousa pls review this commit

* improve update_provider_choices readbility

* unncessary classmethod

* command runner tests

* fixing wrong attribution

* some tests for the charting manager

* changing fetcher and models to have a shorter name

* @the-prax suggestions

* fixing references to av

* making function a private fild so the user don't access it

* unbreaking changes

* Improve snake_case generation

* adding av to pyproject.toml

* stating the function as a private field while keeping validation for function specific fields

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Colin Delahunty <72827203+colin99d@users.noreply.github.com>
* rename EOD to Historical in providers

* renamed EOD to Historical in sdk/provider

* rename _eod to _historical in standard_models

* rename _eod to _historical in providers directory

* cleanup

* rename EOD to Historical in extensions

* static

* Revert "rename EOD to Historical in extensions"

* rename EOD to Historical in relevant extensions

* fix spelling in init

* static

* linting

* intrinio y u alone

* so sorry intrinio

* static

* ruff

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>
@deeleeramone deeleeramone added the v4 PRs for v4 label Sep 3, 2023
@reviewpad reviewpad bot added the feat XL Extra Large feature label Sep 3, 2023
@the-praxs the-praxs changed the base branch from feature/add-data-providers to feature/add-data-providers-fix September 4, 2023 21:14
Base automatically changed from feature/add-data-providers-fix to feature/openbb-sdk-v4 September 4, 2023 21:29
@the-praxs the-praxs changed the base branch from feature/openbb-sdk-v4 to fix/provider-bugs September 5, 2023 19:27
@reviewpad reviewpad bot added feat S Small T-Shirt size Feature and removed feat XL Extra Large feature labels Sep 5, 2023
@the-praxs the-praxs merged commit 4588fa3 into fix/provider-bugs Sep 5, 2023
@the-praxs the-praxs deleted the feature/sdk-v4-bug-fixes branch September 5, 2023 19:32
@deeleeramone deeleeramone restored the feature/sdk-v4-bug-fixes branch September 5, 2023 22:43
@the-praxs the-praxs deleted the feature/sdk-v4-bug-fixes branch September 6, 2023 05:04
@the-praxs the-praxs restored the feature/sdk-v4-bug-fixes branch September 6, 2023 05:04
@the-praxs the-praxs deleted the feature/sdk-v4-bug-fixes branch September 6, 2023 05:04
@the-praxs the-praxs restored the feature/sdk-v4-bug-fixes branch September 6, 2023 05:04
@the-praxs the-praxs deleted the feature/sdk-v4-bug-fixes branch September 6, 2023 05:09
the-praxs added a commit that referenced this pull request Sep 19, 2023
* added `StockNews` to yfinance

* make `symbol` optional in `IncomeStaementData`

* added financial statements

* intrinio provider package init

* added `StockEOD` model

* added `ForexPairs` model

* added `StockNews` model

* added `GlobalNews` model

* added Fetchers for models

* static files

* cleanup

* fetch all pages in intrinio

* static

* refactored `StockQuote` standard model

* refactored `StockQuote` fmp model

* added `StockQuote` for Intrinio

* static files

* Adding AV and StockEOD (#5365)

* boilerplate for av provider

* adding stock eod for av

* ruff n black

* changing available functions for better ux

* docstring

* move debug_mode out of system settings (#5366)

* move debug_mode out of system settings

* Update router.py

* update readme with debug mode

* dev_mode hub service + openbb_ namespace to Env

* update readme

* copy env to avoid problems w global state

* Linting

* rebuild python interface + fix py38

* Add @classproperty to fetcher for nested return types (#5371)

* add classproperty to fetcher and cpi.py

* type var bound

* validate model in registry map

* better exception msg

* fix fred mypy and add data_type

* typo

* specific model validation

* specific model validation

* reformat error msg

* better

* msg

* add black & ruff to core deps

* supress linter errors if not debug mode

* pylint + regen fixedincome

* py38 compat

* fix bug getting provider for placeholder cmds + ruff

* Revert "fix bug getting provider for placeholder cmds + ruff"

This reverts commit 1328c0b.

* ruff

* avoid bug for placeholder and no provider

* expose available providers in provider interface

* poetry toml to match ci ruff version

* first we ruff and then we black

* add back noqa: E501 to docstrings inside package

* use singleton for provider interface: same functionality (#5372)

* use singleton for provider interface: same functionality

* ruff

* docstring typo

* update readme with tuna reference

* move to correct readme

* Merge

* Some error handling (#5357)

* Removed prints (#5367)

* Removed prints

* FIxed typing

* FIxed typing

* Fixed typing

* Fixed typing

* Fixed typing

* Fixed typing

* Fixed typing

* Fix

* Reverted stuff

* Feature/move package (#5374)

* move package to openbb_sdk

* avoid test writing files

* black

* revert toml changes, will do it in another PR

* run build in a separate process

* change error msg

* rename func

* change msg

* rename func + msg

* change param name

* update readme

* lazy load some deps

* Update README.md

* modules is better

* doc

* ruff

* pylint

* fix autocomplete (#5383)

* Core tests (#5375)

* makeing the function easier - @montezdesousa pls review this commit

* improve update_provider_choices readbility

* unncessary classmethod

* command runner tests

* fixing wrong attribution

* some tests for the charting manager

* changing fetcher and models to have a shorter name

* @the-prax suggestions

* fixing references to av

* making function a private fild so the user don't access it

* unbreaking changes

* Improve snake_case generation

* adding av to pyproject.toml

* stating the function as a private field while keeping validation for function specific fields

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Colin Delahunty <72827203+colin99d@users.noreply.github.com>

* Rename EOD to Historical in code and files (#5395)

* rename EOD to Historical in providers

* renamed EOD to Historical in sdk/provider

* rename _eod to _historical in standard_models

* rename _eod to _historical in providers directory

* cleanup

* rename EOD to Historical in extensions

* static

* Revert "rename EOD to Historical in extensions"

* rename EOD to Historical in relevant extensions

* fix spelling in init

* static

* linting

* intrinio y u alone

* so sorry intrinio

* static

* ruff

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* feature/quandl-provider: Add Quandl provider to SDK-V4 (#5339)

* adds futures term structure from cboe

* Linting

* error handling for empty response

* codespell

* rename term_structure --> curve

* unused imports

* Improve import speed for CBOE

* add quandl provider

* doc strings

* black

* moves series id definitions to a separate file

* black

* add pandas dependency

* fix pyproject.toml

* refactor to load nothing on runtime

* black

* futures_curve model

* fix docstring

* static directory file no longer needed

* helper for european index constituents

* remove abstract request function which gets rid of Numpy import

* remove Numpy from dependencies

* add Europe class to helpers and European index data

* symbol.upper()

* add european_index to economy module

* add US indices in

* adds cboe as provider for major_indices_eod

* enforce snake_case for FMP columns

* add

* fix FMP non-intraday

* add european_index_constituents

* black

* update docstring in economy_router

* add cot_search() and makes helper function for querying the Commitment of Traders Reports

* refactor quandl

* docstring

* refactor cboe provider

* black

* merge feature/openbb-sdk-v4 into branch

* add Intrinio options chains

* codespell exception

* remove manual camelCase conversion in FMP StockEOD and MajorIndicesEOD

* fix polygon - MajorIndicesEOD

* fix FMP intraday stripping the time

* more codespell

* put validators back - fmp

* enhance yFinance indices

* timestamp to str

* yahoo index - empty no breaky

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* hotfix intrinio

* better cboe stock_info error handling

* fix merge

* handling for if ^ in symbol

* make alpha_vantage work

* fix yfinance intraday - stock historical

* fix yfinance major_indices_historical

* fix yfinance crypto_historical

* fix yfinance forex_historical

* fix yfinance crypto and forex

* economy router and package rebuild

* black

* add multi-ticker support for cboe stock_historical

* use yf_download for crypto and forex instead of yf(Ticker).history()

* potentially allow multiple tickers

* feature/sdk v4 bug fixes - Fixing some bugs (#5401)

* added `StockNews` to yfinance

* make `symbol` optional in `IncomeStaementData`

* added financial statements

* intrinio provider package init

* added `StockEOD` model

* added `ForexPairs` model

* added `StockNews` model

* added `GlobalNews` model

* added Fetchers for models

* static files

* cleanup

* fetch all pages in intrinio

* static

* refactored `StockQuote` standard model

* refactored `StockQuote` fmp model

* added `StockQuote` for Intrinio

* static files

* Adding AV and StockEOD (#5365)

* boilerplate for av provider

* adding stock eod for av

* ruff n black

* changing available functions for better ux

* docstring

* move debug_mode out of system settings (#5366)

* move debug_mode out of system settings

* Update router.py

* update readme with debug mode

* dev_mode hub service + openbb_ namespace to Env

* update readme

* copy env to avoid problems w global state

* Linting

* rebuild python interface + fix py38

* Add @classproperty to fetcher for nested return types (#5371)

* add classproperty to fetcher and cpi.py

* type var bound

* validate model in registry map

* better exception msg

* fix fred mypy and add data_type

* typo

* specific model validation

* specific model validation

* reformat error msg

* better

* msg

* add black & ruff to core deps

* supress linter errors if not debug mode

* pylint + regen fixedincome

* py38 compat

* fix bug getting provider for placeholder cmds + ruff

* Revert "fix bug getting provider for placeholder cmds + ruff"

This reverts commit 1328c0b.

* ruff

* avoid bug for placeholder and no provider

* expose available providers in provider interface

* poetry toml to match ci ruff version

* first we ruff and then we black

* add back noqa: E501 to docstrings inside package

* use singleton for provider interface: same functionality (#5372)

* use singleton for provider interface: same functionality

* ruff

* docstring typo

* update readme with tuna reference

* move to correct readme

* Merge

* Some error handling (#5357)

* Removed prints (#5367)

* Removed prints

* FIxed typing

* FIxed typing

* Fixed typing

* Fixed typing

* Fixed typing

* Fixed typing

* Fixed typing

* Fix

* Reverted stuff

* Feature/move package (#5374)

* move package to openbb_sdk

* avoid test writing files

* black

* revert toml changes, will do it in another PR

* run build in a separate process

* change error msg

* rename func

* change msg

* rename func + msg

* change param name

* update readme

* lazy load some deps

* Update README.md

* modules is better

* doc

* ruff

* pylint

* fix autocomplete (#5383)

* Core tests (#5375)

* makeing the function easier - @montezdesousa pls review this commit

* improve update_provider_choices readbility

* unncessary classmethod

* command runner tests

* fixing wrong attribution

* some tests for the charting manager

* changing fetcher and models to have a shorter name

* @the-prax suggestions

* fixing references to av

* making function a private fild so the user don't access it

* unbreaking changes

* Improve snake_case generation

* adding av to pyproject.toml

* stating the function as a private field while keeping validation for function specific fields

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Colin Delahunty <72827203+colin99d@users.noreply.github.com>

* Rename EOD to Historical in code and files (#5395)

* rename EOD to Historical in providers

* renamed EOD to Historical in sdk/provider

* rename _eod to _historical in standard_models

* rename _eod to _historical in providers directory

* cleanup

* rename EOD to Historical in extensions

* static

* Revert "rename EOD to Historical in extensions"

* rename EOD to Historical in relevant extensions

* fix spelling in init

* static

* linting

* intrinio y u alone

* so sorry intrinio

* static

* ruff

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* feature/quandl-provider: Add Quandl provider to SDK-V4 (#5339)

* adds futures term structure from cboe

* Linting

* error handling for empty response

* codespell

* rename term_structure --> curve

* unused imports

* Improve import speed for CBOE

* add quandl provider

* doc strings

* black

* moves series id definitions to a separate file

* black

* add pandas dependency

* fix pyproject.toml

* refactor to load nothing on runtime

* black

* futures_curve model

* fix docstring

* static directory file no longer needed

* helper for european index constituents

* remove abstract request function which gets rid of Numpy import

* remove Numpy from dependencies

* add Europe class to helpers and European index data

* symbol.upper()

* add european_index to economy module

* add US indices in

* adds cboe as provider for major_indices_eod

* enforce snake_case for FMP columns

* add

* fix FMP non-intraday

* add european_index_constituents

* black

* update docstring in economy_router

* add cot_search() and makes helper function for querying the Commitment of Traders Reports

* refactor quandl

* docstring

* refactor cboe provider

* black

* merge feature/openbb-sdk-v4 into branch

* add Intrinio options chains

* codespell exception

* remove manual camelCase conversion in FMP StockEOD and MajorIndicesEOD

* fix polygon - MajorIndicesEOD

* fix FMP intraday stripping the time

* more codespell

* put validators back - fmp

* enhance yFinance indices

* timestamp to str

* yahoo index - empty no breaky

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* hotfix intrinio

* better cboe stock_info error handling

* fix merge

* handling for if ^ in symbol

* make alpha_vantage work

* fix yfinance intraday - stock historical

* fix yfinance major_indices_historical

* fix yfinance crypto_historical

* fix yfinance forex_historical

* fix yfinance crypto and forex

* economy router and package rebuild

* black

* add multi-ticker support for cboe stock_historical

* use yf_download for crypto and forex instead of yf(Ticker).history()

* potentially allow multiple tickers

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Colin Delahunty <72827203+colin99d@users.noreply.github.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>

* no reset index on yfinance crypto/forex

* added missing index search

* `stock_info` removed merge symbols

* static

* `\%` to `%` in FMP historical models

* removed fields

* set `period` and `cik` to `Optional`

* fix yfinance financial statements

* cleaned duplicate code

* static

* static

* fixed yfinance stock news

* improved yfinance stock news

* cleanup

* set `date` field type to `datetime` (again)

* add date validator

* `BaseSymbol` eliminated

* linting

* Add Quandl functions to economy router

* YFinance Statement Bugs

* Fix/provider-bugs-intrinio: fix bad API key response for options symbols (#5419)

* fix bad api key response for symbol directory

* move model functions into Fetcher class as static methods

* make ruff and black play together

* description fix

* move options tickers and expirations to helpers

* vw->vwap

* Remove confusing columns in cpi model

* `news.globalnews` has `limit` set as standardized param

* intrinio stocks historical fetch data robustly

* fix too many .reset_index()

* ruff

* fixed standardization

* cleanup

* `stocks.news` params standardized

* polygon `StockNews` model revamped

* intrinio `StockNews` model params standardized

* `link` and `final_link` set as `Optional` to fix validation errors

* benzinga cleanup

* linting

* set `type` to `Optional`

* removed extra cash_flow file

* `FinancialStatementQueryParams` eliminated

* Polygon `StockHistorical` returns all results

* `PolygonFundamentalQueryParams` eliminated

* Added source in doc

* `^` needed in symbol for FMP

* fix validation error for negative eps values

* change to latest v2 params

* correct descriptions

* removed `stocks.dd` and moved `sec` to `fa`

* `obb.news.globalnews` revamped

* `stocks.news` revamped

* `list` to `List`

* adding List instead of list

* `options.chains` Intrinio cleanup and standardization

* linting

* cleanup intrinio

* static

* bump package versions

* fixed date validation errors for yfinance

* YFinance updates

* Fix FMP tests

* Fix Intrinio

* Re-record Benzinga

* Fix Polygon fetchers and tests

* Re-record and fix CBOE

* Re-record and fix Quandl

* Some FMP fixes and hardcoded tests

* fmp stock ownership refactor and fix test

* fix fmp treasury rate `transform_query` to pass test

* fix fmp dividend calendar `transform_query` to pass test

* fix fmp historical models `transform_query` to pass test

* Fix Polygon fetchers and tests

* Fix Intrinio test and fetcher

* Re-record and fix YFinance

* Fred

* Update test generator

* CBOE cleanup

* Fix tests (again)

* static

* fix more tests

* fix yfinance tests

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Colin Delahunty <72827203+colin99d@users.noreply.github.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat S Small T-Shirt size Feature v4 PRs for v4
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants