Feature/provider refactoring (#5266)

* remove comment * replace registry description * black and add some todos * remove unused var * allow description and move provider to title * fix the data field descriptions * refactor globalnews * added `"limit"` to metadata descriptions * fix cpi command and refactor * refactored `AnalystEstimates` * add concat param * refactored `AvailableIndices` for `economy.available_indices` * sort and set index to date * place it inside exception * move literals to core * rename descriptions dict * add descriptions to cpi data * change description * black * refactored `CompanyOverview` for `stocks.fa.overview` * Updated `DATA_DESCRIPTIONS` in provider metadata * Updated `symbol` description in `DATA_DESCRIPTIONS` * fix stock_news alias * pass standard limit * fix stock news * fix to datframe * rebuild python interface * refactored `CryptoEOD` for `crypto.load` * added Field descriptions * fix polygon stock_news * fix stock_news data * ruff * refactored `DividendCalendar` for `stocks.fa.cal` * refactored `EarningsCalendar` for `stocks.fa.cal` * refactored `EarningsCallTranscript` for `stocks.fa.transcript` * refactored `ExecutiveCompensation` for `stocks.fa.comp` * first round, refactor registry, executor, map * ruff * Allow alias in the provider models for provider specific params * Updated stock news to use alias in query * small fixes * fix interface bug * black * exceptions and typing * ruff * fix docstring * sort providers * push static assets without docstrings * date Field and optional dates params in `CryptoEOD` for fmp and polygon * refactored `ForexEOD` for `forex.load` * add comment * refactored `ForexPairs` and added Polygon provider * removed `_price` models from fmp and polygon * Updated `economy.index` to use `MajorIndicesEOD` * removed `_price` from sdk models * refactored `MajorIndicesEOD` for `economy.index` * refactored `StockEOD` for `stocks.load` * regenerate sdk * ruff * fix bug with global_news * typing * add docstrings to registry map * update docstrings * add executor query arg * follow convention * docstring * inject provider interface instead * doc * safer this way * docs * add abstractmethod decorator * add explicit fetcher property model_name * remove mode_name prop * get model name from query params * force extra params or data to be optional * replace fetcher list by fetcher dict and black * explicit depps in prov interface * explicit depps in prov interface * no formatted creds * no formatted creds * dep injection * dep injection * private methods * just verify credentials * docstring * typing and cut doc * typing * typing * typing * no overkill * ruff * don't block if model not found, just warn * don't block if model not found, just warn * don't block if model not found, just warn * empty list if creds None * check creds before request * just return the data * typing * move query params * ruff * move queryparams * move queryparams * move queryparams * fix a bunch o polygon models * ruff * partial fix fmp models * ruff * refactor more fmps * create dict by alias * refactor more fmp endpoints * bug * add params to transform query * ruff * fix fmp bugs * has to be annotation * rewrite static * typing * append return type to map * support return for generic data type * remove return type for openbb provider * add todo * fix small fred typing error * add more return type info * better message * handle multiple types of data (the fetcher should declare them later) * add comment * ruff * handle returns with generic for now (check juan solution) * fix bug with generic * fix fmp fetchr dict * fill return type in the provider lib for docstrings * polygon stock news * Update economy_router.py (#5254) * remove return type stuff * adapt cpi and overview * rewrite python interface * fix to_dataframe * limit stocks eod output * rename method * mypy * mypy * mypy chart style * mypy provider * mypy ignores * mypy credentials * Added Statements Growth endpoints from FMP * account mypy fix * privatize account method * one more mypy * remove mypy ignore * rename private method * Linting * Docstring generation * Linting * move provider.py into __init__.py * add default to required_credentials -> None * refactored `KeyExecutives` for `stocks.fa.mgmt` * refactored `HistoricalEmployees` for `stocks.fa.emp` * provider docstring * docstring * refactored `HistoricalStockSplits` for `stocks.fa.split` * standardised date in stockeod * refactored ``InstitutionalOwnership` for `stocks.fa.own` * removed unused imports * add `check_fields=False` in `@validator` * historical dividens * remove unused helper * move single used helpers to where they are used * move single used helpers to where they are used * move single used helpers to where they are used * standardise date, vwap in crypto eod and forex eod * rename __commands__ to __extensions__ * change coverage separator * the map cannot change sep * sry * sry again * remove comment * refactored `MajorIndicesConstituents` for `economy.const` * add some descriptions about provider * refactored `PriceTargetConsensus` for `stocks.fa.pt` * refactored `PriceTarget` for `stocks.fa.pta` * refactored `RevenueBusinessLine` for `stocks.fa.revseg` * change metadata file name * docstring * remove unecessary arg * ruff & black * refactored `RevenueGeographic` for `stocks.fa.revgeo` * refactored `RiskPremium` for `economy.risk` * refactored `SECFilings` for `stocks.dd.sec` * refactored `ShareStatistics` for `stocks.fa.shrs` * refactored `StockInsiderTrading` for `stocks.fa.ins` * peers * stock splits * refactored `KeyMetrics` for `stocks.fa.metrics` * refactored `StockMultiples` for `stocks.multiples` * refactor income statement * rebuild * fmp balance * added `StocksOwnership` to `stocks.fa.own` * added `InstitutionalOwnership` to `stocks.fa.ins_own` * refactored `StocksOwnership` for `stocks.fa.own` * cleanup * refactored `TreasuryRates` for `fixedincome.treasury` * reorder balance items * fix polygon bs * refactored `IncomeStatement` for `stocks.fa.balance` * refactored base and types * added static files * cleanup * cleanup * refactored `CashFlowStatement` for `stocks.fa.cash` * added `stocks/fa` static file * black and ruff --------- Co-authored-by: Pratyush Shukla <ps4534@nyu.edu> Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
OpenBB-finance · Aug 2, 2023 · be864d0 · be864d0
1 parent 2066d7d
commit be864d0
Show file tree

Hide file tree

Showing 173 changed files with 8,982 additions and 9,742 deletions.
diff --git a/openbb_sdk/extensions/charting/openbb_charting/core/chart_style.py b/openbb_sdk/extensions/charting/openbb_charting/core/chart_style.py
@@ -30,7 +30,7 @@ class ChartStyle:
     """
 
     STYLES_REPO = Path(__file__).parent.parent / "styles"
-    user_styles_directory = STYLES_REPO
+    user_styles_directory: Path = STYLES_REPO
 
     plt_styles_available: Dict[str, Path] = {}
     plt_style: str = "dark"
@@ -58,7 +58,7 @@ def __new__(cls, *args, **kwargs):  # pylint: disable=W0613
     def __init__(
         self,
         plt_style: Optional[str] = "",
-        user_styles_directory: Optional[Union[str, Path]] = None,
+        user_styles_directory: Optional[Path] = None,
     ):
         """Initialize the class.
 

diff --git a/openbb_sdk/extensions/economy/openbb_economy/economy_router.py b/openbb_sdk/extensions/economy/openbb_economy/economy_router.py
@@ -13,7 +13,7 @@
 router = Router(prefix="")
 
 
-# Remove this once the placeholders are replaced with real commands
+# Remove this once the placeholders are replaced with real commands.
 
 # pylint: disable=unused-argument
 
@@ -33,11 +33,10 @@ def const(
     standard_params: StandardParams,
     extra_params: ExtraParams,
 ) -> CommandOutput[BaseModel]:
-    """"""
+    """Get the constituents of an index."""
     return CommandOutput(results=Query(**locals()).execute())
 
 
-# pylint: disable=too-many-arguments
 @router.command(model="CPI")
 def cpi(
     cc: CommandContext,
@@ -57,7 +56,7 @@ def cpi_options(
     return CommandOutput(results=Empty())
 
 
-@router.command(model="MajorIndicesPrice")
+@router.command(model="MajorIndicesEOD")
 def index(
     cc: CommandContext,
     provider_choices: ProviderChoices,

diff --git a/openbb_sdk/extensions/stocks/openbb_stocks/ca/ca_router.py b/openbb_sdk/extensions/stocks/openbb_stocks/ca/ca_router.py
@@ -17,7 +17,7 @@
 
 
 @router.command(model="StockPeers")
-def get(
+def peers(
     cc: CommandContext,
     provider_choices: ProviderChoices,
     standard_params: StandardParams,

diff --git a/openbb_sdk/extensions/stocks/openbb_stocks/fa/fa_router.py b/openbb_sdk/extensions/stocks/openbb_stocks/fa/fa_router.py
@@ -33,6 +33,17 @@ def balance(
     return CommandOutput(results=Query(**locals()).execute())
 
 
+@router.command(model="BalanceSheetGrowth")
+def balance_growth(
+    cc: CommandContext,
+    provider_choices: ProviderChoices,
+    standard_params: StandardParams,
+    extra_params: ExtraParams,
+) -> CommandOutput[BaseModel]:
+    """Balance Sheet Statement Growth."""
+    return CommandOutput(results=Query(**locals()).execute())
+
+
 @router.command(model="DividendCalendar")
 def cal(
     cc: CommandContext,
@@ -55,6 +66,17 @@ def cash(
     return CommandOutput(results=Query(**locals()).execute())
 
 
+@router.command(model="CashFlowStatementGrowth")
+def cash_growth(
+    cc: CommandContext,
+    provider_choices: ProviderChoices,
+    standard_params: StandardParams,
+    extra_params: ExtraParams,
+) -> CommandOutput[BaseModel]:
+    """Cash Flow Statement Growth."""
+    return CommandOutput(results=Query(**locals()).execute())
+
+
 @router.command(model="ExecutiveCompensation")
 def comp(
     cc: CommandContext,
@@ -191,6 +213,17 @@ def income(
     return CommandOutput(results=Query(**locals()).execute())
 
 
+@router.command(model="IncomeStatementGrowth")
+def income_growth(
+    cc: CommandContext,
+    provider_choices: ProviderChoices,
+    standard_params: StandardParams,
+    extra_params: ExtraParams,
+) -> CommandOutput[BaseModel]:
+    """Income Statement Growth."""
+    return CommandOutput(results=Query(**locals()).execute())
+
+
 @router.command(model="StockInsiderTrading")
 def ins(
     cc: CommandContext,
@@ -202,6 +235,17 @@ def ins(
     return CommandOutput(results=Query(**locals()).execute())
 
 
+@router.command(model="InstitutionalOwnership")
+def ins_own(
+    cc: CommandContext,
+    provider_choices: ProviderChoices,
+    standard_params: StandardParams,
+    extra_params: ExtraParams,
+) -> CommandOutput[BaseModel]:
+    """Institutional Ownership."""
+    return CommandOutput(results=Query(**locals()).execute())
+
+
 @router.command
 def key() -> CommandOutput[Empty]:  # type: ignore
     return CommandOutput(results=Empty())
@@ -251,14 +295,14 @@ def overview(
     return CommandOutput(results=Query(**locals()).execute())
 
 
-@router.command(model="InstitutionalOwnership")
+@router.command(model="StockOwnership")
 def own(
     cc: CommandContext,
     provider_choices: ProviderChoices,
     standard_params: StandardParams,
     extra_params: ExtraParams,
 ) -> CommandOutput[BaseModel]:
-    """Institutional Ownership."""
+    """Stock Ownership."""
     return CommandOutput(results=Query(**locals()).execute())
 
 

diff --git a/openbb_sdk/providers/benzinga/openbb_benzinga/__init__.py b/openbb_sdk/providers/benzinga/openbb_benzinga/__init__.py
@@ -0,0 +1,17 @@
+"""Benzinga provider module."""
+from openbb_provider.abstract.provider import Provider
+
+from openbb_benzinga.models.global_news import BenzingaGlobalNewsFetcher
+from openbb_benzinga.models.stock_news import BenzingaStockNewsFetcher
+
+benzinga_provider = Provider(
+    name="benzinga",
+    website="https://www.benzinga.com/",
+    description="""Benzinga is a financial data provider that offers an API
+     focused on information that moves the market.""",
+    required_credentials=["api_key"],
+    fetcher_dict={
+        "GlobalNews": BenzingaGlobalNewsFetcher,
+        "StockNews": BenzingaStockNewsFetcher,
+    },
+)
diff --git a/openbb_sdk/providers/benzinga/openbb_benzinga/models/global_news.py b/openbb_sdk/providers/benzinga/openbb_benzinga/models/global_news.py
@@ -2,79 +2,98 @@
 
 
 from datetime import datetime
-from typing import Dict, List, Optional
+from typing import Any, Dict, List, Literal, Optional
 
-from openbb_provider.abstract.data import QueryParams
 from openbb_provider.abstract.fetcher import Fetcher
-from openbb_provider.helpers import data_transformer, get_querystring
+from openbb_provider.helpers import get_querystring
 from openbb_provider.models.global_news import GlobalNewsData, GlobalNewsQueryParams
-from pydantic import Field
+from openbb_provider.models.stock_news import StockNewsData
+from pydantic import Field, validator
 
-from openbb_benzinga.utils.helpers import BenzingaBaseNewsData, get_data
+from openbb_benzinga.utils.helpers import BenzingaImage, get_data
 
 
-class BenzingaGlobalNewsQueryParams(QueryParams):
+class BenzingaGlobalNewsQueryParams(GlobalNewsQueryParams):
     """Benzinga Global News query.
 
     Source: https://docs.benzinga.io/benzinga/newsfeed-v2.html
-
-    Parameter
-    ---------
-    page : int (default: 0)
-        The page of the data to retrieve.
-    pageSize : int (default: 15)
-        The number of results to return per page.
-    displayOutput : str (default: "headline")
-        The type of data to return. Options are "headline", "summary", "full", "all".
-    date : Optional[datetime]
-        The date of the news to retrieve.
-    dateFrom : Optional[datetime]
-        The start date of the news to retrieve.
-    dateTo : Optional[datetime]
-        The end date of the news to retrieve.
-    updatedSince : Optional[int]
-        The number of seconds since the news was updated.
-    publishedSince : Optional[int]
-        The number of seconds since the news was published.
-    sort : Optional[str]
-        The order in which to sort the news. Options are:
-        "published_at", "updated_at", "title", "author", "channel", "ticker", "topic", "content_type".
-    isin : Optional[str]
-        The ISIN of the news to retrieve.
-    cusip : Optional[str]
-        The CUSIP of the news to retrieve.
-    tickers : Optional[str]
-        The tickers of the news to retrieve.
-    channels : Optional[str]
-        The channels of the news to retrieve.
-    topics : Optional[str]
-        The topics of the news to retrieve.
-    authors : Optional[str]
-        The authors of the news to retrieve.
-    content_types : Optional[str]
-        The content types of the news to retrieve.
     """
 
-    page: int = Field(default=0)
-    pageSize: int = Field(default=15)
-    displayOutput: str = Field(default="headline")
-    date: Optional[datetime] = None
-    dateFrom: Optional[datetime] = None
-    dateTo: Optional[datetime] = None
-    updatedSince: Optional[int] = None
-    publishedSince: Optional[int] = None
-    sort: Optional[str] = None
-    isin: Optional[str] = None
-    cusip: Optional[str] = None
-    tickers: Optional[str] = None
-    channels: Optional[str] = None
-    topics: Optional[str] = None
-    authors: Optional[str] = None
-    content_types: Optional[str] = None
-
-
-class BenzingaGlobalNewsData(BenzingaBaseNewsData):
-    url: str
+    pageSize: int = Field(
+        default=15, description="The number of results to return per page."
+    )
+    displayOutput: Literal["headline", "summary", "full", "all"] = Field(
+        default="headline", description="The type of data to return."
+    )
+    date: Optional[datetime] = Field(
+        default=None, description="The date of the news to retrieve."
+    )
+    dateFrom: Optional[datetime] = Field(
+        default=None, description="The start date of the news to retrieve."
+    )
+    dateTo: Optional[datetime] = Field(
+        default=None, description="The end date of the news to retrieve."
+    )
+    updatedSince: Optional[int] = Field(
+        default=None,
+        description="The number of seconds since the news was updated.",
+    )
+    publishedSince: Optional[int] = Field(
+        default=None,
+        description="The number of seconds since the news was published.",
+    )
+    sort: Optional[
+        Literal[
+            "published_at",
+            "updated_at",
+            "title",
+            "author",
+            "channel",
+            "ticker",
+            "topic",
+            "content_type",
+        ]
+    ] = Field(
+        default=None,
+        description="The order in which to sort the news. Options are: published_at,"
+        " updated_at, title, author, channel, ticker, topic, content_type.",
+    )
+    isin: Optional[str] = Field(
+        default=None, description="The ISIN of the news to retrieve."
+    )
+    cusip: Optional[str] = Field(
+        default=None, description="The CUSIP of the news to retrieve."
+    )
+    tickers: Optional[str] = Field(
+        default=None, description="The tickers of the news to retrieve."
+    )
+    channels: Optional[str] = Field(
+        default=None, description="The channels of the news to retrieve."
+    )
+    topics: Optional[str] = Field(
+        default=None, description="The topics of the news to retrieve."
+    )
+    authors: Optional[str] = Field(
+        default=None, description="The authors of the news to retrieve."
+    )
+    content_types: Optional[str] = Field(
+        default=None, description="The content types of the news to retrieve."
+    )
+
+
+class BenzingaGlobalNewsData(StockNewsData):
+    """Benzinga Global News data."""
+
+    class Config:
+        fields = {"date": "created", "text": "body"}
+
+    image: List[BenzingaImage] = Field(
+        description="The images associated with the news."
+    )
+
+    @validator("date", pre=True)
+    def time_validate(cls, v):  # pylint: disable=E0213
+        return datetime.strptime(v, "%a, %d %b %Y %H:%M:%S %z")
 
 
 class BenzingaGlobalNewsFetcher(
@@ -86,22 +105,17 @@ class BenzingaGlobalNewsFetcher(
     ]
 ):
     @staticmethod
-    def transform_query(
-        query: GlobalNewsQueryParams, extra_params: Optional[Dict] = None
-    ) -> BenzingaGlobalNewsQueryParams:
-        return BenzingaGlobalNewsQueryParams(
-            page=query.page, **extra_params if extra_params else {}
-        )
+    def transform_query(params: Dict[str, Any]) -> BenzingaGlobalNewsQueryParams:
+        return BenzingaGlobalNewsQueryParams(**params)
 
     @staticmethod
     def extract_data(
         query: BenzingaGlobalNewsQueryParams, credentials: Optional[Dict[str, str]]
     ) -> List[BenzingaGlobalNewsData]:
-        if credentials:
-            api_key = credentials.get("benzinga_api_key")
+        api_key = credentials.get("benzinga_api_key") if credentials else ""
 
         base_url = "https://api.benzinga.com/api/v2/news"
-        querystring = get_querystring(query.dict(), [])
+        querystring = get_querystring(query.dict(by_alias=True), [])
         request_url = f"{base_url}?{querystring}&token={api_key}"
         data = get_data(request_url)
 
@@ -113,6 +127,5 @@ def extract_data(
     @staticmethod
     def transform_data(
         data: List[BenzingaGlobalNewsData],
-    ) -> List[GlobalNewsData]:
-        processors = {"image": lambda x: "" if x == [] else x[0].url}
-        return data_transformer(data, GlobalNewsData, processors)
+    ) -> List[BenzingaGlobalNewsData]:
+        return data