Feature/finviz screener

[mmistroni]

Pull Request Template for OpenBB Developers

0. Finviz Screener:

   • Enhancement to Finviz Provider to leverage Finviz Screener capabilities

1. Why?

   • Leverages Finviz Screener filters for stock filtering or stock discovery: these filters are more flexible than the ones provided by FMP screener

2. What? (1-3 sentences or a bullet point list):

   • Added FinvizScreener provider that leverages finviz finance library to allow more flexible stock filtering

3. Impact (1-2 sentences or a bullet point list):

   • No impact on existing code base at the moment as the Provider is not being linked to any routers

4. Testing Done:

   • Validated via HTTP unit tests, providing sample filters as an example

[deeleeramone]

Hello, and thanks for the PR! This one is definitely on the radar of "things to add", so it's really nice to see someone taking the initiative.

I'm going to bring up some high-level things, and am open to collaborating on the details of the implementation. Please don't take anything personally, or as an obligation to commit, the effort in submitting is very much appreciated nonetheless.

Everyone will appreciate a diverse, robust, screener that is easy-to-use, repeatable, and reliable. Let's work towards making it a reality.

Don't worry about linters and tests right now.

The main challenge to overcome is that nobody will know how to use it, and they will not be likely to figure it out on their own.

The good news is that most of the dirty work has already been done by the finvizfinance library . Integrating and communicating that to the user will still require a lot of mapping behind the scenes to make it plug-and-play smooth for them.

What we need to do is break down the 'possibilities' into digestible pieces that can be reliably known and configured by the end user.

• A template will be necessary.
• Choices should be validated by the model (can be accomplished by defining Literal types), eliminating any guess work and providing the user with actionable information for how they can adjust/correct their choices.

The finvizfinance library gives us entry points for the "category" of metrics to screen from:

• overview
• performance
• valuation
• technical
• ownership

You have overview exposed, but the above can be parameterized as a top-level parameter whose value will have an impact on the availability of other parameters, as well as the content of the returned data.

Consequently, the returned data model needs to reflect all possibilities - similar to the groups model, which is a bit less dense than the stock screener outputs are.

From there the parameters can be broken down into two basic groups:

• global
• category

To start, let's just isolate the global choices.

At the architecture level, we need to ensure that:

• The provider inherits from the existing standard model and maps to the existing EquityScreener Provider Interface model in the Finviz Provider definition.
  • There are existing parameters, for other providers, that can also be mapped to known accepted values from Finviz - 'exchange', 'marketcap', 'volume', 'industry', 'sector', all come to mind. These should all be explicitly defined as Optional[Literal['value1', 'value2', 'value3', etc.] with a default value of None.
  • Additional global choices should be defined similarly.

When these considerations are addressed, the next logical steps will become much more apparent. This type of function is a complex integration, and there aren't very many shortcuts to apply other than leveraging the work accomplished by the other libraries in the chain.

We can divide and conquer, thoughts?

[mmistroni]

Sure...no problem...yes difficult to see how user can use it as it should b exposed via some routers..and should also be in line with existing fmp screener..but fmp screener accepts en mktcap > 10000 where finviz has predefined options . Let me have a thought..based on your comments above and will report back. Thanks

…

On Tue, 16 Jul 2024, 04:46 Danglewood, ***@***.***> wrote: ***@***.**** requested changes on this pull request. Hello, and thanks for the PR! This one is definitely on the radar of "things to add", so it's really nice to see someone taking the initiative. I'm going to bring up some high-level things, and am open to collaborating on the details of the implementation. Please don't take anything personally, or as an obligation to commit, the effort in submitting is very much appreciated nonetheless. Everyone will appreciate a diverse, robust, screener that is *easy-to-use*, repeatable, and reliable. Let's work towards making it a reality. Don't worry about linters and tests right now. *The main challenge to overcome is that nobody will know how to use it, and they will not be likely to figure it out on their own.* The good news is that most of the dirty work has already been done by the finvizfinance library <https://github.com/lit26/finvizfinance/blob/master/finvizfinance/constants.py>. Integrating and communicating that to the user will still require a lot of mapping behind the scenes to make it plug-and-play smooth for them. What we need to do is break down the 'possibilities' into digestible pieces that can be reliably known and configured by the end user. - A template will be necessary. - Choices should be validated by the model, eliminating any guess work and providing the user with actionable information for how they can adjust/correct their choices. The finvizfinance library gives us entry points for the "category" of metrics to screen from: - overview - performance - valuation - technical - ownership You have overview exposed, but the above can be parameterized as a top-level parameter whose value will have an impact on the availability of other parameters, as well as the content of the returned data. Consequently, the returned data model needs to reflect all possibilities - similar to the groups model <https://github.com/OpenBB-finance/OpenBB/blob/develop/openbb_platform/providers/finviz/openbb_finviz/models/compare_groups.py>, which is a bit less dense than the stock screener outputs are. From there the parameters can be broken down into two basic groups: - global - category To start, let's just isolate the global choices. At the architecture level, we need to ensure that: - The provider inherits from the existing standard model and maps to the existing EquityScreener Provider Interface model in the Finviz Provider definition. - There are existing parameters <https://docs.openbb.co/platform/reference/equity/screener>, for other providers, that can also be mapped to known accepted values from Finviz <https://github.com/lit26/finvizfinance/blob/master/finvizfinance/constants.py> - 'exchange', 'marketcap', 'volume', 'industry', 'sector', all come to mind. These should all be explicitly defined as Optional[Literal['value1', 'value2', 'value3', etc.] with a default value of None. - Additional global choices should be defined similarly. When these considerations are addressed, the next logical steps will become much more apparent. This type of function is a complex integration, and there aren't very many shortcuts to apply other than leveraging the work accomplished by the other libraries in the chain. We can divide and conquer, thoughts? — Reply to this email directly, view it on GitHub <#6586 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACPNCDSXU4ALYQYFK45OWYTZMSJRPAVCNFSM6AAAAABK33CMQKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDCNZZGE2TAMRRGY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[mmistroni]

@deeleeramone
Ok here's my plan
1 - Make the FinvizScreener 'compatible' with EquityScreener so that at least it supporst what existing screener do
- i will need also some mapping, as FMP params are just numbers - whereas the finviz screener has pre-defined strings -
2 - Need to somehow 'fit' an extra parameter which will allow to leverage all the other possibilities available from finviz (and which are not avaiable in FMP). perhaps, as a start, i can start by having a look at what you have in your group model...
I'll start with something and report back
pls feel free to add / comment / dispute
rgs

[deeleeramone]

1 - i will need also some mapping, as FMP params are just numbers - whereas the finviz screener has pre-defined strings

Nasdaq has a similar filter setup as mktcap:

mktcap : Union[Literal['all', 'mega', 'large', 'mid', 'small', 'micro'], str]
    Filter by market cap.
    Mega - > 200B
    Large - 10B - 200B
    Mid - 2B - 10B
    Small - 300M - 2B
    Micro - 50M - 300M
 Multiple comma separated items allowed. (provider: nasdaq)

Finviz has just one more category, 'nano', otherwise it's pretty much the same.

2 - Need to somehow 'fit' an extra parameter which will allow to leverage all the other possibilities available from finviz (and which are not avaiable in FMP). perhaps, as a start, i can start by having a look at what you have in your group model... I'll start with something and report back pls feel free to add / comment / dispute rgs

I can work on some logic for preset integrations that can function as an override to any hard-coded params.

[mmistroni]

Hello
ok , so 2 qqq

1 -

There are existing parameters, for other providers, that can >>also be mapped to known accepted values from [Finviz]>>(https://github.com/lit26/finvizfinance/blob/master/finvizfinance/constants.py) - 'exchange', 'marketcap', 'volume', 'industry', >>'sector', all come to mind. These should all be explicitly defined as Optional[Literal['value1', 'value2', 'value3', etc.] with a >>default value of None.

• FMPEQSP has mktcap_min, mktcap_max, etc
• NasdaqEQSP has mktcap, extsubcategory etc

you were suggesting that eg FinvizEquityScreenerQueryParams should incorporate parameters from both - somehow - ?

i guess that would make sense as, as i have seen, atm FMP and Nasdaq QueryParams are quite different and introducing a third screener with completely brand new parameters will just confuse everyone ?

2 -

A template will be necessary.
Had a look at utils\finviz_helper, is the idea that if i want to build a custom screener i will do it via a template which already configures the filter to use, rather than passing in the filters to use in a dictionary parameters at runtime, like in the codebase below ?
price_filters = {
'20-Day Simple Moving Average': 'Price above SMA20',
'50-Day Simple Moving Average': 'Price above SMA50',
'200-Day Simple Moving Average': 'Price above SMA200',
'52-Week High/Low': '0-10% below High'
}

desc_filters = {
    'Average Volume': 'Over 200K',
    'Float' : 'Under 50M',
}

fund_filters = {
    'Average Volume': 'Over 200K',
    'Float': 'Under 50M',
    'EPS growththis year': 'Over 20%',
    'EPS growthnext year': 'Over 20%',
    'EPS growthqtr over qtr': 'Over 20%',
    'Sales growthqtr over qtr': 'Over 20%',
    'EPS growthpast 5 years': 'Over 20%',
    'Gross Margin': 'Positive (>0%)',
    'Return on Equity': 'Positive (>0%)',
    'InstitutionalOwnership': 'Over 20%'
}

filters_dict = price_filters
filters_dict.update(desc_filters)
filters_dict.update(fund_filters)

return obb.equity.screener(filters=filters_dict, provider='finviz') ?

Reason why i m asking is because my planned usage of OBB is not 'interactive'. I am planning to setup a server which runs uvicorn / obb REST, and have a separate process which calls OBB REST api to retrieve the data.
And in this usage i am planning to pass parameters at runtime as i dont want to reboot / update the uvicorn machine every time i come up with a new screener
But i guess i am an edge-case, so i'll b happy to go along and get the 'template' based version running, and then perhaps i'll have a look on how to achieve my REST stuff

Kind regards
Marco

[mmistroni]

sorry i m off for a week. thought to let you know i case you dont see any replies from me.i'll pick up this in Aug.
rgds

[deeleeramone]

sorry i m off for a week. thought to let you know i case you dont see any replies from me.i'll pick up this in Aug. rgds

Not meaning to steal your thunder, but I opened a new PR that incorporates your requirements and contribution to the effort, #6601. An entry point for filters_dict is provided, as well as presets, so there is quite a bit of flexibility in operation.

Presets are all read on run time, so there is no need to reload the Python interpreter. Both, presets and filters_dict, will raise actionable errors from bad key/values.

I appreciate your contribution to the community, and would love to include some of your favourite presets in the library defaults. Try it out and let me know if it accomplishes your requirements.

[mmistroni]

Great!! Sure I will try it out once back as I have at least 6 custom presets .....thanks for bringing it through!! Kind regards

…

On Mon, 29 Jul 2024, 18:27 Danglewood, ***@***.***> wrote: sorry i m off for a week. thought to let you know i case you dont see any replies from me.i'll pick up this in Aug. rgds Not meaning to steal your thunder, but I opened a new PR that incorporates your requirements and contribution to the effort, #6601 <#6601>. An entry point for filters_dict is provided, as well as presets, so there is quite a bit of flexibility in operation. Screenshot.2024-07-29.at.9.22.21.AM.png (view on web) <https://github.com/user-attachments/assets/e24ce7f3-af29-439b-95fc-e5ba51845e2d> Presets are all read on run time, so there is no need to reload the Python interpreter. Both, presets and filters_dict, will raise actionable errors from bad key/values. I appreciate your contribution to the community, and would love to include some of your favourite presets in the library defaults. Try it out and let me know if it accomplishes your requirements. — Reply to this email directly, view it on GitHub <#6586 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACPNCDWT7INICMRFJ4JEUZ3ZOZUPZAVCNFSM6AAAAABK33CMQKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJWGM4DEMZQGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>