Add extra columns for custom screener

[lit26]

Description

Fix #86

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update
• Code styling or code optimize

What you did

[ljilekor]

There is still an issue

The new columns where not added to NUMBER_COL in util.py

[lit26]

Oh, I will take a look.

[ljilekor]

And I also suspect something is wrong with the 'Dividend' column.
Or should I say, columnS

There seems to be 2. (dollar price and %yield)
The initial dataframe created in the screener_view() from Custom.py
contains 'Dividend' twice!

But once we're in the _get_table() overview.py , the second 'Dividend', one of the new columns, is not there.

• I pumped the resulting 'frame' in a 'new_df' DataFrame for convenience...
  new_df = pd.DataFrame(frame)
  

Causing the
return pd.concat([df, pd.DataFrame(frame)], ignore_index=True)
to FAIL (different cols)
Reindexing only valid with uniquely valued Index objects

[ljilekor]

This is caused by the 'info_dict' key getting overwritten (instead of added)

in screener/overview.py in the _get_table() function

frame = []
for row in rows:
    cols = row.findAll("td")[1:]
    info_dict = {}
    for i, col in enumerate(cols):
        # check if the col is number
        if i not in num_col_index:
            info_dict[table_header[i]] = col.text
        else:
            info_dict[table_header[i]] = number_covert(col.text)
    frame.append(info_dict)

table_header[i] has 'Dividend' twice.
So the 'Dividend' key is overwritten...

Ambiguity stuff... Quite an annoying bug to fix.
The quick and simple way would be

• to rename the 2nd 'Dividend' to 'Dividend Yield' in the COLUMNS of custom.py (or the 1st, don't know by heart)
• to handle table_header[i]=='Dividend' as a special case.

but this is just a suggestion.
Maybe there are other implications...
Never easy

[lit26]

Will be fixed in #91