-
Notifications
You must be signed in to change notification settings - Fork 1.1k
modules are too long and have too many symbols #235
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I agree that it could be better, but I also don't know what would be ideal. Some things I've thought about in the past:
Further point on documentation: a huge number of those lines are thorough documentation. There are many functions with documentation that's twice as long as the code. Some modules might be 50% documentation. We created the style guide so that the api-level symbols would at least be uniform, but there are a lot of symbols in the PV world. I'm all for consolidation if you see an opportunity. |
@mikofski as far as I understand it, the length of a module is just a matter of taste in python. An indicator for a bad structured project is more the length of classes, methods or functions. Restructuring the whole project includes the danger of creating new bugs. I would rather have a look at the biggest "god" classes/functions and split them step by step. What classes/functions do you have in mind? |
Continuing comments from #261 the top level modules can be converted to subpackages which would preserve the current API, but allow more flexibility: current state:
becomes:
Then from pvlib.irradiance.core import extraradiation, aoi_projection, aoi, ...
from pvlib.irradance.decomposition import disc, dirint, liujordan, erbs
from pvlib.irradance.transposition import isotropic, haydavies, perez, klucher, reindl, king
# any other symbols used by all irradiance methods Then break up irradiance into separate modules.
# code for extraradiation, aoi, etc. that was in irradiance.py Now when someone wants to add something new to irradiance, they import it into the |
First, while I don't want to break the API for fun, if it needs to change for the long term health of the library then we should change it. We've broken user code in every 0.x release and I don't intend to stop until we decide on a 1.0. So, don't feel too constrained by the past. Next, when making the new API docs, it felt pretty natural to break up the irradiance and pvsystem sections into subsections. That's probably a clue that the modules themselves should be chopped up too, though you could certainly take issue with some of the function classifications. I didn't make subsections for atmosphere or clear sky, and I don't think I'd support subpackages for them at this point either. I can imagine a solarposition subpackage that contains spa.py, a new ephemeris.py, and the remaining functionality in core.py. modelchain is long and complicated, but it does something complicated that I don't know how to do otherwise. Some of forecast might end up in iotools. Concerning implementation, pandas and some other pydata packages use Finally, we could wait and see if the subpackage approach for io tools actually leads to more people contributing more easily maintained code. |
I like the
The nice thing about using I wrote a post that might be related called recommended Python project layout that has links and some of my observations. |
Anyone in favor of closing this? I think of this more as a long term strategy discussion than a particular bug that needs fixing or a feature to implement. Maybe continue the discussion in google groups or the wiki? |
@mikofski thanks for circling back on this. I agree we should close this. |
Several modules have become quite long which IMO makes them difficult to use and maintain. Obviously modules can be any size and structure, but I think that long files are difficult to navigate, and modules with too many symbols are error prone due to shadowing and namespace collision.
pvlib.pvsystem
pvlib.clearsky
pvlib.atmosphere
pvlib.forecast
pvlib.irradiance
pvlib.location
pvlib.modelchain
pvlib.solarposition
pvlib.spa
pvlib.tmy
pvlib.tools
pvlib.tracking
In particular
pvsystem
,forecast
,irradiance
andspa
are very long and contain a lot of symbols.I don't know what would be ideal, but IMO I think a maximum of 500 lines and 10 symbols is a good baseline.
The SPA algorithm presents a unique challenge, as this code was ported, so maybe for reference sake, it should stay as it is. However, I think that the very large arrays of coefficients in
irradiance
andspa
take a lot of space. These could easily be moved to HDF5 files usingh5py
which would have some memory and speed enhancements, but I'll create a related issue for that.References for large modules and many symbols.
The text was updated successfully, but these errors were encountered: