Should data processing functions have their own modules?

[willschlitzer]

In #685, @weiji14 asked the question of if data processing modules like makecpt and grd2cpt should be split into their own Python scripts, much like the plotting modules will be. My opinion is they should be split up. The other files aren't nearly as big as base_plotting.py, but I think it's best to standardize all functions in this way, and it makes it easier to find the function you're looking for (I wasn't sure if grd2cpt should be under mathops.py or grdops.py?).

I'm happy to do it. I'm assuming the process would be similar to the changes in base_plotting.py, and just involve copying the functions to their own scripts in pygmt.src, but how do we "tell" PyGMT where to look for it beyond the init file, as there isn't a base_plotting-equivalent script to import into?

[michaelgrund]

I completely agree with you @willschlitzer. Splitting up all the modules allows to arrange everything in a consistent way. Would also support you in this intention if you need help.

[seisman]

The other files aren't nearly as big as base_plotting.py, but I think it's best to standardize all functions in this way, and it makes it easier to find the function you're looking for

I agree with you.

how do we "tell" PyGMT where to look for it beyond the init file, as there isn't a base_plotting-equivalent script to import into

I think we need to change these lines in __init__.py:

pygmt/pygmt/__init__.py

Lines 14 to 24 in 2345486

	# Import modules to make the high-level GMT Python API
	from pygmt import datasets
	from pygmt.figure import Figure
	from pygmt.filtering import blockmedian
	from pygmt.gridding import surface
	from pygmt.gridops import grdcut, grdfilter
	from pygmt.mathops import makecpt
	from pygmt.modules import GMTDataArrayAccessor, config, grdinfo, info, which
	from pygmt.sampling import grdtrack
	from pygmt.session_management import begin as _begin
	from pygmt.session_management import end as _end

[willschlitzer]

@seisman So would we completely eliminate the files like mathops.py and gridops.py and just import directly into pygmt/__init__.py with something like from pygmt.src.makecpt import makecpt? If so, how will that affect Sphinx building the documentation?

[seisman]

So would we completely eliminate the files like mathops.py and gridops.py and just import directly into pygmt/__init__.py with something like from pygmt.src.makecpt import makecpt?

Yes to me.

If so, how will that affect Sphinx building the documentation?

I'm not sure. We need to give it a try.

[weiji14]

Do we want to get this in for PyGMT v0.3.0 or the next release (v0.3.1 or v0.4.0)?

[seisman]

I'm OK to both. I think I can finish PR #832 soon.

[weiji14]

Ok, but we should merge #808 before #832. One step at a time ;)

[seisman]

These two PRs won't cause conflicts.

[weiji14]

There might be a minor conflict in the pygmt/src/__init__.py file, but yeah, shouldn't be too hard to resolve hopefully.

[seisman]

Todo list after PR #832:

• config: wrapper of GMT's set command. pygmt/config.py? pygmt/src/config.py? pygmt/src/set.py?
• GMTDataArrayAccessor: pygmt/datatypes.py?
• x2sys_init and x2sys_cross: do we want to put these functions into two files or one file pygmt/src/x2sys.py (Move x2sys.py to src folder and split it into x2sys_init and x2sys_cross #951)

[seisman]

Ping @weiji14 to separate x2sys_init and x2sys_cross.

[weiji14]

Todo list after PR #832:

* [ ]  `config`: wrapper of GMT's `set` command. `pygmt/config.py`? `pygmt/src/config.py`? `pygmt/src/set.py`?
* [ ]  `GMTDataArrayAccessor`: `pygmt/datatypes.py`?

Right, do we want to move the config and GMTDataArrayAccessor classes out of pygmt/modules.py? I wouldn't consider them 'data processing' modules, they're more about plot configuration and metadata related.

[seisman]

We should. I think pygmt.config should be in pygmt/src/config.py, but I'm not sure where to put GMTDataArrayAccessor.

[maxrjones]

We should. I think pygmt.config should be in pygmt/src/config.py, but I'm not sure where to put GMTDataArrayAccessor.

Could something like either pygmt/src/accessors.py or pygmt/src/xarray_extension.py work?

[weiji14]

We should. I think pygmt.config should be in pygmt/src/config.py, but I'm not sure where to put GMTDataArrayAccessor.

Could something like either pygmt/src/accessors.py or pygmt/src/xarray_extension.py work?

I'm ok with moving pygmt.config to pygmt/src/config.py since gmt set is a pure GMT function. But I'm inclined to keep GMTDataArrayAccessor outside of the src folder. Maybe do the following:

1. Move class config from pygmt/modules.py to pygmt/src/config.py
2. Rename pygmt/modules.py to pygmt/accessors.py

[seisman]

• Move class config from pygmt/modules.py to pygmt/src/config.py
• Rename pygmt/modules.py to pygmt/accessors.py

Sounds good.

[maxrjones]

• Move class config from pygmt/modules.py to pygmt/src/config.py
• Rename pygmt/modules.py to pygmt/accessors.py

Sounds good.

I can help with this if needed.