Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve parametric noise generator #7

Closed
dnerini opened this issue Jul 25, 2018 · 1 comment
Closed

Improve parametric noise generator #7

dnerini opened this issue Jul 25, 2018 · 1 comment
Labels
enhancement New feature or request good first issue Good for newcomers medium priority

Comments

@dnerini
Copy link
Member

dnerini commented Jul 25, 2018

Currently the parametric noise generator method initialize_param_2d_fft_filter() only uses one spectral slope beta:

def initialize_param_2d_fft_filter(X, **kwargs):
"""Takes a 2d input field and produces a fourier filter by using the Fast
Fourier Transform (FFT).
Parameters
----------
X : array-like
Two-dimensional square array containing the input field. All values are
required to be finite.
Optional kwargs
---------------
win_type : string
Optional tapering function to be applied to X.
Default : flat-hanning
model : string
The parametric model to be used to fit the power spectrum of X.
Default : power-law
weighted : bool
Whether or not to apply the sqrt(power) as weight in the polyfit() function.
Default : True
Returns
-------
F : array-like
A two-dimensional array containing the parametric filter.
It can be passed to generate_noise_2d_fft_filter().
"""
if len(X.shape) != 2:
raise ValueError("the input is not two-dimensional array")
if np.any(~np.isfinite(X)):
raise ValueError("X contains non-finite values")
if X.shape[0] != X.shape[1]:
raise ValueError("a square array expected, but the shape of X is (%d,%d)" % \
(X.shape[0], X.shape[1]))
# defaults
win_type = kwargs.get('win_type', 'flat-hanning')
model = kwargs.get('model', 'power-law')
weighted = kwargs.get('weighted', True)
L = X.shape[0]
X = X.copy()
if win_type is not None:
X -= X.min()
tapering = build_2D_tapering_function((L, L), win_type)
else:
tapering = np.ones_like(X)
if model.lower() == 'power-law':
# compute radially averaged PSD
psd = _rapsd(X*tapering)
# wavenumbers
if L % 2 == 0:
wn = np.arange(0, int(L/2)+1)
else:
wn = np.arange(0, int(L/2))
# compute spectral slope Beta
if weighted:
p0 = np.polyfit(np.log(wn[1:]), np.log(psd[1:]), 1, w=np.sqrt(psd[1:]))
else:
p0 = np.polyfit(np.log(wn[1:]), np.log(psd[1:]), 1)
beta = -p0[0]
# compute 2d filter
if L % 2 == 1:
XC,YC = np.ogrid[-int(L/2):int(L/2)+1, -int(L/2):int(L/2)+1]
else:
XC,YC = np.ogrid[-int(L/2):int(L/2), -int(L/2):int(L/2)]
R = np.sqrt(XC*XC + YC*YC)
R = fft.fftshift(R)
F = R**(-beta)
F[~np.isfinite(F)] = 1
else:
raise ValueError("unknown parametric model %s" % model)
return F

State of the art methods usually include two spectral slopes and a scale break that need to be fit to the 1d power spectrum.

@dnerini dnerini added enhancement New feature or request medium priority good first issue Good for newcomers labels Jul 25, 2018
@dnerini
Copy link
Member Author

dnerini commented Jul 27, 2018

First implementation in b1a9bcb, but still needs more testing and some better selection of initial and boundary conditions for the fitting.

@dnerini dnerini closed this as completed Mar 13, 2019
sidekock added a commit that referenced this issue Nov 29, 2024
sidekock added a commit that referenced this issue Feb 3, 2025
* Refactored all names in the steps blending code from old to new

* Made some name changes but test still do not pass

* Fixed naming changes, now the tests pass

* Built the rough scaffolding for the blending class

* Refactored untill no rain case

* Added code to estimation of ar parameters of radar

* Next go, start with forecast loop #7

* Added some uniformity between nowcast and blending steps. Now at # 8.4 for the refactoring

* Small changes since prev commit

* All code is tranfered. Last part of the main loop needs to be refactored

* Everything is refactored, no test ran as of yet

* Old forecast function is updated to fit newly refactored code

* Removed old code which is no longer used

* 6 more tests that fail

* All tests pass, still need to fix TODOs

* Updated gitignore

* Cleanup of params and state dataclasses, next step: better typing

* Cleanup of params and state dataclasses, now all tests pass

* Added correct typing to all parts of params and state

* Ready for pull request

* Made changes for Codacy review

* Added aditional tests which currently fail in master branch

* Update .gitignore

Co-authored-by: mats-knmi <145579783+mats-knmi@users.noreply.github.com>

* Used the __zero_precip_time in __zero_precipitation_forecast()

* Changed typing hints to python 3.10+ version

* Added comments back to the State dataclass

* Changed the self.__state.velocity_perturbations = [] to self.__params.velocity_perturbations = [] in __initialize_random_generators

* Added code changes as suggested by Ruben, comments and documentation to come later

* Added frozen functionality to dataclasses, removed reset_state and fixed seed assingments

* Added frozen dataclass to nowcast

* The needed checks are done for this TODO so it can be removed

* Use the seed in all rng in blending code (#449)

* Use seed for all rng to make a test run completely deterministic

* fix probmatching test and some copy paste oversights

* Add test for vel_pert_method

* Change the test so that it actually runs the lines that need to be covered

* Removed deepcopy of worker_state. The state is now accessable to all workers at the same time

* Update to probmatching comments to keep in track with main

* Fix for multithreading issue, this produces exactly the same results as the master

* Added additional documentation

* Bump version

* Updates some files that do not pass the new black version

* Updated examples to work with new black version

---------

Co-authored-by: mats-knmi <145579783+mats-knmi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers medium priority
Projects
None yet
Development

No branches or pull requests

1 participant