Strange Aesara warning and broken pipe error when fitting model with missing values #5339
Comments
fonnesbeck
changed the title
|
@fonnesbeck Can you still reproduce the original problem? |
|
Coincidentally, I had a work colleague report this issue earlier today. This was using 4.1.7 on Python 3.10. Only saw it briefly over a Zoom call, but appears to be the exact same issue, and also resolved by running a single chain. His model was a very simple, non-hierarchical univariate linear regression. |
|
I'm able to reproduce this locally with this model import numpy as np
import pymc as pm
import arviz as az
# %%
def build_model():
data = np.loadtxt(
pm.get_data("efron-morris-75-data.tsv"), delimiter="\t", skiprows=1, usecols=(2, 3)
)
atbats = pm.floatX(data[:, 0])
hits = pm.floatX(data[:, 1])
N = len(hits)
with pm.Model() as model:
phi = pm.Uniform("phi", lower=0.0, upper=1.0)
pareto_dist = pm.Pareto.dist(alpha=1.0001, m=1.5)
kappa = pm.Bound("kappa", pareto_dist, lower=1.0)
thetas = pm.Beta("thetas", alpha=phi * kappa, beta=(1.0 - phi) * kappa, shape=N)
ys = pm.Binomial("ys", n=atbats, p=thetas, observed=hits)
return model
# %%
model = build_model()
with model:
trace = pm.sample(1000, target_accept=0.99)which yields |
|
I was experiencing the same issue on my Macbook Pro (M1 Pro Chip) regardless of the model. I changed the context from "forkserver" to "fork" in the following code portion in parallel_sampling.py. It sometimes(?) works when set to "spawn" I no longer get the broken pipe error with cores set > 1. I'm not sure why it works or whether "fork" or "spawn" is better. re: the issue that the code references. I don't know if it is still necessary to have the context set as forkserver, as when I tested with fork/spawn I had seaborn & matplotlib imported prior. Versions and main components
|
@fonnesbeck I've been using fork for the past few days and haven't had any crashes when importing/using matplotlib or seaborn and sampling works fine now with multiple cores set. |
|
OK, feel free to create a PR if you like. Otherwise I can. |
|
Created a PR (#6218). It was my first, so hopefully I did everything right. |
|
We should split the warning message into a separate issue |
|
FYI I have also been getting this error on an M1 Mac. Some custom aesara stuff in the model, and also some discrete variables with MH sampling, but no missing values (that I know about). PyMC Version: 4.2.2 |
|
This works locally for me (on Ubuntu). @fonnesbeck can you check if the changes in #6218 fixed the problem on your end? |
|
I tried this model, and I don't see any Aeppl warning, so maybe it has solved itself? import pymc as pm
import numpy as np
y = np.zeros((20, 10))
y[0, 5] = np.nan
y[1, 3] = np.nan
age = np.arange(10)
coords = {
'pitcher': range(20),
'age': age,
}
with pm.Model(coords=coords) as age_model:
z_mu = pm.Normal('z_mu', mu=0, sigma=1, dims='pitcher')
s_mu = pm.HalfCauchy('s_mu', 3)
m_mu = pm.Normal('m_mu', mu=92, sigma=5)
mu = pm.Deterministic('mu', m_mu + s_mu * z_mu)
# GRW
rho = pm.Normal('rho', mu=0, sigma=np.append(0.001, np.ones(len(age)-1)))
age_curve = pm.Deterministic('age_curve', rho.cumsum(), dims='age')
theta = mu.dimshuffle(0, 'x') + age_curve
sigma = pm.HalfNormal('sigma', 10)
velo = pm.Normal('velo', theta, sigma=sigma, observed=y)
pm.sample() |
|
Appears to be fixed. |
I'm fitting a manually-constructed GRW model in v4beta that includes a likelihood with a lot of missing values. As the model is fit using
sample, I get several of these errors.The model is below; the node it is referring to is the likelihood standard deviation. If missing value is filled, the model runs normally.
When run, the model quickly dies with a broken pipe, apparently related to multiprocessing:
Running with a single chain resolves the error (but not the warning)
Versions and main components
The text was updated successfully, but these errors were encountered: