Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: "Create model from equations" produces/shows wrong models #5805

Open
liunelson opened this issue Dec 10, 2024 · 1 comment
Open

[BUG]: "Create model from equations" produces/shows wrong models #5805

liunelson opened this issue Dec 10, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@liunelson
Copy link
Member

liunelson commented Dec 10, 2024

Describe the bug

Note:

  • in all these cases, the "Edited by AI" equations are all correct.
  • since the rate laws are wrong, it seems that the problem lies around the juncture between good LaTeX and MIRA's template_model_from_sympy_odes
  • the problem seems to be with sympy.parsing.latex.parse_latex(ode_latex_str) producing incorrect SymPy expressions; example
sympy.parsing.latex.parse_latex(r"\frac{d V(t)}{d t} = a b I(t) (1 - g)")
# produces Eq(Derivative(V(t), t), a*(I(t)*b(1 - g))) where parameter 'b' is treated as a function of (1-g)

sympy.parsing.latex.parse_latex(r"\frac{d I(t)}{d t} = k E(t) - d I(t)")
# produces Eq(Derivative(I(t), t), -dI*t + k*E(t)) where parameter 'd' became 'dI'
  • thus, instructing the equation-styling agent to add or retain * in the edited LaTeX as indication of multiplication probably could fix Case 2; currently, the agent is instructed to remove all *
  • for Case 3, it seems that the source of the problem is again parse_latex where it adds unnecessary parentheses
sympy.parsing.latex.parse_latex(r"\frac{d I(t)}{d t} = b * S(t) * I(t) - k * g * I(t) - (1 - k) * g * I(t)")
# produces Eq(Derivative(I(t), t), -g*(1 - k)*I(t) + ((b*S(t))*I(t) - g*k*I(t))) where the second term is wrapped in parentheses that MIRA then interprets as a single template
  • the solution for Case 1, 3 may be to use a LLM to convert the LaTeX to SymPy with instructions to avoid this sort of parenthesis issue

Case 1

Basic SIR model with added birth and death processes (l birth rate and every state dies at a rate of m)

\frac{d S}{d t} = -b S I + l - m S
\frac{d I}{d t} = b S I - g I - m I
\frac{d R}{d t} = g I - m R

Incorrect model diagram and rate laws:

Correct model (with its own, unrelated bugs):

  • should be 6 templates with rate laws I*S*b, l, S*m, I*g, I*m, R*m
  • S is converted to I, I is converted to R, production of S, degradation/death of S, I, R individually
  • model1.json, made using Sympy and MIRA directly
odes = [
    sympy.Eq(S(t).diff(t), - b * S(t) * I(t) + l - m * S(t)),
    sympy.Eq(I(t).diff(t), b * S(t) * I(t) - g * I(t) - m * I(t)),
    sympy.Eq(R(t).diff(t), g * I(t) - m * R(t)),
]
model1 = template_model_from_sympy_odes(odes)

Case 2 - Controlled production

Production of viral particles in wastewater (Phan et al. 2023 from last monthly demo)

\frac{d S}{d t} = -l S I
\frac{d E}{d t} = l S I - k E
\frac{d I}{d t} = k E - d I
\frac{d V}{d t} = a b (1 - g) I

Incorrect model:

Correct model:

odes = [
    sympy.Eq(S(t).diff(t), - l * S(t) * I(t)),
    sympy.Eq(E(t).diff(t), l * S(t) * I(t) - k * E(t)),
    sympy.Eq(I(t).diff(t), k * E(t) - d * I(t)),
    sympy.Eq(V(t).diff(t), a * b * (1 - g) * I(t)),
]
model1b = template_model_from_sympy_odes(odes)

Case 3 - Branching ratio (2)

The I state splits into two streams, one into V and other into R.

\frac{d S}{d t} = -b S I
\frac{d I}{d t} = b S I - k g I - (1 - k) g I
\frac{d R}{d t} = k g I
\frac{d V}{d t} = (1 - k) g I

Incorrect model:

Correct model:

odes = [
    sympy.Eq(S(t).diff(t), - b * S(t) * I(t)),
    sympy.Eq(I(t).diff(t), b * S(t) * I(t) - k * g * I(t) - (1 - k) * g * I(t)),
    sympy.Eq(R(t).diff(t), k * g * I(t)),
    sympy.Eq(V(t).diff(t), (1 - k) * g * I(t))
]
model2a = template_model_from_sympy_odes(odes)
@liunelson liunelson added the bug Something isn't working label Dec 10, 2024
@liunelson liunelson changed the title [BUG]: "Create model from equations" produces wrong models (probably not caused by MIRA) [BUG]: "Create model from equations" produces/shows wrong models (probably not caused by MIRA) Dec 10, 2024
@liunelson liunelson changed the title [BUG]: "Create model from equations" produces/shows wrong models (probably not caused by MIRA) [BUG]: "Create model from equations" produces/shows wrong models Dec 10, 2024
@liunelson
Copy link
Member Author

Now that I think of it, as in the "Note" above, we maybe forced to create a GoLLM task to convert the cleaned LaTeX to SymPy since most of the cases are due to extra () introduced by SymPy's parse_latex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants