Computing residuals when scaling is logarithmic

[IndrajeetPatil]

log(x) is ln(x) in R, right?
Seems wrong to me, I would expect log10(predicted)-log10(observed) and not ln(predicted)-ln(observed)???

Originally posted by @Yuri05 in #1085 (comment)

Referring to the code here:

OSPSuite-R/R/utilities-data-combined.R

Lines 234 to 238 in 1252a50

	if (scaling %in% c(tlf::Scaling$lin, "identity")) {
	pairedData <- dplyr::mutate(pairedData, residualValues = predictedValues - yValues)
	} else {
	pairedData <- dplyr::mutate(pairedData, residualValues = log(predictedValues) - log(yValues))
	}

[IndrajeetPatil]

This was done to stay faithful to what is done in the reporting engine:

https://github.com/Open-Systems-Pharmacology/OSPSuite.ReportingEngine/blob/3a3aa9ac469643a0a50db34ca4b4aafdeae71250/R/utilities-goodness-of-fit.R#L189-L194

But, yes, you are correct that log(x) in R is log(x, exp(1)), i.e. ln(x). Therefore, the correct way to do this would be using log10().

[Yuri05]

@sfrechen @PavelBal Is calculating of logarithmic residuals using ln (and not log10) in RE correct?

[sfrechen]

Natural logarithm is correct. The residuals should follow then a lognormal distribution.

[PavelBal]

Then the PK-Sim implementation is not correct?

[sfrechen]

Let's say: uncommon. And not directly comparable to a lognormal distribution.

[Yuri05]

Let's say: uncommon. And not directly comparable to a lognormal distribution.

Then I wonder, why in the "Histogram of residuals" in RE we plot normal distribution all the time??? @sfrechen

[sfrechen]

Then I wonder, why in the "Histogram of residuals" in RE we plot normal distribution all the time???

Well, if the assumed error model is logarithmic, i.e. epsilon follows log normal distribution (which is usually the case), then plotting the logarithmized residuals against a normal distribution is correct (because log(residual) follows then a normal distribution while residual follows a log normal distribution)

If the assumed error model is linear, i.e. epsilon follows normal distribution, then plotting the "raw" residuals against normal distribution is also correct.

Whether the residuals are logarithmized or not should depend on the underlying assumption of the error model!

[Yuri05]

a) We don't plot log(residual) in the histogram. We plot residual.
b) Actually, the residuals can be negative. How can they be lognormal distributed then???

[sfrechen]

a) We don't plot log(residual) in the histogram. We plot residual.

Hm. As said: Whether the residuals are logarithmized or not should depend on the underlying assumption of the error model, i.e. if the parameter identification is done in log scale for an output, then the residuals should be calculated as res = log(obs)-log(sim).

b) Actually, the residuals can be negative. How can they be lognormal distributed then???

Sorry, I was not accurate:

If the assumed error model is logarithmic, i.e. log(y)=log(f)+eps and epsilon follows a normal distribution, thus exp(epsilon) follows log normal distribution, then plotting the logarithmized residuals against a normal distribution is correct.

[PavelBal]

Do we have an agreement?