Error in stats.kendalls_tau as Keras Metric #1417
Comments
gonzalesMK
changed the title
|
You can wrap kendalls_tau with tf.py_function: def tf_kendalls_tau(y_true, y_pred):
kt = tf.py_function(
kendalls_tau,
(y_true, y_pred),
tf.float32
)
return kt
model.compile(optimizer="Adam", loss="mse", metrics=tf_kendalls_tau)The error is caused by calling the TensorFlow graph version of the function. When you call model.fit(), AutoGraph converts the kendall_tau function into a TensorFlow graph. We can use tf.py_function to prevent this, which allows us to represent kendall_tau in the graph using Python constructs. While this works, I don't think this issue is resolved. Calling the AutoGraph-converted version of tfp.stats.kendalls_tau throws this error. I'm going to look into why this is the case. Does someone with more experience already see an issue here? The AutoGraph-converted version of tfp.stats.kendalls_tau should be callable, right? Full code that worked for me: import tensorflow_probability as tfp
import tensorflow as tf
import numpy as np
def kendalls_tau(y_true, y_pred):
a = tf.reshape(y_true, shape=(-1,))
b = tf.reshape(y_pred, shape=(-1,))
kendall = tfp.stats.kendalls_tau(a, b)
return kendall
def tf_kendalls_tau(y_true, y_pred):
kt = tf.py_function(
kendalls_tau,
(y_true, y_pred),
tf.float32
)
return kt
inputs = tf.keras.layers.Input(shape=(3,))
outputs = tf.keras.layers.Dense(2)(inputs)
model = tf.keras.models.Model(inputs=inputs, outputs=outputs)
model.compile(optimizer="Adam", loss="mse", metrics=tf_kendalls_tau)
x = np.random.random((2, 3))
y = np.random.randint(0, 2, (2, 2) )
model.fit(x, y) |
|
I've started looking at this, but the stats function is not suitable to use as a keras Metric, for a number of reasons. You probably want to use the approximate version which is O(n) instead and can be found in tensorflow addons, as that was intended for this use case. the tfp version expects two [n] tensors and as far as I know keras models cannot output scalars. |
|
I've looked into this and I think there's a few issues, but fundamentally I'm not certain the original proposed use case makes much sense - if using the py_function shim you can use the scipy version of Kendall's Tau - but even then I'm not certain it is doing what one would want as the scipy version return nan for lists of length 1. It's possible to remove the assertions and make the tfp kendall's tau behave more like the scipy one, but still working on it. |
|
Thank you for following up. I'd love to help out on this. Let me know if there's anything I can do that would be valuable. |
|
I've created #1455 which changes behavior to more closely match scipy's implementation. I don't know if this addresses all of the issues raised here but wanted to share it in case it helps. |
|
I think this is fixed. |
I am trying to use TensorFlow probability as a metric in Keras. With respect to kendalls_tau, I get the following error:
How can I fix this?
The text was updated successfully, but these errors were encountered: