Regression Loss

[tstibor]

Hi there,
I have a set of images of galaxies and labels are (x_1, x_2) \in [-1,1] which are related
to principle axes of the galaxies, so the learning problem is 2D regression. Can somebody give me hint how implement that in caffe.

Would it be the right way to add in caffe.proto ( message Datum {...} )
an additional field, e.g.
optional float label_float = 7;
and use EuclideanLossLayer<Dtype>?

Thanks,
Thomas

[kloudkl]

Issue #149 "Implement MultiLabel losses and data input" aims to solve the problem. Another related issue is #144 "Multi label data layer".

[kloudkl]

Luckily, @sergeyk has already implemented a multi-label data layer (#147, #203).

[ChenglongChen]

I have the same question and not sure if I am using the HDF5DataLayer in the right way. Hope someone here can give me some guidance.

In the file generate_sample_data.py:
num_cols = 8
num_rows = 10
height = 5
width = 5
total_size = num_cols * num_rows * height * width

data = np.arange(total_size)
data = data.reshape(num_rows, num_cols, height, width)
data = data.astype('float32')
label = np.arange(num_rows)[:, np.newaxis]
label = label.astype('float32')

So, here we have batch_size = 10, input channel size = 8, and image size of 5 x 5.
Regarding stibor's question, if I understand it correctly, the label in this case should be a 2 x 10 (or 10 x 2) matrix, i.e., a 2-d vector for each 8 x 5 x 5 input and therefore we have in total 10 2-d vector. In code, we should have something similar as
label = np.arange(2_num_rows).reshape(2, num_rows)
or
label = np.arange(2_num_rows).reshape(num_rows, 2)

Am I right?

Then using the EuclideanLossLayer, will finally suit the need of stibor. But I am not sure if EuclideanLossLayer has already supported such 2-D regression task.

[sergeyk]

The second reshape you give looks correct. I'm not sure about the Euclidean
loss layer, sorry. Let us know if it works!

On Tuesday, March 18, 2014, ChenglongChen notifications@github.com wrote:

I have the same question and not sure if I am using the HDF5DataLayer in
the right way. Hope someone here can give me some guidance.
In the file generate_sample_data.py:
num_cols = 8
num_rows = 10
height = 5
width = 5
total_size = num_cols * num_rows * height * width

data = np.arange(total_size)
data = data.reshape(num_rows, num_cols, height, width)
data = data.astype('float32')
label = np.arange(num_rows)[:, np.newaxis]
label = label.astype('float32')

So, here we have batch_size = 10, input channel size = 8, and image size
of 5 x 5.
Regarding stibor's question, if I understand it correctly, the label in
this case should be a 2 x 10 matrix, i.e., a 2-d vector for each 8 x 5 x 5
input and therefore we have in total 10 2-d vector. In code, we should have
something similar as
label = np.arange(2_num_rows).reshape(2, num_rows)
or
label = np.arange(2_num_rows).reshape(num_rows, 2)

Am I right?

Then using the EuclideanLossLayer, will finally suit the need of stibor.
But I am not sure if EuclideanLossLayer has already supported such 2-D
regression task.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/210#issuecomment-37915249
.

[zyan0]

I have tried several models with different parameters for regression with Euclidean loss layer. The output of the final fc layer is no better than the mean of input points.
Has anybody here ever trained a regression model with Caffe successfully?

[thbupt]

@yzheng624 I have the same problem, at each iteration the output of the final fc is almost the same no matter what the input is . I don't know whether you have solved your problem.

[aravindhm]

I had a similar problem initially when I replaced the logistic regression from the imagenet model with a regression loss. This problem, was however, solved when I copied layer 1 and 2 from the reference model instead of training them. Also, when initially the outputs were almost the same, they were becoming equal after layer 3 (not entirely sure whether layer 3 or layer 4).

Note that I was using a leveldb for input and was regressing 25 outputs.

Did you try a shallow model without a normalization layer for your data?

[shelhamer]

See #881 for the latest suggestions.