Using the CRF layer with some other segmentation network

[gursimarsingh]

Hi,
I wish to use the CRF layer with my own segmentation network which is an encoder-decoder network with skip connections. If I use the crf layer as the last layer what kind of things I should keep in mind?
The output of my model has number of channels equal to number of classes (2 in my case) and activation is sigmoid. I connect this layer directly with the CRF layer without any modification to CRF parameters like theta_alpha, theta_beta and theta_gamma. The final loss function that I am using is binary cross entropy. Output of the CRF is being directly fed to the loss function. At the time of inference I am taking argmax for getting the labels. But the output label for every pixel is mostly the same.
Can anyone please help me to understand that what are the things that are particular to the FCN output such that the CRF layer works with it?

Also, If you could please share the training script?

[lixingang]

Hi,
Is this problem solved? I'm in the similar situation.

[gursimarsingh]

No

[gyheart]

No
Is this problem solved? I'm in the similar problem. @gursimarsingh @lixingang

[JadBatmobile]

i have done this in my own keras network.

[gyheart]

i have done this in my own keras network.

Can you share your code? Thanks.

[jovahe]

I add the crfrnn layer in my model such as unet as following, but I dose not know whether is right.
def add_crfrnn(base_model, cofig):
img_input = base_model.input
x = base_model.get_layer('final_conv').output
x = CrfRnnLayer(image_dims=(config.img_w, config.img_h),
num_classes=config.nb_classes,
theta_alpha=160.,
theta_beta=3.,
theta_gamma=3.,
num_iterations=10,
name='crfrnn')([x, img_input])

x = Activation(config.activation, name=config.activation)(x)
model=Model(img_input, x)
model.summary()
return model

[falconjhc]

i have done this in my own keras network.
Hi, in your work, is CRF trained?

[falconjhc]

I add the crfrnn layer in my model such as unet as following, but I dose not know whether is right.
def add_crfrnn(base_model, cofig):
img_input = base_model.input
x = base_model.get_layer('final_conv').output
x = CrfRnnLayer(image_dims=(config.img_w, config.img_h),
num_classes=config.nb_classes,
theta_alpha=160.,
theta_beta=3.,
theta_gamma=3.,
num_iterations=10,
name='crfrnn')([x, img_input])

x = Activation(config.activation, name=config.activation)(x)
model=Model(img_input, x)
model.summary()
return model

Hi, if it is done in this way, does the CRF be trained?