han-transformer not work

[Continue7777]

i have tried p1_HierarchicalAttention_model_transformer.py with your test function,but the loss dose not converge。

[brightmart]

I think you are run on toy task, and loss is decreasing. if you want coverage, you can use real task,

[Continue7777]

I run p1_HierarchicalAttention_model.py ,its work fine .and i just want to try transformer,but i dosen't work,so i try you test function,but it dose not work too?

[brightmart]

got it. transformer need to fix a problem. before that you can use this:

https://github.com/google-research/bert/blob/master/modeling.py

[Continue7777]

thanks

[brightmart]

I think transformer work better now. as someone find a bug which was fixed:
#93

[Continue7777]

hey,that's me . and i just try your transofrmer code.it's work. but i campare to textcnn, just as the same conclusion in your github(https://github.com/brightmart/text_classification). transformer isn't better than textcnn ,may be my dataset is not large enough. But, i think transformer may be more potential. I will try pre-train like bert and find whether it's help.

[brightmart]

good to hear from you. please keep me update.