Using .add_tokens

[MichaelHirn]

If I'm correct you are using the description of an entity (or relationship) and tokenize that description. The entities and relationships do not have their own tokens, right?

Did you try to learn an embedding for an entity/relationship or does that not really make any sense?

[yao8839836]

@MichaelHirn

Yes, I am only using tokens in the description of an entity or a relation.

The embedding for an entity/relationship can be the average of their description token embeddings (hidden state of BERT), but in the knowledge graph completion tasks, the embeddings are not necessary.

[MichaelHirn]

Ahh I see, thanks for your comment.

I'm wondering if you have tried creating a new token (optionally initialising the token with a related embedding or even with the average of the token embeddings) and learning an embedding for that new token. I'm trying to adapt your approach for our use-case where using the description of an entity does not work and and am curious if you have tried that approach and if so how it worked out.

[yao8839836]

@MichaelHirn

I didn't try this.