You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Getting WARNING:mteb.models:Failed to extract metadata from model: 'CrossEncoder' object has no attribute 'model_card_data'. Upgrading to sentence-transformers v3.0.0 or above is recommended. even though I'm using the below
on the following snippet:
frommtebimportMTEBimportmtebfromsentence_transformersimportCrossEncoder, SentenceTransformercross_encoder=CrossEncoder("cross-encoder/ms-marco-TinyBERT-L-2-v2")
dual_encoder=SentenceTransformer("all-MiniLM-L6-v2")
tasks=mteb.get_tasks(tasks=["NFCorpus"], languages=["eng"])
subset="default"# subset name used in the NFCorpus dataseteval_splits= ["test"]
evaluation=MTEB(tasks=tasks)
evaluation.run(
dual_encoder,
eval_splits=eval_splits,
save_predictions=True,
output_folder="results/stage1",
)
evaluation.run(
cross_encoder,
eval_splits=eval_splits,
top_k=5,
save_predictions=True,
output_folder="results/stage2",
previous_results=f"results/stage1/NFCorpus_{subset}_predictions.json",
)
The text was updated successfully, but these errors were encountered:
@isaac-chung Just for future reference, I ended up talking to Tom about this and he said that model_card_data will probably be implemented for CrossEncoders in a couple of months.
#1186 (comment)
on the following snippet:
The text was updated successfully, but these errors were encountered: