[Bugfix] Fix nomic max_model_len

[noooop]

because we did not use the nomic context extension method.

For nomic-ai/nomic-embed-text-v1, input length greater than 2048 will result nan,
For nomic-embed-text-v2-moe the length is set to 512 by sentence_bert_config.json.

context extension

offline

from vllm import LLM

rope_theta = 1000
factor = 4.0
original_max_position_embeddings = 2048

# Use yarn to extend context
hf_overrides = {
    "rope_theta": rope_theta,
    "rope_scaling": {
        "rope_type": "yarn",
        "factor": factor,
        "original_max_position_embeddings": original_max_position_embeddings
    },
    "max_model_len": int(original_max_position_embeddings * factor)
}

llm = LLM(model="nomic-ai/nomic-embed-text-v1",
          trust_remote_code=True,
          task="embed",
          hf_overrides=hf_overrides)

online

vllm serve nomic-ai/nomic-embed-text-v1 --trust-remote-code --hf-overrides '{"rope_theta": 1000, "rope_scaling": {"rope_type": "yarn", "factor": 4.0, "original_max_position_embeddings": 2048}, "model_max_length": 8192}'

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[noooop]

@DarkLight1337

I have written two examples of context_extension, but I don't know where the documentation should be placed.

[noooop]

@DarkLight1337

This is already the best I could do with the code and tests, this piece of code looks very funny.

[noooop]

Is it possible to merge this fix before the v0.9.0 release?

[noooop]

@DarkLight1337

how about now

[DarkLight1337]

See if the test passes now

[noooop]

Thanks for reviewing

[noooop]

FAILED models/language/pooling/test_gte.py::test_models_mteb[model_info9] - AssertionError

should be fixed by #18747

In [1]: import pytest

In [2]: MTEB_EMBED_TOL = 1e-4

In [3]: vllm_main_score = 0.7583957427443586

In [5]: st_main_score = 0.758473459018872

In [6]: st_main_score == pytest.approx(vllm_main_score, abs=MTEB_EMBED_TOL)
Out[6]: True

In [7]: st_main_score == pytest.approx(vllm_main_score, rel=MTEB_EMBED_TOL)
Out[7]: False