Add a regression test for the embedding

[collijk]

Background

There are many open issues related to the maximum tokens allowed to be passed to the embedding. This is a quick regression test to formally document the issue.

There are many related PRs and issues:

• Fix the maximum context length issue by chunking #3222
• Maximum context length exceeded after read_file, ingest_file, search_files #2801
• Maximum context length exceeded after google_search #2871
• Maximum context length exceeded after get_hyperlinks #2906
• Maximum context length exceeded after execute_shell #3244

Changes

Add regression test for attempting to embed a string that's too long.

Documentation

Comments in the test explaining what it's for.

Test Plan

This

PR Quality Checklist

• My pull request is atomic and focuses on a single change.
• I have thoroughly tested my changes with multiple different prompts.
• I have considered potential risks and mitigations for my changes.
• I have documented my changes clearly and comprehensively.
• I have not snuck in any "extra" small tweaks changes

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 27, 2023 6:33pm

[codecov]

Codecov Report

Patch and project coverage have no change.

Comparison is base (427b864) 54.83% compared to head (cbda732) 54.83%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3422   +/-   ##
=======================================
  Coverage   54.83%   54.83%           
=======================================
  Files          65       65           
  Lines        3018     3018           
  Branches      508      508           
=======================================
  Hits         1655     1655           
  Misses       1227     1227           
  Partials      136      136           

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.