Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Demo #264

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open

Demo #264

wants to merge 16 commits into from

Conversation

XuhuiZhou
Copy link
Member

Closes #

πŸ“‘ Description

βœ… Checks

  • My pull request adheres to the code style of this project
  • My code requires changes to the documentation
  • I have updated the documentation as required
  • All the tests have passed
  • Branch name follows type/descript (e.g. feature/add-llm-agents)
  • Ready for code review

β„Ή Additional Information

XuhuiZhou and others added 2 commits December 4, 2024 20:58
* api doc

* add PUT

* add an temp example for websocket

* websocket

* update readme

* Update README.md

* update websocket live simulation api doc

* [autofix.ci] apply automated fixes

* update websocket doc

* add api server with websocket as well as a client

* fix mypy errors

* support stopping the chat

* add 404 to the status code

* fix mypy issue

* update the returned message types

* redesign websocket api

* update websocket, fix mypy error

* add example of using websocket

* clean code & change to existing functions for simulation

* fix typing mismatch

* update doc & mypy type fix

* add type check for run_async_server

* move example

---------

Co-authored-by: Hao Zhu <prokilchu@gmail.com>
Co-authored-by: Zhe Su <360307598@qq.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* add customizable evaluation dimensions

* add docs

* fix mypy error & refactor examples

* add docs for evaluation dimensions

* update docs and examples

* add test cases and fix mypy issue

* fix mypy issue

* Fix test_create_custom_dimension to use CustomEvaluationDimension.get(pk) (#262)

Co-authored-by: openhands <openhands@all-hands.dev>

* Fix/custom eval dimension test (#263)

* Fix test_create_custom_dimension to use CustomEvaluationDimension.get(pk)

* Update documentation for SotopiaDimension and EvaluationDimensionBuilder

* [autofix.ci] apply automated fixes

* Add API documentation for evaluation dimensions

* Refine API documentation for evaluation_dimensions.py to match style

* [autofix.ci] apply automated fixes

---------

Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* add doc

---------

Co-authored-by: XuhuiZhou <zhouxuhui2018@gmail.com>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Copy link

codecov bot commented Dec 8, 2024

Codecov Report

Attention: Patch coverage is 4.09449% with 1218 lines in your changes missing coverage. Please review.

Project coverage is 25.91%. Comparing base (61f190e) to head (302835a).

Files with missing lines Patch % Lines
sotopia/api/fastapi_server.py 0.00% 329 Missing ⚠️
ui/pages/render_chat_websocket.py 0.00% 183 Missing ⚠️
ui/rendering/render_elements.py 0.00% 93 Missing ⚠️
ui/pages/add_evaluation_dimension.py 0.00% 82 Missing ⚠️
sotopia/api/websocket_utils.py 0.00% 71 Missing ⚠️
ui/rendering/render_utils.py 0.00% 69 Missing ⚠️
sotopia/server.py 6.06% 62 Missing ⚠️
ui/app.py 0.00% 47 Missing ⚠️
sotopia/database/evaluation_dimensions.py 40.90% 39 Missing ⚠️
ui/pages/add_scenarios.py 0.00% 38 Missing ⚠️
... and 11 more

❗ There is a different number of reports uploaded between BASE (61f190e) and HEAD (302835a). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (61f190e) HEAD (302835a)
3 2
@@             Coverage Diff             @@
##             main     #264       +/-   ##
===========================================
- Coverage   74.47%   25.91%   -48.57%     
===========================================
  Files          61       66        +5     
  Lines        3162     3616      +454     
===========================================
- Hits         2355      937     -1418     
- Misses        807     2679     +1872     
Files with missing lines Coverage Ξ”
sotopia/api/__init__.py 0.00% <ΓΈ> (ΓΈ)
sotopia/database/__init__.py 95.45% <100.00%> (-4.55%) ⬇️
sotopia/database/logs.py 54.54% <100.00%> (-39.34%) ⬇️
sotopia/database/persistent_profile.py 90.76% <100.00%> (+1.11%) ⬆️
sotopia/database/serialization.py 44.68% <0.00%> (-53.20%) ⬇️
sotopia/envs/parallel.py 21.19% <0.00%> (-56.53%) ⬇️
ui/rendering/__init__.py 0.00% <0.00%> (ΓΈ)
ui/pages/intro.py 0.00% <0.00%> (ΓΈ)
ui/pages/display_characters.py 0.00% <0.00%> (ΓΈ)
ui/pages/display_evaluation_dimensions.py 0.00% <0.00%> (ΓΈ)
... and 15 more

... and 25 files with indirect coverage changes

XuhuiZhou and others added 14 commits December 11, 2024 11:19
…lationshio (#265)

* temp run

* add relationship api

* fix mypy error

* update relationship api

* simulate episode non-streaming

* modify sim episodes

* add simulation status

* task error

* add background task

* [autofix.ci] apply automated fixes

* back to arun one episode

* upload the code

* use rq to execute background tasks

* temp sol

---------

Co-authored-by: Hao Zhu <prokilchu@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* initial framework

* initial conv

* fix module error

* feat: Add 3 new features to Moderator (#266)

* feat:introduce booting procedure, saving, and ending chat to moderator

* fix: moderator will now ignore none AgentAction, Observations now don't have to include all channels in the mapping

* merge changes of example into the original one

* fix: 1. save() method now accepts push_to_db config 2. booting()'s waiting time is changed to 0.1 sec

* fix: rewrite booting() so that different agent will receive different background information

* fix: moderator now inherits from Node directly, instead of from BaseAgent

---------

Co-authored-by: JXZhou <JXZhou>

* add save condition for moderator

* push to db false

* to fully stop

* stopping all agents

* fix mypy

* fix mypy error

---------

Co-authored-by: JXZhou <156194797+JXZhou0224@users.noreply.github.com>
* prototype for modal serving

* add openai secret

* fix type annotation

* add doc

* bug fix for simulation api

* add customize model, evaluator model and evaluation dimensions

* Implement modal API server with Redis integration and FastAPI setup

- Added a new script for the modal API server that initializes a Redis instance.
- Created a persistent volume for Redis data and included a function to download initial data if not present.
- Configured a Docker image with necessary dependencies including Redis Stack and FastAPI.
- Implemented a web API class that sets up and cleans up the Redis connection, ensuring readiness before serving requests.
- Integrated the SotopiaFastAPI application within the modal framework.

---------

Co-authored-by: XuhuiZhou <zhouxuhui2018@gmail.com>
* initial

* initial ui

* merge main

* add new ui

* switch to fastAPI

* websocket check

* fix render episode error

* add  page; make a simplified page and still WIP

* [autofix.ci] apply automated fixes

* fix simplified streaming version

* semi-done character page + avatar assets

* Fixed character card styling

* [autofix.ci] apply automated fixes

* unified rendering and chat display

* updated chat character icons

* add some tags

* add typing

* temp fix

* add characters avatar to simulation

* fix episode full avatar

* go to modal config

* clean up code

* add modal streamlit app

* clean codebase except websocket

* remove repeated local css

* clean websocket

* fix get name error

* fix errors

* pre render scenario

* add custom eval

* change streamlit to dynamic path

* new uv

* revert to previous install commands

* a fix for modal

* add customized dimension

* [autofix.ci] apply automated fixes

* sort scenarios in simulation

* for demo video

* update deploy instruction

* update intro page

* update intro page

* [autofix.ci] apply automated fixes

* update intro page

* add customized dimensions

* update api link and modal environment

* move folder

* fix relative import

* update modal image build

* use uv to build environment

* change folder name

* change test

* fix modal serve

* environment change

* refactor

* fix ui

---------

Co-authored-by: Zhe Su <360307598@qq.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: astrophie <sophie.w.feng@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants