Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add feedback component #793

Merged
merged 56 commits into from
Jul 18, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
0cab918
🚧 Setup baseline Feedback component
Etesam913 Jun 13, 2022
1ffe2b0
🎨 Applied final styling to Feedback component
Etesam913 Jun 13, 2022
8b9858c
✨ Added width prop to Feedback component
Etesam913 Jun 13, 2022
d230cf3
✨ Added feedback component to example task
Etesam913 Jun 13, 2022
f8fb47b
🚧 First iteration of review_feedback_for_task.py
Etesam913 Jun 13, 2022
2a06017
🐛 Fixed bug where feedback was not set as reviewed
Etesam913 Jun 13, 2022
6b73014
🔨 Updated hydra config file for tips-example
Etesam913 Jun 13, 2022
d1d2a1d
🥅 Added error handling to feedback
Etesam913 Jun 13, 2022
902f72c
🥅 Added size error handling for tips
Etesam913 Jun 13, 2022
e99fb79
🐛 Fixed layout issues with feedback error message
Etesam913 Jun 14, 2022
68ca650
✨ Added ability to set maxTextLengths
Etesam913 Jun 14, 2022
9082b6e
✨ Added toxicity property to feedback
Etesam913 Jun 14, 2022
7f310be
✨ Added ability to filter feedback by toxicity
Etesam913 Jun 14, 2022
21e71f3
🔀 Merged add-feedback-component with main
Etesam913 Jun 16, 2022
9a21622
🐛 Fixed bug with some strings not being recognized
Etesam913 Jun 19, 2022
30b3b62
🐛 Fixed bug with feedback not being a str
Etesam913 Jun 22, 2022
5765e2a
🔀 Merged with add-tips-example
Etesam913 Jun 27, 2022
0f0179c
🔀 Merged with add-cypress-to-static-task
Etesam913 Jun 27, 2022
14439ea
🔀 Merged with add-tips-example
Etesam913 Jun 27, 2022
6468487
✏️ Changed classnames of feedback component
Etesam913 Jun 27, 2022
5aa4b8d
🚧 Began work on adding questions prop
Etesam913 Jun 27, 2022
f309aca
🎨 Made touchups to questions in feedback
Etesam913 Jun 27, 2022
2731651
🏗 Continued work on adding questions to feedback
Etesam913 Jun 28, 2022
0026923
🐛 Updated disable condition for submit button
Etesam913 Jun 28, 2022
0311bc9
🎨 Reworked createFeedback() for it to handle question answers
Etesam913 Jun 28, 2022
5cb173a
✨ Added ability to filter by question when reviewing
Etesam913 Jun 29, 2022
e17a7aa
↩️ Reverted weird changes to server.js test file
Etesam913 Jun 29, 2022
57c5b47
📝 Added documentation on questions prop
Etesam913 Jun 29, 2022
390c3df
🔀 Merged with add-tips-example
Etesam913 Jun 29, 2022
2afa28c
🎨 Added red border to Tips input during error
Etesam913 Jun 29, 2022
e4bdcae
🔀 Merged with add-tips-example
Etesam913 Jun 30, 2022
2af4cd6
🎨 Added background container to feedback
Etesam913 Jun 30, 2022
7e0e052
Merge branch 'add-tips-example' of https://github.com/facebookresearc…
Etesam913 Jun 30, 2022
2e87e7b
🐛 Fixed bug with headerLength being miscalculated
Etesam913 Jul 1, 2022
94f4a18
🔀 Merged with add-tips-example
Etesam913 Jul 2, 2022
ba619cf
✨ Added text to show that feedback is optional
Etesam913 Jul 2, 2022
81327fa
🎨 Improved general styling of feedback
Etesam913 Jul 3, 2022
dda6398
🔀 Merged with add-tips-example
Etesam913 Jul 3, 2022
22b1f78
Merge branch 'add-tips-example' of https://github.com/facebookresearc…
Etesam913 Jul 3, 2022
1a420d8
🐛 Fixed bug with maxLengths not defined
Etesam913 Jul 3, 2022
34b2786
🚨 Fixed some warnings
Etesam913 Jul 3, 2022
8d5dcab
🚛 Moved feedback code from clientIO to agent
Etesam913 Jul 4, 2022
9626e08
✨ Added feedback to _AgentStateMetadata
Etesam913 Jul 8, 2022
7a3f499
Merge branch 'add-tips-example' of https://github.com/facebookresearc…
Etesam913 Jul 9, 2022
e5bc840
Merge branch 'add-tips-example' of https://github.com/facebookresearc…
Etesam913 Jul 10, 2022
3932d3c
🐛 Made FeedbackTextArea disabled when loading
Etesam913 Jul 10, 2022
ba79b8f
🔀 Merged with add-tips-example
Etesam913 Jul 12, 2022
f1a1aa7
🔀 Merged with main
Etesam913 Jul 14, 2022
2d22537
🐛 Fixed bug with feedback submission
Etesam913 Jul 14, 2022
5c9f0d8
Merge branch 'version-bump' of https://github.com/facebookresearch/Me…
Etesam913 Jul 15, 2022
ed39799
➕ Added detoxify to pyproject.toml
Etesam913 Jul 15, 2022
8732bd5
➕ Adds worker_addons.dependencies section to pyproject
Etesam913 Jul 15, 2022
a09ce88
🔀 Merged with main
Etesam913 Jul 17, 2022
4c284b9
🎨 Feedback can now be submitted without detoxify
Etesam913 Jul 17, 2022
4ccb26c
Merge branch 'main' of https://github.com/facebookresearch/Mephisto i…
Etesam913 Jul 18, 2022
8feb698
🎨 Non-headless version of feedback in shadow-dom
Etesam913 Jul 18, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3,056 changes: 110 additions & 2,946 deletions examples/static_react_task/webapp/package-lock.json

Large diffs are not rendered by default.

28 changes: 22 additions & 6 deletions examples/static_react_task_with_tips/webapp/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
*/

import React from "react";
import { Tips } from "mephisto-worker-addons";
import { Tips, Feedback } from "mephisto-worker-addons";

function OnboardingComponent({ onSubmit }) {
return (
Expand Down Expand Up @@ -75,9 +75,25 @@ function SimpleFrontend({ taskData, isOnboarding, onSubmit, onError }) {
</button>
</div>
</div>
</div>
<div style={{ margin: "15rem 2rem 2rem auto", width: "fit-content" }}>
<Tips headless width="30rem" maxHeight="30rem" />

<div style={{ marginTop: "15rem", width: "fit-content" }}>
<Tips
width="30rem"
maxHeight="30rem"
placement="top-start"
maxHeaderLength={50}
maxTextLength={100}
/>
<div style={{ margin: "1rem 0 0", width: "28rem" }}>
<Feedback
maxTextLength={200}
questions={[
"What is your favorite part of this task?",
"Were you satisfied with this task?",
]}
/>
</div>
</div>
</div>
</section>
</div>
Expand Down
7 changes: 7 additions & 0 deletions mephisto/abstractions/_subcomponents/agent_state.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ class _AgentStateMetadata:
task_start: Optional[float] = None
task_end: Optional[float] = None
tips: Optional[List[Dict[str, Any]]] = None
feedback: Optional[List[Dict[str, Any]]] = None


# TODO(#567) File manipulations should ultimately be handled by the MephistoDB, rather than
Expand Down Expand Up @@ -275,6 +276,12 @@ def get_tips(self) -> Optional[List[Dict[str, Any]]]:
"""
return self.metadata.tips

def get_feedback(self) -> Optional[List[Dict[str, Any]]]:
"""
Return the tips for this task, if it is available
"""
return self.metadata.feedback

def update_metadata(self, property_name: str, property_value: Any) -> None:
if self.metadata is not None:
assert (
Expand Down
44 changes: 39 additions & 5 deletions mephisto/data_model/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,13 @@

from typing import Optional, Mapping, Dict, Any, cast, TYPE_CHECKING

try:
from detoxify import Detoxify

DETOXIFY_INSTALLED = True
except ImportError:
DETOXIFY_INSTALLED = False

if TYPE_CHECKING:
from mephisto.data_model.unit import Unit
from mephisto.data_model.assignment import Assignment
Expand Down Expand Up @@ -256,6 +263,7 @@ def handle_submit(self, submit_data: Dict[str, Any]) -> None:

def handle_metadata_submit(self, data: Dict[str, Any]) -> None:
"""Handles the submission of metadata (as of now that is tips and feedback)"""

if "tips" in data:
"""Handles the submission of a tip"""
assert (
Expand All @@ -266,25 +274,51 @@ def handle_metadata_submit(self, data: Dict[str, Any]) -> None:
new_tip_header = data["tips"]["header"]
new_tip_text = data["tips"]["text"]
copy_of_tips = None
item_to_add = {
tip_to_add = {
"id": str(uuid4()),
"header": new_tip_header,
"text": new_tip_text,
"accepted": False,
}
if self.state.metadata.tips is None:
self.state.update_metadata(
property_name="tips", property_value=[item_to_add]
property_name="tips", property_value=[tip_to_add]
)
else:
copy_of_tips = self.state.metadata.tips.copy()
copy_of_tips.append(item_to_add)
copy_of_tips.append(tip_to_add)
self.state.update_metadata(
property_name="tips", property_value=copy_of_tips
)

if "feedback" in data:
print("Feedback received")
elif "feedback" in data:
questions_and_answers = data["feedback"]["data"]
for question_obj in questions_and_answers:
new_feedback_text = question_obj["text"]
new_feedback_toxicity = (
Detoxify("original").predict(new_feedback_text)["toxicity"]
if DETOXIFY_INSTALLED == True
else None
)
feedback_to_add = {
"id": str(uuid4()),
"question": question_obj["question"],
"text": new_feedback_text,
"reviewed": False,
"toxicity": None
if new_feedback_toxicity is None
else str(new_feedback_toxicity),
}
if self.state.metadata.feedback is None:
self.state.update_metadata(
property_name="feedback", property_value=[feedback_to_add]
)
else:
copy_of_feedback = self.state.metadata.feedback.copy()
copy_of_feedback.append(feedback_to_add)
self.state.update_metadata(
property_name="feedback", property_value=copy_of_feedback
)

def shutdown(self) -> None:
"""
Expand Down
Loading