Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DD, FFlow, Certainty Docs #277

Closed
wants to merge 848 commits into from
Closed

DD, FFlow, Certainty Docs #277

wants to merge 848 commits into from

Conversation

agshruti12
Copy link
Contributor

Pull Request Template:
If you are merging in a feature or other major change, use this template to check your pull request!

Basic Info

What's this pull request about?
Conceptual documentation for discursive diversity, forward flow, and certainty features.

My PR Adds or Improves Documentation

If your feature is about documentation, ensure that you check the boxes relevant to you.

Docstrings

  • Docstrings: I have followed the proper documentation format (https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html; Google Format recommended).
  • Docstrings: Every function in the file has a block quote comment with a description of the feature.
  • Docstrings: Every input argument is documented.
  • Docstrings: The output type is documented, along with a description of what the output is for.
  • Docstrings: I have linked the feature under the Table of Contents (docs/source/features/index.rst)

Feature Wiki

  • Conceptual Wiki: I made a copy of the TEMPLATE (docs/source/features_conceptual/TEMPLATE.rst)
  • Conceptual Wiki: I replaced the word TEMPLATE at the top of the file with the name of the feature (.. _TEMPLATE:) Please do NOT delete any of the punctuation (the .._ and :) in the header, as this is important for referencing the feature in the Table of Contents!
  • Conceptual Wiki: I have answered the six sections of the template to the best of my ability.
  • Conceptual Wiki: I have linked the feature under the Table of Contents (docs/source/features_conceptual/index.rst).

General Documentation

  • My documentation is linked in a toctree.
  • I have confirmed that make clean and make html do not generate breaking errors.

My PR is About Adding a New Feature to the Code Repository

Adding Feature to the Feature Dictionary

  • I have edited the feature_dictionary.py file with an appropriate entry for my feature. Below is a sample entry; I confirm that all fields are accurately filled out.
  "Function Word Accommodation": {
    "columns": ["function_word_accommodation"],
    "file": "./features/word_mimicry.py",
    "level": "Chat",
    "semantic_grouping": "Variance",
    "description": "The total number of function words used in a given turn that were also used in the previous turn. Function words are defined as a list of 190 words from the source paper.",
    "references": "(Ranganath et al., 2013)",
    "wiki_link": "https://github.com/Watts-Lab/team-process-map/wiki/C.9-Mimicry:-Function-word,-Content-word,-BERT,-Moving",
    "function": ChatLevelFeaturesCalculator.calculate_word_mimicry,
    "dependencies": [],
    "preprocess": [],
    "vect_data": False,
    "bert_sentiment_data": False
  }
  • If my feature is at the chat level, my dictionary entry is in the top half of the file; if my feature is at the conversation level, my dictionary entry is in the bottom half of the file (below the comment that says, ### Conversation Level).

Documentation

Did you document your feature? You should follow the same requirements as above:

  • Docstrings: I have followed the proper documentation format (https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html; Google Format recommended).
  • Docstrings: Every function in the file has a block quote comment with a description of the feature.
  • Docstrings: Every input argument is documented.
  • Docstrings: The output type is documented, along with a description of what the output is for.
  • Docstrings: I have linked the feature under the Table of Contents (docs/source/features/index.rst)

Code Basics

  • My feature is a .py file.
  • My feature uses snake case in the name. That means the name of the format is my_feature, NOT myFeature (camel case).
  • My feature has the name, NAME_features.py, where NAME is the name of my feature.
  • My feature is located in src/features/.

Testing

  • I have thought about test cases for my features, with inputs and expected outputs.
  • I have added test cases for my feature under the tests/ folder.
  • My feature passes the automated testing suite.

The location of my tests are here:

[PASTE LINK HERE]

If you check all the boxes above, then you ready to merge!

xehu and others added 30 commits April 24, 2024 19:09
Bumps [gunicorn](https://github.com/benoitc/gunicorn) from 20.0.4 to 22.0.0.
- [Release notes](https://github.com/benoitc/gunicorn/releases)
- [Commits](benoitc/gunicorn@20.0.4...22.0.0)

---
updated-dependencies:
- dependency-name: gunicorn
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.6.2 to 3.9.4.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](aio-libs/aiohttp@v3.6.2...v3.9.4)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [markdown2](https://github.com/trentm/python-markdown2) from 2.3.8 to 2.4.0.
- [Changelog](https://github.com/trentm/python-markdown2/blob/master/CHANGES.md)
- [Commits](trentm/python-markdown2@2.3.8...2.4.0)

---
updated-dependencies:
- dependency-name: markdown2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [rsa](https://github.com/sybrenstuvel/python-rsa) from 4.0 to 4.7.
- [Changelog](https://github.com/sybrenstuvel/python-rsa/blob/main/CHANGELOG.md)
- [Commits](sybrenstuvel/python-rsa@version-4.0...version-4.7)

---
updated-dependencies:
- dependency-name: rsa
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Change FeatureBuilder Input to Pandas DataFrame
…gunicorn-22.0.0

Bump gunicorn from 20.0.4 to 22.0.0 in /feature_engine
…aiohttp-3.9.4

Bump aiohttp from 3.6.2 to 3.9.4 in /feature_engine
…rsa-4.7

Bump rsa from 4.0 to 4.7 in /feature_engine
…markdown2-2.4.0

Bump markdown2 from 2.3.8 to 2.4.0 in /feature_engine
xehu and others added 24 commits July 9, 2024 13:38
Bumps [zipp](https://github.com/jaraco/zipp) from 3.17.0 to 3.19.1.
- [Release notes](https://github.com/jaraco/zipp/releases)
- [Changelog](https://github.com/jaraco/zipp/blob/main/NEWS.rst)
- [Commits](jaraco/zipp@v3.17.0...v3.19.1)

---
updated-dependencies:
- dependency-name: zipp
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [setuptools](https://github.com/pypa/setuptools) from 68.2.2 to 70.0.0.
- [Release notes](https://github.com/pypa/setuptools/releases)
- [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst)
- [Commits](pypa/setuptools@v68.2.2...v70.0.0)

---
updated-dependencies:
- dependency-name: setuptools
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
…setuptools-70.0.0

Bump setuptools from 68.2.2 to 70.0.0 in /feature_engine
…zipp-3.19.1

Bump zipp from 3.17.0 to 3.19.1 in /feature_engine
* First draft

* emily edits to helena's branch

* Do TODOs

* update read path of features.pkl

* update path

* update feature dict

* fix bugs

* fix bugs

* update test cases to test custom features

* remove features.pkl

* first round of cleanup and breaking out the vector-dependent features

* edit featurebuilder constructor

* update feature dictionary

* update pytest coverage to use the feature dictionary

* update PR template with instructions to fill out feature_dictionary.py

* update PR template

* skips the embeddings steps if unnecessary

* restore featurize.py to testing datasets

---------

Co-authored-by: zhouhelena <helenazhou04@gmail.com>
Co-authored-by: Xinlan Emily Hu <xehu@wharton.upenn.edu>
* file restructuring: rename feature_engine to src and rename testing to tests and move one level up

* begin process of removing dependency on tpm-data

* Closes #230.

* updates to package structure

* Closes #256.
This reverts commit 9e7f78e.
* docstrings updates

* docstrings updates

* docstrings updates

* turn-taking wiki page updates

* word_ttr wiki page updates

* turn_taking wiki updates

* emily edits

---------

Co-authored-by: Xinlan Emily Hu <xehu@wharton.upenn.edu>
* added docstrings for politeness_v2.py

* added docstrings for politeness_v2_helper.py

* added docstrings for reddit_tags.py

* added docstrings for word_mimicry.py

* Update index.rst

politeness_v2, politeness_v2_helper,reddit_tags,word_mimicry

* local checks pass

---------

Co-authored-by: Xinlan Emily Hu <xehu@wharton.upenn.edu>
* allow force regenerate vectors

* updating node_modules

* update .gitignore

* local testing complete

---------

Co-authored-by: Xinlan Emily Hu <xehu@wharton.upenn.edu>
* add pyproject.toml

* Update README.md with team-comm-tools rather than team-process-map

* Update README.md to remove outdated requirement (#264)

* delete junit

* move preprocessing notebooks to tests

* remove unnecessary deps

* update requirements

* more slimming of reqs

* remove packaging related deps

* get rid of requests and xgboost deps

* addressing #267

* edit src paths

* test with python 3.7

* test  python 3.7

* test 3.8

* test 3.8 pt. 2

* test 3.9

* test 3.10

* restore to 3.11; play with docs

* Update README.md with new path names.

* standardize package structure and solve path issues

* update requirement.txt path in workflow

* update workflow file

* updating test workflow

* update test workflow

* update test workflow

* update test workflow

* update test workflow

* update imports in example

* fix bugs

* move lexiconx_dict.pkl to features/assets

* update dependencies

* delete legacy files and remove constant nltk import

* clean up package structure and warnings

* resolve relative imports issue in sphinx

* create single installation script

* commit setup script

* update setup script and documentationZ

* update README to point to setup script

* add linkes to website and Rtd to readme

* disable tokenizer parallelism to avoid error

* add badges to home page

* Named Entity Recognition + Valence Tests (#226)

* valence testing

* rearranging files

* intermediate ner testing

* NER testing

* fix featurizer

* fix featurize bug

* updating test dataset + function

* code coverage

* burstiness

* move testing FB's into run_tests.py

* move NER dataframe to test file

* adding complex tests back to run_tests.py

* add chat_complex_df and conv_complex_df to run_tests.py

* correct dataset paths

* rebase

* changing references as part of rebase

* correcting FB calls based on latest interface updates

* correct run_tests.py

* add dd tests

* burstiness fix

* dd tests add

* forward flow tests

* src changes

* testing timestamp variations

* src changes

* update test ds

* fix formatting

* fix formatting

---------

Co-authored-by: Xinlan Emily Hu <xehu@wharton.upenn.edu>
Co-authored-by: Xinlan Emily Hu <xehu@cs.stanford.edu>

* Amy/website (#270)

* website updates

* renaming tpm-website to website

* deploying via gh-pages

* changed from tpm-website to website

* deployed website

* copyright and team

* team headshots and footer

* edits to the pages

* website updates

* updated links

* updated homepage

* link updates

* mobile compatibility

* mobile adjustments

* navbar mobile updates

* whitespace edits

* homepage updates

* feature table

* website updates

* renaming tpm-website to website

* deploying via gh-pages

* changed from tpm-website to website

* edits to the pages

* website updates

* updated links

* updated homepage

* link updates

* mobile compatibility

* mobile adjustments

* navbar mobile updates

* homepage updates

* add table of features

* updated team page titles

* include flask in requirements.txt

* updates to table of features

* load pages from top

* fix to 404 issues

* moved build under website folder

* add flask back into requirements

---------

Co-authored-by: Xinlan Emily Hu <xehu@cs.stanford.edu>
Co-authored-by: Xinlan Emily Hu <xehu@wharton.upenn.edu>

---------

Co-authored-by: sundy1994 <yuxuanzh@seas.upenn.edu>
Co-authored-by: Shruti Agarwal <46203852+agshruti12@users.noreply.github.com>
Co-authored-by: amytangzheng <145236844+amytangzheng@users.noreply.github.com>
* positive bert docs

* mimicry bert docs

* copied mimicry bert to moving mimicry as template

* moving mimicry docs

* updating function & content word accommodation docs

* updating content word docs

* edits to content word accommodation

* edits to other mimicry pages

* technical docstrings + conceptual fixes

* removing unfinished files

* upgrade some packages and check for requirements conflicts

* update check embeddings to resolve error in which vectors were being generated even when they were not requested

* reset version number

* Priya/docs 2 (#269)

* added docstrings for politeness_v2.py

* added docstrings for politeness_v2_helper.py

* added docstrings for reddit_tags.py

* added docstrings for word_mimicry.py

* Update index.rst

politeness_v2, politeness_v2_helper,reddit_tags,word_mimicry

* documentation

* index updated

* textblob polarity and subjectivity

* textblob polarity and subjectivity

* added feature names in .rst files

* proportion of first person pronouns

* hedges

* dale chall score

* time difference

* positivity z scores

* positivity z scores

* politeness strategies - Convokit

* replaced TEMPLATE with feature name for concecptual features

* implemented suggestions

* implemented suggestions

* reset version number

---------

Co-authored-by: Xinlan Emily Hu <xehu@cs.stanford.edu>
Co-authored-by: sundy1994 <yuxuanzh@seas.upenn.edu>

* update Positivity (BERT) feature name

* update feature dict.

* update documentation and add flash to pyproject

* update readme

* slight changes to readme

* add sponsor info to readme

* fix errors in priya and shruti's documentation and ensure docs build properly

* correct typo in readme

* add readme contribution link

* add github getting started guide

* update conf

* add nltk resources to ensure feature builder works

* add nltk resources to ensure feature builder works

* update positivity related features

* update feature_dict with latest writeups and update examples with new setup

* update feature_dict with latest writeups and update examples with new setup

* update documentation to clarify that nltk needs to be downloaded in python

* small commit to make readthedocs.yml python syntax consistent

* update docs a little more

* change heading for rtd

* fix dependency issues with spacy and nltk

* clarify docs

* update docs to include nltk one-liner

---------

Co-authored-by: agshruti12 <agshruti2901@gmail.com>
Co-authored-by: Priya DCosta <76833529+PriyaDCosta@users.noreply.github.com>
Co-authored-by: sundy1994 <yuxuanzh@seas.upenn.edu>
* remove en_core_web_sm from dependencies, update 'import_nltk.py' to 'download_resources.py' that automatically download resources if not found

* update test workflow

* fix test workflow

* update documentation

* build docs

---------

Co-authored-by: sundy1994 <yuxuanzh@seas.upenn.edu>
@xehu xehu changed the base branch from main to dev August 15, 2024 17:23
xehu and others added 3 commits August 15, 2024 17:37
* delete small test lexicons

* move .pkl files to assets and remove from GH

* filesystem cleanup

* update certainty pickle location

* remove unpickling certainty

* remove lexicons from pyproject

* change lexical pkl path

* add error handling when lexicons are not found

* update warning message

* add legal caveat and update name of certainty pkl to be correct
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants