Skip to content

Function preprocess_text does not seem to strip punctuations #15

Open
@govindgnair23

Description

@govindgnair23
def preprocess_text(text):
    text = ' '.join(word.lower() for word in text.split(" "))
    text = re.sub(r"([.,!?])", r" \1 ", text)
    text = re.sub(r"[^a-zA-Z.,!?]+", r" ", text)
    return text

Calling preprocess_text('Are you a, boy or a girl?') returns:

''are you a , boy or a girl ? "

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions