Search: strip HTML tags while indexing

[pdurbin]

#1320 is a high level ticket about supporting HTML. This ticket is about stripping out those HTML tag while we are indexing fields into Solr.

[pdurbin]

As of fa3a94f I'm using JSoup to strip HTML tags before indexing the following:

• descriptions of dataverses
• dataset fields whose FieldType is "TEXTBOX"

@scolapasta said it's fine that there is no HTML in the cards. For example, if you put a link in a dataverse description, you can click it from the dataverse landing page but not from the card for that dataverse. We do this because we don't want <h1> tags and other weird stuff to be put in the cards.

Passing to QA.

[sbarbosadataverse]

@kcondon --I won't be able to see if this (html tags) is showing up until we migrate correct?

[pdurbin]

@sbarbosadataverse the way you would test this:

• put some HTML in some places (descriptions, etc.)
• make sure the HTML is showing up when you look at the page (you can see bold or links or whatever)
• do a search on words you used in the HTML
• see if you can see any HTML tags when you search (you shouldn't!)

I hope this helps!

[sbarbosadataverse]

@pdurbin
while html tags don't show up on search facets displayed, if i search for "http" I get. If this is expected I will close. Not sure if we can do anything about it

[pdurbin]

@sbarbosadataverse that's what I expect to see. Thanks for the screenshot. Please feel free to get other opinions before closing if you have any doubt, though.

[sbarbosadataverse]

works as designed