Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filters for htmlEscape and htmlUnescape #1061

Merged
merged 4 commits into from
Dec 7, 2022
Merged

Add filters for htmlEscape and htmlUnescape #1061

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7289930
Add grammar to support htmlEscape and htmlUnescape
ad8lmondy Dec 5, 2022
db486f9
Add htmlEscape and htmlUnescape filters
ad8lmondy Dec 5, 2022
e6028f3
Add support for htmlEscape and htmlUnescape in hurlfmt
ad8lmondy Dec 5, 2022
43487b0
Add integration tests for htmlEscape and htmlUnescape filters
ad8lmondy Dec 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion contrib/vim/syntax/hurl.vim
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ syntax keyword operator == != > >= < <= not
syntax keyword query status url header cookie body jsonpath xpath regex variable duration sha256 md5 bytes
syntax keyword predicate startsWith endsWith matches exists includes isInteger isFloat isBoolean isString isCollection
syntax match predicate "contains"
syntax keyword filter count regex urlEncode urlDecode
syntax keyword filter count regex urlEncode urlDecode htmlEscape htmlUnescape
syntax match escapeNumberSign "\\#"
syntax match escapeQuote "\\\""
syntax region string start='"' end='"' contains=escapeQuote
Expand Down
5 changes: 4 additions & 1 deletion docs/spec/hurl.grammar
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -428,8 +428,11 @@ url-encode-filter: "urlEncode"

url-decode-filter: "urlDecode"

to-int: "toInt"
html-encode-filter: "htmlEscape"

html-decode-filter: "htmlUnescape"

to-int: "toInt"

# Lexical Grammar

Expand Down
7 changes: 7 additions & 0 deletions integration/tests_ok/filter.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,27 @@
<span class="line"><span class="version">HTTP</span> <span class="number">200</span></span>
<span class="line section-header">[Captures]</span>
<span class="line"><span class="name">url</span><span>:</span> <span class="query-type">jsonpath</span> <span class="string">"$.url"</span></span>
<span class="line"><span class="name">html</span><span>:</span> <span class="query-type">jsonpath</span> <span class="string">"$.html"</span></span>
<span class="line section-header">[Asserts]</span>
<span class="line"><span class="query-type">jsonpath</span> <span class="string">"$.list"</span> <span class="filter-type">count</span> <span class="predicate-type">==</span> <span class="number">3</span></span>
<span class="line"><span class="query-type">jsonpath</span> <span class="string">"$.message"</span> <span class="filter-type">regex</span> <span class="regex">/Hello (.*)!/</span> <span class="predicate-type">==</span> <span class="string">"Bob"</span></span>
<span class="line"><span class="query-type">jsonpath</span> <span class="string">"$.url"</span> <span class="predicate-type">==</span> <span class="string">"https://mozilla.org/?x=шеллы"</span></span>
<span class="line"><span class="query-type">jsonpath</span> <span class="string">"$.url"</span> <span class="filter-type">urlEncode</span> <span class="predicate-type">==</span> <span class="string">"https%3A//mozilla.org/%3Fx%3D%D1%88%D0%B5%D0%BB%D0%BB%D1%8B"</span></span>
<span class="line"><span class="query-type">jsonpath</span> <span class="string">"$.encoded_url"</span> <span class="filter-type">urlDecode</span> <span class="predicate-type">==</span> <span class="string">"https://mozilla.org/?x=шеллы"</span></span>
<span class="line"><span class="query-type">variable</span> <span class="string">"url"</span> <span class="filter-type">urlEncode</span> <span class="filter-type">urlDecode</span> <span class="predicate-type">==</span> <span class="string">"{{url}}"</span></span>
<span class="line"><span class="query-type">jsonpath</span> <span class="string">"$.html"</span> <span class="predicate-type">==</span> <span class="string">"a &gt; b &amp;&amp; a &lt; c"</span></span>
<span class="line"><span class="query-type">jsonpath</span> <span class="string">"$.html"</span> <span class="filter-type">htmlEscape</span> <span class="predicate-type">==</span> <span class="string">"a &amp;gt; b &amp;amp;&amp;amp; a &amp;lt; c"</span></span>
<span class="line"><span class="query-type">jsonpath</span> <span class="string">"$.encoded_html"</span> <span class="filter-type">htmlUnescape</span> <span class="predicate-type">==</span> <span class="string">"a &gt; b &amp;&amp; a &lt; c"</span></span>
<span class="line"><span class="query-type">variable</span> <span class="string">"html"</span> <span class="filter-type">htmlEscape</span> <span class="filter-type">htmlUnescape</span> <span class="predicate-type">==</span> <span class="string">"{{html}}"</span></span>
<span class="line"><span class="query-type">jsonpath</span> <span class="string">"$.id"</span> <span class="filter-type">toInt</span> <span class="predicate-type">==</span> <span class="number">123</span></span>
<span class="line"><span class="query-type">jsonpath</span> <span class="string">"$.score"</span> <span class="filter-type">toInt</span> <span class="predicate-type">==</span> <span class="number">1</span></span>
<span class="json"><span class="line">{</span>
<span class="line"> "list": [1,2,3],</span>
<span class="line"> "message": "Hello Bob!",</span>
<span class="line"> "url": "https://mozilla.org/?x=шеллы",</span>
<span class="line"> "encoded_url": "https://mozilla.org/?x=%D1%88%D0%B5%D0%BB%D0%BB%D1%8B",</span>
<span class="line"> "html": "a &gt; b &amp;&amp; a &lt; c",</span>
<span class="line"> "encoded_html": "a &amp;gt; b &amp;amp;&amp;amp; a &amp;lt; c",</span>
<span class="line"> "id": "123",</span>
<span class="line"> "score": 1.6</span>
<span class="line">}</span></span>
Expand Down
7 changes: 7 additions & 0 deletions integration/tests_ok/filter.hurl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,27 @@ GET http://localhost:8000/filter
HTTP 200
[Captures]
url: jsonpath "$.url"
html: jsonpath "$.html"
[Asserts]
jsonpath "$.list" count == 3
jsonpath "$.message" regex /Hello (.*)!/ == "Bob"
jsonpath "$.url" == "https://mozilla.org/?x=шеллы"
jsonpath "$.url" urlEncode == "https%3A//mozilla.org/%3Fx%3D%D1%88%D0%B5%D0%BB%D0%BB%D1%8B"
jsonpath "$.encoded_url" urlDecode == "https://mozilla.org/?x=шеллы"
variable "url" urlEncode urlDecode == "{{url}}"
jsonpath "$.html" == "a > b && a < c"
jsonpath "$.html" htmlEscape == "a &gt; b &amp;&amp; a &lt; c"
jsonpath "$.encoded_html" htmlUnescape == "a > b && a < c"
variable "html" htmlEscape htmlUnescape == "{{html}}"
jsonpath "$.id" toInt == 123
jsonpath "$.score" toInt == 1
{
"list": [1,2,3],
"message": "Hello Bob!",
"url": "https://mozilla.org/?x=шеллы",
"encoded_url": "https://mozilla.org/?x=%D1%88%D0%B5%D0%BB%D0%BB%D1%8B",
"html": "a > b && a < c",
"encoded_html": "a &gt; b &amp;&amp; a &lt; c",
"id": "123",
"score": 1.6
}
2 changes: 1 addition & 1 deletion integration/tests_ok/filter.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"entries":[{"request":{"method":"GET","url":"http://localhost:8000/filter"},"response":{"status":200,"captures":[{"name":"url","query":{"type":"jsonpath","expr":"$.url"}}],"asserts":[{"query":{"type":"jsonpath","expr":"$.list"},"filters":[{"type":"count"}],"predicate":{"type":"equal","value":3}},{"query":{"type":"jsonpath","expr":"$.message"},"filters":[{"type":"regex","expr":{"type":"regex","value":"Hello (.*)!"}}],"predicate":{"type":"equal","value":"Bob"}},{"query":{"type":"jsonpath","expr":"$.url"},"predicate":{"type":"equal","value":"https://mozilla.org/?x=шеллы"}},{"query":{"type":"jsonpath","expr":"$.url"},"filters":[{"type":"urlEncode"}],"predicate":{"type":"equal","value":"https%3A//mozilla.org/%3Fx%3D%D1%88%D0%B5%D0%BB%D0%BB%D1%8B"}},{"query":{"type":"jsonpath","expr":"$.encoded_url"},"filters":[{"type":"urlDecode"}],"predicate":{"type":"equal","value":"https://mozilla.org/?x=шеллы"}},{"query":{"type":"variable","name":"url"},"filters":[{"type":"urlEncode"},{"type":"urlDecode"}],"predicate":{"type":"equal","value":"{{url}}"}},{"query":{"type":"jsonpath","expr":"$.id"},"filters":[{"type":"toInt"}],"predicate":{"type":"equal","value":123}},{"query":{"type":"jsonpath","expr":"$.score"},"filters":[{"type":"toInt"}],"predicate":{"type":"equal","value":1}}],"body":{"type":"json","value":{"list":[1,2,3],"message":"Hello Bob!","url":"https://mozilla.org/?x=шеллы","encoded_url":"https://mozilla.org/?x=%D1%88%D0%B5%D0%BB%D0%BB%D1%8B","id":"123","score":1.6}}}}]}
{"entries":[{"request":{"method":"GET","url":"http://localhost:8000/filter"},"response":{"status":200,"captures":[{"name":"url","query":{"type":"jsonpath","expr":"$.url"}},{"name":"html","query":{"type":"jsonpath","expr":"$.html"}}],"asserts":[{"query":{"type":"jsonpath","expr":"$.list"},"filters":[{"type":"count"}],"predicate":{"type":"equal","value":3}},{"query":{"type":"jsonpath","expr":"$.message"},"filters":[{"type":"regex","expr":{"type":"regex","value":"Hello (.*)!"}}],"predicate":{"type":"equal","value":"Bob"}},{"query":{"type":"jsonpath","expr":"$.url"},"predicate":{"type":"equal","value":"https://mozilla.org/?x=шеллы"}},{"query":{"type":"jsonpath","expr":"$.url"},"filters":[{"type":"urlEncode"}],"predicate":{"type":"equal","value":"https%3A//mozilla.org/%3Fx%3D%D1%88%D0%B5%D0%BB%D0%BB%D1%8B"}},{"query":{"type":"jsonpath","expr":"$.encoded_url"},"filters":[{"type":"urlDecode"}],"predicate":{"type":"equal","value":"https://mozilla.org/?x=шеллы"}},{"query":{"type":"variable","name":"url"},"filters":[{"type":"urlEncode"},{"type":"urlDecode"}],"predicate":{"type":"equal","value":"{{url}}"}},{"query":{"type":"jsonpath","expr":"$.html"},"predicate":{"type":"equal","value":"a > b && a < c"}},{"query":{"type":"jsonpath","expr":"$.html"},"filters":[{"type":"htmlEscape"}],"predicate":{"type":"equal","value":"a &gt; b &amp;&amp; a &lt; c"}},{"query":{"type":"jsonpath","expr":"$.encoded_html"},"filters":[{"type":"htmlUnescape"}],"predicate":{"type":"equal","value":"a > b && a < c"}},{"query":{"type":"variable","name":"html"},"filters":[{"type":"htmlEscape"},{"type":"htmlUnescape"}],"predicate":{"type":"equal","value":"{{html}}"}},{"query":{"type":"jsonpath","expr":"$.id"},"filters":[{"type":"toInt"}],"predicate":{"type":"equal","value":123}},{"query":{"type":"jsonpath","expr":"$.score"},"filters":[{"type":"toInt"}],"predicate":{"type":"equal","value":1}}],"body":{"type":"json","value":{"list":[1,2,3],"message":"Hello Bob!","url":"https://mozilla.org/?x=шеллы","encoded_url":"https://mozilla.org/?x=%D1%88%D0%B5%D0%BB%D0%BB%D1%8B","html":"a > b && a < c","encoded_html":"a &gt; b &amp;&amp; a &lt; c","id":"123","score":1.6}}}}]}
2 changes: 2 additions & 0 deletions integration/tests_ok/filter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ def filter():
"message": "Hello Bob!",
"url": "https://mozilla.org/?x=шеллы",
"encoded_url": "https://mozilla.org/?x=%D1%88%D0%B5%D0%BB%D0%BB%D1%8B",
"html": "a > b && a < c",
"encoded_html": "a &gt; b &amp;&amp; a &lt; c",
"id": "123",
"score": 1.6
}"""
1 change: 1 addition & 0 deletions packages/hurl/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ float-cmp = "0.9.0"
glob = "0.3.0"
hex = "0.4.3"
hex-literal = "0.3.4"
html-escape = "0.2.12"
hurl_core = { version = "2.0.0-SNAPSHOT", path = "../hurl_core" }
indexmap = "1.9.2"
libflate = "1.2.0"
Expand Down
31 changes: 31 additions & 0 deletions packages/hurl/src/runner/filter.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ fn eval_filter(
FilterValue::Count {} => eval_count(value, &filter.source_info),
FilterValue::UrlEncode { .. } => eval_url_encode(value, &filter.source_info),
FilterValue::UrlDecode { .. } => eval_url_decode(value, &filter.source_info),
FilterValue::HtmlEscape { .. } => eval_html_encode(value, &filter.source_info),
FilterValue::HtmlUnescape { .. } => eval_html_decode(value, &filter.source_info),
FilterValue::ToInt { .. } => eval_to_int(value, &filter.source_info),
}
}
Expand Down Expand Up @@ -153,6 +155,35 @@ fn eval_url_decode(value: &Value, source_info: &SourceInfo) -> Result<Value, Err
}
}

fn eval_html_encode(value: &Value, source_info: &SourceInfo) -> Result<Value, Error> {
match value {
Value::String(value) => {
let mut enco = String::from(value);
let encoded = html_escape::encode_text_to_string(value, &mut enco);
Ok(Value::String(encoded.to_string()))
}
v => Err(Error {
source_info: source_info.clone(),
inner: RunnerError::FilterInvalidInput(v._type()),
assert: false,
}),
}
}

fn eval_html_decode(value: &Value, source_info: &SourceInfo) -> Result<Value, Error> {
match value {
Value::String(value) => {
let decoded = html_escape::decode_html_entities(value).to_string();
Ok(Value::String(decoded))
}
v => Err(Error {
source_info: source_info.clone(),
inner: RunnerError::FilterInvalidInput(v._type()),
assert: false,
}),
}
}

fn eval_to_int(value: &Value, source_info: &SourceInfo) -> Result<Value, Error> {
match value {
Value::Integer(v) => Ok(Value::Integer(*v)),
Expand Down
2 changes: 2 additions & 0 deletions packages/hurl_core/src/ast/core.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -877,5 +877,7 @@ pub enum FilterValue {
},
UrlEncode {},
UrlDecode {},
HtmlEscape {},
HtmlUnescape {},
ToInt {},
}
6 changes: 6 additions & 0 deletions packages/hurl_core/src/format/html.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1083,6 +1083,12 @@ impl Htmlable for FilterValue {
}
FilterValue::UrlEncode {} => "<span class=\"filter-type\">urlEncode</span>".to_string(),
FilterValue::UrlDecode {} => "<span class=\"filter-type\">urlDecode</span>".to_string(),
FilterValue::HtmlEscape {} => {
"<span class=\"filter-type\">htmlEscape</span>".to_string()
}
FilterValue::HtmlUnescape {} => {
"<span class=\"filter-type\">htmlUnescape</span>".to_string()
}
FilterValue::ToInt {} => "<span class=\"filter-type\">toInt</span>".to_string(),
}
}
Expand Down
12 changes: 12 additions & 0 deletions packages/hurl_core/src/parser/filter.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ pub fn filter(reader: &mut Reader) -> ParseResult<'static, Filter> {
regex_filter,
url_encode_filter,
url_decode_filter,
html_encode_filter,
html_decode_filter,
to_int_filter,
],
reader,
Expand Down Expand Up @@ -98,6 +100,16 @@ fn url_decode_filter(reader: &mut Reader) -> ParseResult<'static, FilterValue> {
Ok(FilterValue::UrlDecode {})
}

fn html_encode_filter(reader: &mut Reader) -> ParseResult<'static, FilterValue> {
try_literal("htmlEscape", reader)?;
Ok(FilterValue::HtmlEscape {})
}

fn html_decode_filter(reader: &mut Reader) -> ParseResult<'static, FilterValue> {
try_literal("htmlUnescape", reader)?;
Ok(FilterValue::HtmlUnescape {})
}

fn to_int_filter(reader: &mut Reader) -> ParseResult<'static, FilterValue> {
try_literal("toInt", reader)?;
Ok(FilterValue::ToInt {})
Expand Down
9 changes: 9 additions & 0 deletions packages/hurlfmt/src/format/json.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -535,6 +535,15 @@ impl ToJson for FilterValue {
FilterValue::UrlDecode { .. } => {
attributes.push(("type".to_string(), JValue::String("urlDecode".to_string())));
}
FilterValue::HtmlEscape { .. } => {
attributes.push(("type".to_string(), JValue::String("htmlEscape".to_string())));
}
FilterValue::HtmlUnescape { .. } => {
attributes.push((
"type".to_string(),
JValue::String("htmlUnescape".to_string()),
));
}
FilterValue::ToInt { .. } => {
attributes.push(("type".to_string(), JValue::String("toInt".to_string())));
}
Expand Down
4 changes: 4 additions & 0 deletions packages/hurlfmt/src/format/token.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1152,6 +1152,10 @@ impl Tokenizable for Filter {
FilterValue::Count { .. } => vec![Token::FilterType(String::from("count"))],
FilterValue::UrlEncode { .. } => vec![Token::FilterType(String::from("urlEncode"))],
FilterValue::UrlDecode { .. } => vec![Token::FilterType(String::from("urlDecode"))],
FilterValue::HtmlEscape { .. } => vec![Token::FilterType(String::from("htmlEscape"))],
FilterValue::HtmlUnescape { .. } => {
vec![Token::FilterType(String::from("htmlUnescape"))]
}
FilterValue::ToInt { .. } => vec![Token::FilterType(String::from("toInt"))],
}
}
Expand Down