Repository navigation
templating using Request.Body jsonpath will replace characters with HTML character encodings #963
Description
Description of the bug
When templating is enabled for JSON request/response, if the request body is:
{ "name": "O'Reilly" }
And the response body is set to:
{ "name": "{{ Request.Body 'jsonpath' '$.name' }}"
The Hoverfly will return:
{ "name": "O@apos;Reilly" }
Steps to reproduce the issue
Import attached Hoverfly simulation file hoverfly-templating-example.json, and set Hoverfly to simulate.
Templated POST response is:
{
"id": {{ randomIntegerRange 100 199 }},
"name": "{{ Request.Body 'jsonpath' '$.name' }}",
"username": "{{ Request.Body 'jsonpath' '$.username' }}",
"email": "{{ Request.Body 'jsonpath' '$.email' }}",
"address": {
"street": "{{ Request.Body 'jsonpath' '$.address.street' }}",
"suite": "{{ Request.Body 'jsonpath' '$.address.suite' }}",
"city": "{{ Request.Body 'jsonpath' '$.address.city' }}",
"zipcode": "{{ Request.Body 'jsonpath' '$.address.zipcode' }}",
"geo": {
"lat": "{{ Request.Body 'jsonpath' '$.address.geo.lat' }}",
"lng": "{{ Request.Body 'jsonpath' '$.address.geo.lng' }}"
}
},
"phone": "{{ Request.Body 'jsonpath' '$.phone' }}",
"website": "{{ Request.Body 'jsonpath' '$.website' }}",
"company": {
"name": "{{ Request.Body 'jsonpath' '$.company.name' }}",
"catchPhrase": "{{ Request.Body 'jsonpath' '$.company.catchPhrase' }}",
"bs": "{{ Request.Body 'jsonpath' '$.company.bs' }}"
}
}
Execute the non-proxied POST:
curl --request POST --data @post-data.json --header "Content-type: application/json" --header "Accept: application/json" http://jsonplaceholder.typicode.com/users
post-data.json contains the following:
{
"name": "Stephanie Hackett DVM",
"username": "kiara.langworth",
"email": "hayley.ferry@yahoo.com",
"address": {
"street": "3027 O'hara Lane",
"suite": "Penthouse",
"city": "Johnstonmouth",
"zipcode": "3341",
"geo": {
"lat": "14.511382",
"lng": "46.748701"
}
},
"phone": "0449-415-927",
"website": "www.hamilton-ohara.net",
"company": {
"name": "O'connell-Tillman",
"catchPhrase": "Synergized tangible parallelism",
"bs": "facilitate killer bandwidth"
}
}Which will return:
{
"name": "Stephanie Hackett DVM",
"username": "kiara.langworth",
"email": "hayley.ferry@yahoo.com",
"address": {
"street": "3027 O'hara Lane",
"suite": "Penthouse",
"city": "Johnstonmouth",
"zipcode": "3341",
"geo": {
"lat": "14.511382",
"lng": "46.748701"
}
},
"phone": "0449-415-927",
"website": "www.hamilton-ohara.net",
"company": {
"name": "O'connell-Tillman",
"catchPhrase": "Synergized tangible parallelism",
"bs": "facilitate killer bandwidth"
},
"id": 11
}
Execute the proxied POST:
curl --proxy http://localhost:8500 --request POST --data @post-data.json --header "Content-type: application/json" --header "Accept: application/json" http://jsonplaceholder.typicode.com/users
Which will return:
{
"id": 192,
"name": "Stephanie Hackett DVM",
"username": "kiara.langworth",
"email": "hayley.ferry@yahoo.com",
"address": {
"street": "3027 O'hara Lane",
"suite": "Penthouse",
"city": "Johnstonmouth",
"zipcode": "3341",
"geo": {
"lat": "14.511382",
"lng": "46.748701"
}
},
"phone": "0449-415-927",
"website": "www.hamilton-ohara.net",
"company": {
"name": "O'connell-Tillman",
"catchPhrase": "Synergized tangible parallelism",
"bs": "facilitate killer bandwidth"
}
}
The address.street and company.name values contain @apos; instead of ' (single quote).
Observed result
{{ Request.Body 'jsonpath' '$.address.street' }} returns 3027 O'hara Lane
{{ Request.Body 'jsonpath' '$.company.name' }} returns O'connell-Tillman
Expected result
{{ Request.Body 'jsonpath' '$.address.street' }} returns 3027 O'hara Lane
{{ Request.Body 'jsonpath' '$.company.name' }} returns O'connell-Tillman
Attached files.zip contains:
$ hoverctl version
+----------+-------------+
| hoverctl | master-3593 |
| hoverfly | v1.3.1 |
+----------+-------------+
- hoverfly-templating-example.json - example simulation file
- post-data.json - curl POST data
Activity
[-]templating using Request.Body jsonpath will replace characters ith HTML character encodings[/-][+]templating using Request.Body jsonpath will replace characters with HTML character encodings[/+]tommysitu commentedon Feb 20, 2021
We use this library for templating: https://github.com/aymerick/raymond
According to the doc, if you need to unescaped value, you can triple the mustache
{{{🧔if you try this, it should work:
{ "name": "{{{ Request.Body 'jsonpath' '$.name' }}}"MartynDavis commentedon Feb 23, 2021
Thank you @tommysitu for the solution.
Triple moustache it is.
I have attached
hoverfly-templating-example-triple-moustache.jsonwhich is the original JSON file with the triple moustache solution.With that said, Hoverfly does needs address the issues with escaping JSON when
jsonpathis used within templating. Consider the case where double quotes and/or newlines are used within the request JSON. Using triple moustache in this instance does not address the issue as malformed JSON is returned.Consider the file
post-data-bad.json, which contains:The
suiteproperty contains double quote andcatchPhrasecontains newline.When the Hoverfly simulates the POST response, malformed JSON is returned:
Data returned is:
The double quote characters within the
suiteproperty are not escaped, and the newline within thecatchPhraseproperty is not encoded as '\n'.hoverfly-templating-example-triple-moustache.zip
post-data-bad.zip