Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduced raw-media-types config option #1349

Merged
merged 4 commits into from
Jul 22, 2019
Merged

Introduced raw-media-types config option #1349

merged 4 commits into from
Jul 22, 2019

Conversation

dan-amoroso
Copy link
Contributor

@dan-amoroso dan-amoroso commented Jul 11, 2019

Should solve #1336 and #1077

It seems functional but needs tests, for now I have only been able to extract rawContentTypes to Config.hs from Types.hs

It was used as a global value in App.hs so I had to make minor changes to a few helper functions.

Posting the WIP here to get feedback on the code while I work on the tests suggested by @steve-chavez

@@ -168,6 +169,7 @@ readOptions = do
<*> (maybe (Right [JSPKey "role"]) parseRoleClaimKey <$> optValue "role-claim-key")
<*> (maybe ["public"] splitExtraSearchPath <$> optValue "db-extra-search-path")
<*> ((\x y -> QualifiedIdentifier x <$> y) <$> dbSchema <*> optString "root-spec")
<*> (fmap encodeUtf8 <$> C.required "raw-output-media-type" (C.list C.string))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made it a required config key to make the behavior as explicit as possible, but I am not sure whats the preferred approach

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It definitely should not be required. Empty list [] should be default.

Also, I think text/plain should remain included since we always have had support for raw output for a text type and users have been using application/octet-stream for getting this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Do you mean text/plain and application/octet-stream should be included by default even if the config variable is left empty?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Dansvidania Yes, correct.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@steve-chavez ok sounds good, thanks.

@@ -6,3 +6,4 @@ server-host = "127.0.0.1"
server-port = 49421

app.settings.external_api_secret = "0123456789abcdef"
raw-output-media-types=["image/jpeg","image/png"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New feature should not affect other tests configs. One dedicated config should be added.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should probably have one new config with text/html added to the list. We already have a test for that type somewhere, and that one should pass.

@@ -81,6 +81,8 @@ _baseCfg = -- Connection Settings
[]
-- No root spec override
Nothing
-- Raw output media types
["text/html"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What we should do is to have another config(smth like raw-media-types.config) that includes this setting, rest of the tests should run with raw-media-types of [].

Then you should move this test https://github.com/PostgREST/postgrest/blob/master/test/Feature/RpcSpec.hs#L467-L482 to its own Spec and have this spec use the previous config.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@steve-chavez yep, I am working on it at the moment. I am hoping to push this before end of day.

@steve-chavez
Copy link
Member

Really important is that you add a couple of tests for the Accept header that chrome/firefox sends and make sure the result is json. That way we make sure we don't have a regression.

I mentioned that chrome sends:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3

In that case pgrst would pick the */* and send json accordingly.

If you have other browsers default Accept headers it would be great to test them as well.

  introduced implicit raw output media type independent from config variable

  added test for behaviors resulting from settings raw-output-media-types
spec :: SpecWith Application
spec = describe "When raw-output-media-types config variable is missing or left empty" $ do
let firefoxAcceptHdrs = acceptHdrs "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
chromeAcceptHdrs = acceptHdrs "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice! This is what I also had in mind.

One thing though, I think it'd be better to test an endpoint such as /items?id=lte.3 and make sure we get the JSON payload. Root / follows some different rules.

To be extra sure we could also test a GET on rpc, an example /rpc/get_projects_below?id=3.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I'll be "afk" till the 21st but will try to fix them asap.

Thanks for the feedback :)

@@ -273,6 +281,9 @@ readOptions = do
|## stored proc that overrides the root "/" spec
|## it must be inside the db-schema
|# root-spec = "stored_proc_name"
|
|## content types to produce raw output
|#raw-output-media-type=["image/png","image/jpg"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, a nitpick. The config should be plural, end in media-types since it accepts a list.

Also, I think the name I proposed is too long. Ultimately we could shorten it to just raw-media-types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@steve-chavez makes sense. Will fix asap.

  fixed tests in RawOuputTypesSpec to check GETting a record and a RPC
@dan-amoroso dan-amoroso changed the title Introduced raw-output-media-types config option Introduced raw-media-types config option Jul 22, 2019
import SpecHelper (acceptHdrs)

spec :: SpecWith Application
spec = describe "When raw-output-media-types config variable is missing or left empty" $ do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raw-output-media-types should be raw-media-types here.

import SpecHelper (acceptHdrs)

spec :: SpecWith Application
spec = describe "When raw-output-media-types is set to \"text/html\"" $
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto. s/raw-output-media-types/raw-media-types

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@steve-chavez ah, sorry, missed those occurrencies. Fixed them now.

test/SpecHelper.hs Outdated Show resolved Hide resolved
@steve-chavez
Copy link
Member

steve-chavez commented Jul 22, 2019

Just realized that text/html should be gone from inside the codebase. So these references to CTTextHtml should be deleted:

data ContentType = CTApplicationJSON | CTSingularJSON
| CTTextCSV | CTTextPlain | CTTextHtml

toMime CTTextHtml = "text/html"

"text/html" -> CTTextHtml

  fixed code formatting on HtmlRawOutputSpec.hs
  removed unused function from SpecHelper.hs
@dan-amoroso
Copy link
Contributor Author

@steve-chavez thanks again for the feedback. All points should be addressed.

@steve-chavez
Copy link
Member

@Dansvidania Great work on this PR. I'm going to merge it now.

@steve-chavez steve-chavez merged commit f5cef20 into PostgREST:master Jul 22, 2019
@steve-chavez steve-chavez mentioned this pull request Mar 16, 2020
monacoremo pushed a commit to monacoremo/postgrest that referenced this pull request Jul 17, 2021
* extracted rawOutputTypes to config variable raw-output-media-types

* removed CTTextHtml from Types.hs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants