Issue refreshing Wix OAuth tokens - cannot send payload as JSON using refresh_token_request compliance hook

The crux of my question below is: Am I correct that there is currently no way to use the refresh_token_request to send a JSON payload to the token request endpoint?

To explain the issue I am having, Wix's OAuth flow requires access token requests and refreshes to be made using (non-standard) JSON request bodies, instead of the standard x-www-urlformencoded body. Documentation for reference:

Access token request: https://dev.wix.com/docs/rest/app-management/oauth-2/request-an-access-token
Access token refresh: https://dev.wix.com/docs/rest/app-management/oauth-2/refresh-an-access-token

I am able to get the initial access token using the access_token_request compliance hook, using a hook very similar to the one described in the PR that introduced the it:

def redacted_company_compliance_fix(session):
    def _request_in_json_not_form_urlencoded(url, headers, request_kwargs):
        headers["Content-Type"] = "application/json"

        if "redirect_uri" in request_kwargs["data"]:
            del request_kwargs["data"]["redirect_uri"]

        request_kwargs["json"] = request_kwargs["data"]

        del request_kwargs["data"]

        return url, headers, request_kwargs

    session.register_compliance_hook("access_token_request", _request_in_json_not_form_urlencoded)
    return session

However, I am not able to figure out a way to use the refresh_token_request compliance hook in a similar way to send a JSON payload. The refresh logic is much more tightly coupled to sending the request payload as x-www-form-urlencoded, sending only explicit kwargs into the self.post method, including an expectation that the body returned from the refresh_token_request hook is a urlencoded string. Here is the code in question:

for hook in self.compliance_hook["refresh_token_request"]:
    log.debug("Invoking refresh_token_request hook %s.", hook)
    token_url, headers, body = hook(token_url, headers, body)

r = self.post(
    token_url,
    data=dict(urldecode(body)),
    auth=auth,
    timeout=timeout,
    headers=headers,
    verify=verify,
    withhold_token=True,
    proxies=proxies,
)

So:

Am I correct that there is no way to manipulate the request here to have it send JSON instead of an x-www-form-urlencoded body?
If so, I am happy to put up a PR, but could use guidance on how to update the refresh_token method and refresh_token_request hook to allow this in a non-breaking way. Would something like this be a good start, allowing non-x-www-form-urlencoded strings to be returned from the refresh_token_request hook?

for hook in self.compliance_hook["refresh_token_request"]:
    log.debug("Invoking refresh_token_request hook %s.", hook)
    token_url, headers, body = hook(token_url, headers, body)

try:
  # attempt to turn body into dictionary
  body = dict(urldecode(body))
except ValueError:
  # allow non-x-www-form-urlencoded bodies to be passed as strings
  pass 

r = self.post(
    token_url,
    data=body,
    auth=auth,
    timeout=timeout,
    headers=headers,
    verify=verify,
    withhold_token=True,
    proxies=proxies,
)

I went ahead and opened a PR with my above suggestion just to prove it out as a possibility, but I'm happy to take this any direction needed based on feedback.

@skray

@skray I see I'm not the only one that ran into something like this!

TL;DR: I was misled by the POST on a sandbox web page regarding scopes and urlencoded data vs. json in the body. Leaving this here for posterity.

For the API I'm testing against, they have a sandbox instance, and allow you to get an auth token there to test against the APIs. Looking at that request in the browser, it took the list of scopes requested and concatenated them with '+'s. so, requesting two different scopes, scopeType_A/scopeBranch_A.scope_A and scopeType_B/scopeBranch_B.scope_B would send the request in the body like so:

{
	"grant_type": "client_credentials",
	"scope": "scopeType_A/scopeBranch_A.scope_A+scopeType_B/scopeBranch_B.scope_B"
}

And the header includes Content-Type: application/json. This returned a valid token.

However, when sending it via the requests_oauthlib library, the body was url encoded as:

grant_type=client_credentials&scope=scopeType_A%2FscopeBranch_A.scope_A%2BscopeType_B%2FscopeBranch_B.scope_B

and the API I was working against was returning a 401 that the scope was invalid.

While digging into it, I thought it was simply that the urlencode-ing of the data was what was going wrong, but it turns out that when its sent in this form, it expects spaces instead of plusses.

While digging into this, I found that the code path for parameters ends up in common.py in oauthlib, line 233, which automatically encodes the parameters. Trying to figure out if this was requested before, I found an issue on the oauthlib project asking about the same thing, with the project re-filing it under the requests_oauthlib lib. Lukasa responded that the API is mandated to support application/x-www-form-urlencoded by the OAuth2 specification.

Most of that is not terribly relevant to this exact issue, but I wanted to put this somewhere that it might come up in Google searches, and you had a detailed-enough issue that it helped me track this down.

@zach-flaglerhealth

Thanks @zach-flaglerhealth.

To clarify for this issue with Wix, they are very explicitly rejecting x-www-form-urlencoded bodies with an HTTP 415 response code and a body of

{
    "errorCode":409,
    "errorDescription":"[business][RECOVERABLE][Hoopoe Core] o.s.w.HttpMediaTypeNotSupportedException - Content type 'application/x-www-form-urlencoded;charset=UTF-8' not supported",
    "success":false,
    "payload":null
}

Hi,
I have the same issue with an obscure OpenAPI endpoint, that requires JSON for token refresh (and creation). So if one tells it to use 'application/json' through the headers using a compliance hook, it seems to work. But it doesn't, because it adds the extra stuff in kwargs (included the required client ID and secret), and it ends up with broken JSON in the body.

This is, in fact very much mentioned here: #451

And either I'll need to make refresh_token hooks work in the way they do for fetch_token, or I'll have to refresh manually.

In fetch_token, it's done that way: body = prepare_request_body() => dict(urldecode(body)) => hooks => request with data compiled from request_kwargs and that allows for headers and body changes so we can slip JSON in, and manipulate data so we have control over what is required for.. compliance. And no extra data there.

But in refresh_token, it's not done the same way: body = prepare_request_body() => hooks => request(data=dict(urldecode(body)), **kwargs). So we need to have a urldecodable body (great...), AND request gets kwargs, passing through.. json and what not. Which breaks stuff.

Not sure how I'll work around that, but that discrepancy between those 2 hooks / approach is quite annoying :(

Navigation Menu

Repository navigation

Issue refreshing Wix OAuth tokens - cannot send payload as JSON using `refresh_token_request` compliance hook #544

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue refreshing Wix OAuth tokens - cannot send payload as JSON using refresh_token_request compliance hook #544

Description

Activity

skray commented on Apr 19, 2024

zach-flaglerhealth commented on Apr 25, 2024

skray commented on Apr 25, 2024

gilou commented on Sep 20, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Issue refreshing Wix OAuth tokens - cannot send payload as JSON using `refresh_token_request` compliance hook #544