Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exception requests.exceptions.JSONDecodeError is raised in APIError constructor #1504

Closed
darttie opened this issue Aug 8, 2024 · 4 comments · Fixed by #1510
Closed

Exception requests.exceptions.JSONDecodeError is raised in APIError constructor #1504

darttie opened this issue Aug 8, 2024 · 4 comments · Fixed by #1510

Comments

@darttie
Copy link

darttie commented Aug 8, 2024

Describe the bug
My code called sh.sheet1.batch_get([gsrange], Dimension.cols) which sends request to google sheets. It looks like some error response was returned, but the following exception was raised when APIError was trying to be constructed:

Traceback (most recent call last):
  File "helper.py", line 37, in get_capabilities_list
    capabilities = sh.sheet1.batch_get([gsrange], Dimension.cols)
  File ".local/lib/python3.10/site-packages/gspread/worksheet.py", line 1096, in batch_get
    response = self.client.values_batch_get(
  File ".local/lib/python3.10/site-packages/gspread/http_client.py", line 255, in values_batch_get
    r = self.request("get", url, params=params)
  File ".local/lib/python3.10/site-packages/gspread/http_client.py", line 128, in request
    raise APIError(response)
  File ".local/lib/python3.10/site-packages/gspread/exceptions.py", line 45, in __init__
    self.error: Mapping[str, Any] = response.json()["error"]
  File ".local/lib/python3.10/site-packages/requests/models.py", line 978, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

To Reproduce
Unfortunately the bug prevents me from knowing what error actually happened. Usually this step succeeds, but once it has failed.

  1. call sh.sheet1.batch_get(['G1:S1'], Dimension.cols)

Expected behavior
Correct APIError is returned

Environment info:

  • Ubuntu
  • Python 3.10.12
  • gspread Version: 6.1.2
@dan-ladd
Copy link

dan-ladd commented Aug 10, 2024 •
edited
Loading

I noticed this intermittently starting on 2024-08-07 around 15:00 UTC. I was getting it on fetch_sheet_metadata()

venv/lib/python3.9/site-packages/gspread/http_client.py", line 303, in fetch_sheet_metadata
->
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

@lavigne958
Copy link
Collaborator

Hi, thank you for raising this issue, I believe this is a problem with Google servers, we receive a valid JSON error message on every error for as many years as I can remember on this project.

possibly it could be a network error too where you don't receive the error response (the connection closes early, etc).

In order to investigate your issue and help further similar situations and possibly provide a solution to #1491 we can change the way we handle errors to:

  1. try to extract HTTP payload as JSON
  2. if it succeed:
  3. we extract the informations as we already do
  4. if it fails:
    1. we build an empty Response object
    2. we get the pure, raw text payload (which could be empty, we don't know)
    3. be set the response and the internal Dict object with response_code=-1 and the error message with whatever text we extracted

This way we don't fail because of weird/empty/malformed JSON response
We still can print/return some error in the same format we do all over the project
We can solve the issue with the mentioned PR above that returns some HTML error message.

@alifeee how do you feel about this ?

lavigne958 added a commit that referenced this issue Sep 8, 2024
@lavigne958
better handler API error parsing.
f6b8e12 
Possibly we encountered an error while parsing the API error JSON, like
connection closes early etc and the received JSON is invalid.

Better handle such scenario and build a valid error Dict to keep the
exception stack flow running and raise this message to the application.

extract the returned text so if we any bit of valid information at least
we can pass that to the application.

This could be helpfull to if we add calls to the API made for web
browser that returns HTML error message, it will fail to decode the
recieved error and raise a proper error.

closes #1504

Signed-off-by: Alexandre Lavigne <[email protected]>
lavigne958 added a commit that referenced this issue Sep 8, 2024
@lavigne958
better handler API error parsing.
db59084 
Possibly we encountered an error while parsing the API error JSON, like
connection closes early etc and the received JSON is invalid.

Better handle such scenario and build a valid error Dict to keep the
exception stack flow running and raise this message to the application.

extract the returned text so if we any bit of valid information at least
we can pass that to the application.

This could be helpfull to if we add calls to the API made for web
browser that returns HTML error message, it will fail to decode the
recieved error and raise a proper error.

closes #1504

Signed-off-by: Alexandre Lavigne <[email protected]>
@lavigne958 lavigne958 mentioned this issue Sep 8, 2024
Merged
@alifeee alifeee pinned this issue Sep 25, 2024
@alifeee
Copy link
Collaborator

alifeee commented Sep 25, 2024

this has been fixed with #1510 and the fix will be included in the next release (which should be in the coming time, see #1516)

alifeee pushed a commit that referenced this issue Oct 3, 2024
@lavigne958 @alifeee
better handler API error parsing.
48efa5b 
Possibly we encountered an error while parsing the API error JSON, like
connection closes early etc and the received JSON is invalid.

Better handle such scenario and build a valid error Dict to keep the
exception stack flow running and raise this message to the application.

extract the returned text so if we any bit of valid information at least
we can pass that to the application.

This could be helpfull to if we add calls to the API made for web
browser that returns HTML error message, it will fail to decode the
recieved error and raise a proper error.

closes #1504

Signed-off-by: Alexandre Lavigne <[email protected]>
(cherry picked from commit db59084)
alifeee pushed a commit that referenced this issue Oct 3, 2024
@lavigne958 @alifeee
better handler API error parsing.
9917d07 
Possibly we encountered an error while parsing the API error JSON, like
connection closes early etc and the received JSON is invalid.

Better handle such scenario and build a valid error Dict to keep the
exception stack flow running and raise this message to the application.

extract the returned text so if we any bit of valid information at least
we can pass that to the application.

This could be helpfull to if we add calls to the API made for web
browser that returns HTML error message, it will fail to decode the
recieved error and raise a proper error.

closes #1504

Signed-off-by: Alexandre Lavigne <[email protected]>
(cherry picked from commit db59084)
@alifeee alifeee mentioned this issue Oct 3, 2024
Closed
@alifeee
Copy link
Collaborator

alifeee commented Oct 3, 2024

hi all

a new release 6.1.3 has been released (on github) (on PyPi) which contains a fix for this issue

we now catch the non-JSON error and turn it into a GSpreadException which should be easier to catch and find out what went wrong with the request

see more context here: #1510

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

better handler API error parsing. burnash/gspread
4 participants
@alifeee @lavigne958 @dan-ladd @darttie