playwright expect_response() now throws error when trying to parse method

[CSorrel58]

Describe the bug:

Even though the response itself has a 200 status, playwright is no longer to read the output of the json(), text(), or body() methods with the latest update. it throws the error

Response.text: Protocol error (Network.getResponseBody): is.setData is not a function
Traceback (most recent call last):

will linclude larger error below

Screenshots:

If applicable, add screenshots to help explain your problem.

To Reproduce:

async with page.expect_response(
            re.compile(r"orchestra/snb/graphql/Search")
        ) as response_info:
            try:
                # try to manually fill in search page first
                await page.click(home_page_sel.search_bar_div)
                await asyncio.sleep(1)
                exposed_clear_button = await page.locator(
                    home_page_sel.exposed_search_clear
                ).is_visible()
                if exposed_clear_button:
                    await page.click(home_page_sel.exposed_search_clear)
                await page.fill(home_page_sel.search_bar_div, "")
                await page.type(home_page_sel.search_bar_div, search_term, delay=300)
                await page.click(home_page_sel.search_button)
                logging.info("Clicked search")
            except Exception as e:
                logging.error(e)
                logging.info(
                    "Failed to click to search page due to above error going directly"
                )
                url = self.get_search_page_url(search_term=search_term)
                logging.info(f"###### Opening Walmart search Page {url}")
                await page.goto(url)
        response = await response_info.value
        logging.info(response.status)
        if response.ok:
            try:
                graphql_response = await response.json()
            except Error:
                body_bytes = await response.body()
                response_text = body_bytes.decode("utf-8", errors="ignore")
                print(response_text)

Version:

Pip package:    v0.4.11 Camoufox:       v135.0-beta.21 (Up to date!)

longer error

body_bytes = await response.body()
                       │        └ <function Response.body at 0x107e48a60>
                       └ <Response url='https://www.walmart.com/orchestra/snb/graphql/Search/247268667098567c560f5d7ebac199a1d8726a7bf193c486318a98ab8...

  File "/Users/clintonsorrel/Library/Caches/pypoetry/virtualenvs/sensing-WkZeiK1S-py3.10/lib/python3.10/site-packages/playwright/async_api/_generated.py", line 595, in body
    return mapping.from_maybe_impl(await self._impl_obj.body())
           │       │                     │    │         └ <function Response.body at 0x1070941f0>
           │       │                     │    └ <Response url='https://www.walmart.com/orchestra/snb/graphql/Search/247268667098567c560f5d7ebac199a1d8726a7bf193c486318a98ab8...
           │       │                     └ <Response url='https://www.walmart.com/orchestra/snb/graphql/Search/247268667098567c560f5d7ebac199a1d8726a7bf193c486318a98ab8...
           │       └ <function ImplToApiMapping.from_maybe_impl at 0x106671900>
           └ <playwright._impl._impl_to_api_mapping.ImplToApiMapping object at 0x1070ddc90>
  File "/Users/clintonsorrel/Library/Caches/pypoetry/virtualenvs/sensing-WkZeiK1S-py3.10/lib/python3.10/site-packages/playwright/_impl/_network.py", line 858, in body
    binary = await self._channel.send("body")
                   │    │        └ <function Channel.send at 0x106673760>
                   │    └ <playwright._impl._connection.Channel object at 0x17b0c6830>
                   └ <Response url='https://www.walmart.com/orchestra/snb/graphql/Search/247268667098567c560f5d7ebac199a1d8726a7bf193c486318a98ab8...
  File "/Users/clintonsorrel/Library/Caches/pypoetry/virtualenvs/sensing-WkZeiK1S-py3.10/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
                 │    │           └ <function Connection.wrap_api_call at 0x107055360>
                 │    └ <playwright._impl._connection.Connection object at 0x179d10100>
                 └ <playwright._impl._connection.Channel object at 0x17b0c6830>
  File "/Users/clintonsorrel/Library/Caches/pypoetry/virtualenvs/sensing-WkZeiK1S-py3.10/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 528, in wrap_api_call
    raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
          └ <function rewrite_error at 0x1065db490>

playwright._impl._errors.Error: Response.body: Protocol error (Network.getResponseBody): is.setData is not a function

[daijro]

Hello, what Python version and OS are you using?

[princ3od]

Hello, what Python version and OS are you using?

Hi @daijro, I've experienced this issue as well

• Python: 3.10
• OS: Linux

My Dockerfile

FROM python:3.10-slim

WORKDIR /app

# Install Python and other tools
RUN apt-get update && apt-get install -y \
    python3 \
    python3-pip \
    xvfb \
    flac \
    && apt-get clean 

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONBUFFERED 1
ENV PORT 8080

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Install Playwright dependencies & Camoufox
RUN playwright install-deps && \
    python3 -m camoufox fetch
    
COPY . .

CMD exec uvicorn main:app --host 0.0.0.0 --port 8080

[daijro]

@princ3od Thanks. Would you (or anyone else reading) be able to provide a small script that I could run and reproduce the issue? I'm looking into why I don't seem to be running into this issue on my environment (resp.body() doesn't seem to raise any errors). Maybe I'm testing it wrong. (Python 3.11, Playwright 1.50, Arch Linux).

I can be reached privately at daijro.dev@gmail.com 👍

[princ3od]

Perhaps your environment still use the FF134 from the cache.

As my old image still work normally, only the newly built-from-scratch images have this issue.

I use docker compose

docker compose up → work

docker compose up --built → raise `: Protocol error (Network.getResponseBody): is.setData is not a function` when try using `.json()`

[cda2]

@princ3od Thanks. Would you (or anyone else reading) be able to provide a small script that I could run and reproduce the issue? I'm looking into why I don't seem to be running into this issue on my environment (resp.body() doesn't seem to raise any errors). Maybe I'm testing it wrong. (Python 3.11, Playwright 1.50, Arch Linux).

I can be reached privately at daijro.dev@gmail.com 👍

I can't share the related work, but I'm encountering this issue in an environment with Python 3.11 and playwright==1.48.0.
Everything was fine up to beta 20, but after upgrading to beta 21, the problem suddenly occurred.

[vitalikaz]

@princ3od Thanks. Would you (or anyone else reading) be able to provide a small script that I could run and reproduce the issue? I'm looking into why I don't seem to be running into this issue on my environment (resp.body() doesn't seem to raise any errors). Maybe I'm testing it wrong. (Python 3.11, Playwright 1.50, Arch Linux).

I can be reached privately at daijro.dev@gmail.com 👍

Same with Python 3.13.2 and Playwright 1.50 (both Javascript and Python clients are affected, both on Debian and MacOS). Rolling back to FF134 like mentioned here solves the issue.

[princ3od]

A little bit overkill for a script but here a full docker set up to reproduce the issue

requirements.txt

camoufox[geoip]==0.4.10
fastapi==0.115.6
uvicorn[standard]==0.34.0

Dockerfile

FROM python:3.10-slim

WORKDIR /app

# Install Python and other tools
RUN apt-get update && apt-get install -y \
    python3 \
    python3-pip \
    xvfb \
    flac \
    && apt-get clean 

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONBUFFERED 1
ENV PORT 8080

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Install Playwright dependencies & Camoufox
RUN playwright install-deps && \
    python3 -m camoufox fetch
    
COPY . .

CMD exec uvicorn main:app --host 0.0.0.0 --port 8081

docker-compose.yml

version: '3'
services:
  service:
    build: .
    environment:
      PORT: 8081
    ports:
      - "8081:8081"
    volumes:
      - .:/app

main.py

from fastapi import FastAPI
from camoufox.async_api import AsyncCamoufox

app = FastAPI()

@app.get("/")
async def run():

    try:
        async with AsyncCamoufox(
            headless="virtual",
            geoip=True,
            block_images=True,
        ) as browser:
            page = await browser.new_page()

            await page.goto("https://www.youtube.com/", wait_until="commit")

            async with page.expect_response(
                lambda response: "guide" in response.url
            ) as response_value:
                response = await response_value.value
                json_response = await response.json()
                print(json_response)

    except Exception as e:
        print(f"Error: {e}", flush=True)
        return {"status": "error"}
    return {"status": "success"}

Run

docker compose up --build                        

Then, open localhost:8081 and observe the error in the log

Error

Error: : Protocol error (Network.getResponseBody): is.setData is not a function

[daijro]

@princ3od Thanks for the code snippet! Seems like the issue was only present on sites with certain response encoding types, which made it difficult to test/find. I have just pushed a fix.

[daijro]

v135.0-beta.22 is out now — it should fix the issue 👍

[princ3od]

v135.0-beta.22 is out now — it should fix the issue 👍

Awesome work man! Confirmed the issue is fixed on my newly built images 💪🏻

[CSorrel58]

Thank you @daijro !