Playwright tracing zip file corrupted when terminating worker process prematurely

Playwright writes trace.zip only when the browser context closes cleanly, so a worker killed mid-run (SIGKILL, timeout teardown, OOM) leaves a truncated ZIP with no central directory, and unzip, show-trace, and trace.playwright.dev all report it as corrupted. Root Cause Diagnosis During a test, Playwright streams trace events into a temporary directory (<artifactsDir>/traces/, .trace and .network … Read more

Handling sticky dynamic session IDs in Python Scrapy without losing referer headers

Sticky session IDs break Referer propagation in Scrapy because the session token typically lives in the URL (e.g., ;jsessionid=… or ?sid=…) or in a cookie that the server rotates mid-crawl, while RefererMiddleware sets Referer from response.url of the parent. When the server rewrites the URL with a new session ID, the dupefilter treats it as … Read more

Python Requests SSLError: UNEXPECTED_EOF_WHILE_READING on specific HTTPS hosts

Root Cause Diagnosis The error [SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol means the peer closed the TCP connection without sending a TLS close_notify alert, or without completing the handshake. OpenSSL 3.0+ treats this as a fatal error, while OpenSSL 1.1.1 silently tolerated it. Python 3.10+ builds linked against OpenSSL 3.x therefore fail only … Read more

How to capture network response body in Playwright without blocking navigation

Playwright blocks navigation when a page.on(“response”) handler or route.fetch() call awaits response.body() inline with the navigation promise, because body() resolves only after the full payload is received and page.goto() can wait on the same response lifecycle. Root Cause Diagnosis response.body() returns a promise that settles only after the response stream finishes. If you await it … Read more

Resolving WordPress transient cache buildup causing 504 Gateway Timeout on admin-ajax.php

Without a persistent object cache, WordPress stores every transient as two rows in wp_options, and an expired row is deleted only when get_transient() requests that exact key or when the daily delete_expired_transients cron event runs. Plugins that write per-request or per-user keys leave hundreds of thousands of rows that are never read again, and the … Read more

Resolving Playwright browser download timeout behind corporate proxy

Playwright’s npx playwright install (or postinstall download) fails behind a corporate proxy because the browser binaries are fetched by Playwright’s own Node.js downloader from cdn.playwright.dev (with fallbacks to playwright.download.prss.microsoft.com and playwright-akamai.azureedge.net), not by npm, so npm’s proxy settings are not applied to it. The downloader only honors the HTTPS_PROXY/HTTP_PROXY environment variables, and it fails with … Read more

PyInstaller Executable Failing on Target Machine with Missing libGL.so.1

Root Cause Diagnosis PyInstaller bundles Python modules and most shared libraries, but it deliberately excludes libGL.so.1 through its built-in exclude list, because that library is tied to the host’s GPU driver and graphics stack. Any bundled native module that links against it (typically cv2 from opencv-python, or Qt, pyqtgraph, vtk, Open3D, pygame/pyglet extensions) will fail … Read more

Fixing urllib3 ConnectionResetError(104, ‘Connection reset by peer’) during bulk API requests

Root Cause Diagnosis The server (or an intermediary such as a load balancer, WAF, or reverse proxy) sent a TCP RST on a connection your client believed was still open. This typically happens when a pooled keep-alive connection sits idle past the server’s idle timeout and is closed server-side, or when the server resets connections … Read more

Python script crashing with Segmentation Fault (core dumped) inside Docker Alpine

Alpine Linux uses musl libc instead of glibc, and a Python process segfaults on it when a C extension built for glibc, a glibc-only binary wheel, or code that relies on glibc’s default thread stack size runs against musl’s different ABI and limits. The crash happens in native code, not in Python, so no Python … Read more

Playwright async wait_for_selector timing out on dynamic Shadow DOM elements

Root Cause Diagnosis wait_for_selector times out because the selector either cannot cross the shadow boundary or the element never meets the default state=”visible” condition. Playwright’s CSS and text engines pierce open shadow roots automatically, but they cannot reach closed roots, XPath never pierces shadow DOM, and the > child combinator does not cross a shadow … Read more