Skip to content

Latest commit

 

History

History
176 lines (128 loc) · 13.4 KB

File metadata and controls

176 lines (128 loc) · 13.4 KB
title Python SDK
description Reference of the classes and methods in the Lightpanda Python package, covering every browser action and script replay.

Python SDK

The lightpanda package exposes Browser/AsyncBrowser, which spawn and manage the bundled binary, and Session/AsyncSession, with one method per browser action. See Use the Python SDK for practical documentation, and the generated API reference for every signature and docstring as shipped in the package.

Browser

Browser() spawns the bundled binary when constructed. It is not fork-inheritable: create a fresh instance in a forked child.

Argument Default Description
binary None Path to a specific lightpanda binary. When omitted, resolved from the LIGHTPANDA_BIN environment variable, then the binary bundled in the package, then PATH.
env None Extra environment variables for the spawned process.
timeout 300.0 Seconds to wait for a response before raising ProtocolError.
verbose False Print the spawned process's own logging.
args () Extra CLI flags for the spawned process, for example ["--http-cache-dir", path].
Method Returns Description
new_session() Session Open a new isolated browsing context: its own page, cookies, and memory.
tools (property) dict[str, dict] Every available action, as name → {description, schema}, reported live by the running browser.
close() None Stop the browser process.

with Browser() as b: calls close() on exit.

AsyncBrowser

AsyncBrowser mirrors Browser for asyncio: every call runs on a browser-owned thread pool, so the event loop is never blocked.

Argument Default Description
binary, env, timeout, verbose, args same as Browser Forwarded to the underlying Browser.
max_concurrency 32 Caps method calls executing concurrently across this browser's sessions. Worker threads are created lazily.
Method Returns Description
start() AsyncBrowser Spawn the process and fetch its action list. Idempotent; called automatically on async with entry and by new_session().
new_session() AsyncSession Start the browser if needed, then open a new session.
session() async context manager async with browser.session() as page: opens a session scoped to the block and closes it on exit.
tools (property) dict[str, dict] Same as Browser.tools.
close() None Stop the browser process, unless it was adopted with wrap (see below).
AsyncBrowser.wrap(browser, max_concurrency=32) (classmethod) AsyncBrowser Adopt an already-running Browser for use from asyncio. close() then shuts down only the async facade, leaving the wrapped browser running.

async with AsyncBrowser() as b: calls start() on entry and close() on exit.

Session and AsyncSession

Browser.new_session() and AsyncBrowser.new_session() are the only way to obtain a Session or AsyncSession; do not construct one directly.

Member Description
id (property) The session's id.
close() Close the session. Calls made after close() raise ToolError.
call(action, **kwargs) Invoke any action by name. The methods documented below route through this; it also accepts the action and argument names exactly as the browser declares them, for example page.call("tree", maxDepth=1).

Sessions are context managers too: with browser.new_session() as page: closes the session on exit. Closing the browser ends every session anyway.

Calling an action

Every browser action is a method on Session/AsyncSession, keyword-only, with the action and its arguments in snake_case: the waitForSelector action is wait_for_selector, and its backendNodeId argument is backend_node_id. The methods are generated from the bundled browser's action schemas, so the signatures and docstrings your IDE shows come straight from the binary. The generated reference for the latest release is published at lightpanda.io/docs/python; see Session and AsyncSession there for every method's exact signature and docstring.

A failed action raises ToolError.

In the Arguments column below, ? marks an optional keyword argument, and selector / backend_node_id marks a pair where one of the two is required. Prefer selector for reproducibility; it also wins when you pass both. backend_node_id takes the backendNodeId values returned by a prior tree, links, or find_element call. In the Returns column, JSON is a parsed Python dict or list, text is a plain string.

Navigation and search

These methods bring a page into the browser:

Method Arguments Returns Description
goto url, timeout?, wait_until? text Navigate to a URL and load the page in memory so it can be reused later for info extraction. wait_until accepts the same states as wait_for_state and defaults to load.
search query, timeout? text Run a web search and return results as markdown: a numbered list of {title, url, snippet}. The browser does not navigate; to open a result, call goto with its URL.

Reading the page

These methods read the loaded page without modifying it:

Method Arguments Returns Description
markdown selector?, backend_node_id?, max_bytes?, url?, timeout? text Render the page, or a subtree, as markdown. Scope with selector or backend_node_id to read just the relevant region; use max_bytes to cap long pages.
html selector?, backend_node_id?, max_bytes?, strip?, url?, timeout? text Raw HTML for the document, or a single node's outerHTML when scoped. Verbose; use only when you need attributes that markdown discards. Use max_bytes to cap long pages. strip is an object of element groups to omit: js (script, noscript, script preloads), css (style, stylesheet links), ui (css plus img, picture, video, audio, svg, canvas, iframe) and invisible (elements set to display:none). {"js": True, "css": True} keeps a page dump small.
screenshot path?, selector?, backend_node_id?, full_page?, url?, timeout? text or bytes Render the page, or one node, as a PNG: the text layout Lightpanda computes, not a pixel-accurate rendering (no images, fonts, or CSS colors). path must be a relative path. Without path, the PNG is returned as bytes.
tree url?, timeout?, backend_node_id?, max_depth? text Simplified semantic DOM tree: role, name, value, and backendNodeId per node.
links limit?, url?, timeout? JSON Extract all links as text (visible anchor text, falling back to aria-label/title/image alt), href (resolved URL), and backendNodeId (pass to node_details). One entry per href; hidden links are omitted. limit returns at most that many links, in document order.
node_details backend_node_id JSON Tag, role, name, value, and other state for a node, plus a ready-to-use CSS selector that resolves to it. The way to turn a backendNodeId into a selector.
find_element role?, name? JSON Find interactive elements by role and/or accessible name, with their backendNodeId.
interactive_elements url?, timeout? JSON Every interactive element on the page.
structured_data url?, timeout? JSON Structured data on the page, such as JSON-LD or OpenGraph tags.
detect_forms url?, timeout? JSON Forms on the page: fields, types, and required status.

Data extraction and scripting

Method Arguments Returns Description
extract schema, save? JSON Extract structured data from the current page using a schema mapping output field names to CSS-selector specs.
evaluate script, url?, timeout?, save? typed Evaluate a JavaScript string in the page context and return its value. Runs in the page, so it cannot see your Python variables.

evaluate's return is typed like the JavaScript result: for example 1+1 comes back as the int 2, and ({a:1}) comes back as the dict {"a": 1}.

extract's schema maps output field names to CSS-selector specs. Pass it as a Python dict or list, it's encoded for you; a JSON string also works:

Schema value Result
"<sel>" First match's text, or None
["<sel>"] Every match's text
{"selector": "<sel>", "attr": "<name>"} First match's attribute (href/src resolve to absolute URLs)
[{"selector": "<sel>", "attr": "<name>"}] Every match's attribute
[{"selector": "<sel>", "fields": {...}}] One dict per match, with fields resolved relative to each match

Add "limit": N inside any array spec to cap matches. Every extracted value is a string or None; parse numbers yourself.

Interacting with the page

These methods dispatch real DOM events on the page:

Method Arguments Returns Description
click selector / backend_node_id text Click an interactive element.
fill selector / backend_node_id, value text Fill text into an input element.
scroll backend_node_id?, x?, y? text Scroll the page, or a specific element if backend_node_id is given.
hover selector / backend_node_id text Hover over an element, triggering mouseover and mouseenter.
press key, selector?, backend_node_id? text Press a keyboard key, dispatching keydown and keyup. Targets the document if no element is given.
select_option selector / backend_node_id, value text Select an option in a <select> element by its value.
set_checked selector / backend_node_id, checked text Check (True) or uncheck (False) a checkbox or radio button. Dispatches input, change, and click events.

Waiting

These methods block until the page reaches a condition:

Method Arguments Returns Description
wait_for_selector selector, timeout? text Wait for an element matching a CSS selector to appear, and return its backendNodeId.
wait_for_script script, timeout? text Wait until a JavaScript expression returns truthy, re-evaluated on every tick.
wait_for_state state, timeout? text Wait for the page to reach a load state (load, domcontentloaded, networkalmostidle, networkidle, or done), with no navigation.

State and debugging

Method Arguments Returns Description
get_url none text The URL currently loaded in the session.
get_cookies url?, all? text Cookies stored in the browser. Defaults to the current page's host; pass url for another host or all=True for every cookie.
get_env name? text With name, read one LP_* environment variable. Without it, list the LP_* names that are set.
console_logs none text Buffered console.log/warn/error messages since the last call, which then clears the buffer.

Script replay

run_script and run_script_async (its awaitable variant, run in a worker thread) replay a saved PandaScript with no LLM call, by running lightpanda run <script> and returning its stdout. Installing the package also puts the lightpanda binary itself on PATH.

from lightpanda import run_script

run_script("hn.js", env={"LP_HN_USERNAME": "me"})
Argument Default Description
script required Path to the script file.
env None Extra environment variables for the child process, for example LP_* placeholder values the script reads.
binary None Same resolution as Browser's binary argument.
timeout None Seconds to wait for the process to exit.

A non-zero exit raises ScriptError. Exceeding timeout raises subprocess.TimeoutExpired instead.

Errors

Error Raised when
LightpandaError Base class for every error the package raises.
ProtocolError The connection to the browser process failed: a malformed request, a timeout, or an internal error. Carries a code attribute.
ToolError A browser action reported failure, such as a bad selector, a JS exception inside evaluate, or a call on a closed session.
ScriptError run_script or run_script_async exited with a non-zero status, or the script file doesn't exist (returncode=-1). Carries returncode, stdout, and stderr attributes.