Fetchyr

RPA and authenticated browsing — login flows, MFA, and form automation

2 min read

Fetchyr manages authenticated browser sessions for sites requiring login, MFA, or form automation. Use it when you need to scrape pages behind a login wall.

login

session = client.fetchyr.login(
    url="https://example.com/login",
    username_selector="#email",
    password_selector="#password",
    username="user@example.com",
    password="secret",
    submit_selector="button[type=submit]",
)
print(session.session_id, session.artifact_id)

login() parameters

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | url | string | Yes | Login page URL | | username_selector | string | Yes | CSS selector for username field | | password_selector | string | Yes | CSS selector for password field | | username | string | Yes | Login credential | | password | string | Yes | Login password | | submit_selector | string | No | CSS selector for submit button | | success_url_contains | string | No | URL substring confirming login |

fetch (authenticated)

result = client.fetchyr.fetch(
    url="https://example.com/dashboard",
    session_artifact_id=session.artifact_id,
)
print(result.html, result.artifact_id)

detect_forms

forms = client.fetchyr.detect_forms(
    url="https://example.com/checkout",
    session_artifact_id=session.artifact_id,
)
for form in forms.forms:
    print(form)

detect_mfa / submit_mfa

mfa = client.fetchyr.detect_mfa(
    url="https://example.com/mfa",
    session_artifact_id=session.artifact_id,
)
if mfa.mfa_detected:
    result = client.fetchyr.submit_mfa(
        challenge_id=mfa.challenge_id,
        code="123456",
    )

create_workflow / execute_workflow

workflow = client.fetchyr.create_workflow(
    name="Login flow",
    steps=[
        {"action": "navigate", "url": "https://example.com/login"},
        {"action": "fill", "field": "username", "value": "{username}"},
        {"action": "fill", "field": "password", "value": "{password}"},
        {"action": "click", "selector": "button[type=submit]"},
    ],
)
result = client.fetchyr.execute_workflow(workflow.workflow_id)

check_duplicates

result = client.fetchyr.check_duplicates(
    records=[{"id": "1", "url": "https://example.com/p/1"}],
    domain="example.com",
)
print(result.duplicate_count, len(result.unique_records))

Next steps

• Authenticated scraping guide — complete walkthrough
• Choosing an approach — when to use authenticated scraping
• Advanced modules — module overview