Universal Agent Skills
Back to Collection
Skill/Research/Original by Mehansh

Universal Scraping Architect

Dynamic scraping strategy router with token budgeting and checkpointing.

What It Does

Analyzes target web pages and data extraction goals, selecting the optimal method across Firecrawl, local Python, or hybrid pipelines while planning token quotas and checkpointing state.

Architecture and Design Rationale

Most scraping scripts fail due to poor architectural decisions chosen early in development rather than syntax errors.

Attribution: Original engineering by Mehansh Barthwal. Contributed to alirezarezvani/claude-skills.

When to Activate

  • •Planning web extraction, automated crawling, or document ingestion
  • •Harvesting structured datasets from challenging web structures
  • •Configuring Firecrawl workflows with strict token limits

When to Avoid

  • •Simple single page text fetches that need only basic cURL requests

Skill Specification and Runtime Setup

Full Instruction Specification
# Universal Scraping Architect Specification

## Overview
Analyzes target web pages and data extraction goals, selecting the optimal method across Firecrawl, local Python, or hybrid pipelines while planning token quotas and checkpointing state.

## Rationale
Most scraping scripts fail due to poor architectural decisions chosen early in development rather than syntax errors.

## Core Principles
* Always verify environment constraints and dependencies before executing.
* Enforce negative constraints and reject speculative abstractions.
* Validate output against stated acceptance rubrics.

## Execution Workflow
1. Discovery: Map requirements and inspect active context.
2. Decomposition: Break down the task into discrete, verifiable phases.
3. Execution: Implement surgical changes preserving existing code architecture.
4. Verification: Test all criteria before declaring completion.

Related Skills in Research