PinLeads: Turning browser discovery into a usable lead workflow

Overview

PinLeads discovers business leads from search and public business websites, then gives users tools to review and manage the results. The product spans search setup, asynchronous scraping jobs, lead filtering and exports, account and team settings, subscriptions, and integrations used in outreach workflows.

It is a full-stack SaaS product where the browser automation itself is only one part of the user-visible system.

The problem

Collecting public business details manually is repetitive, but automated collection introduces its own operational problems: browsers consume memory, third-party pages can fail or challenge requests, jobs can outlive an HTTP request, and partial results still need to be useful.

The product needed to turn those long-running and imperfect browser sessions into jobs people could follow, stop, recover, and use.

Architecture

1. 01Next.js appSearch, jobs, filters, lead review
2. 02Express APIAuthentication, job and billing rules
3. 03Puppeteer workersBrowser pool and business extraction
4. 04PostgreSQL + FirebaseLead data, jobs, account state

The Next.js frontend handles search input, job progress, lead review, filtering, and exports. An Express API verifies Firebase identity and owns user, team, usage, and billing rules. Puppeteer work runs asynchronously through browser-pool and job services. User and team settings are held in Firebase; PostgreSQL stores paid-user job history and lead data. Job snapshots are persisted as JSON files so active work can be restored after a server restart.

Engineering decisions

Keep browser work out of the request lifecycle

Scraping starts a durable job and reports progress separately from the initial request. The service persists current progress and accumulated results, then allows the UI to poll status and display useful partial output if a run stops or encounters a challenge.

Manage browser lifetimes explicitly

A configurable browser pool reuses instances and keeps a small cap on concurrent browsers. Jobs can use user-specific proxy settings; CAPTCHA detection and page limits are handled in the scraping workflow, and browser instances are restarted after the configured page threshold to reduce long-running resource pressure.

Keep account and billing decisions on the server

The UI requests state and triggers actions; it does not calculate subscription entitlements or usage. The API coordinates identity, plan limits, teams, storage, Stripe, and optional HubSpot or Zapier sync.

Technical challenges and solution

Browser automation is stateful and failure-prone. PinLeads separates the job lifecycle from scraping execution, persists job state and partial results, and returns browser resources to the pool when work ends. That creates a place to report progress and failures without holding a user request open for a multi-query search.

The system also has to coordinate distinct stores: Firebase for account, team, and billing state; PostgreSQL for the paid lead dataset and history; and local files for recoverable in-flight jobs. Keeping these responsibilities visible makes retention and operational behavior easier to reason about.

What I built

I built the product across the Next.js application, Express services, Firebase authentication and account flows, asynchronous scraping orchestration, Puppeteer browser management, lead storage, subscriptions, and outreach integrations.

Engineering takeaways

• Browser automation needs job state, resource limits, and recovery designed alongside extraction logic.
• Useful progress and partial results matter because external sites make perfect completion an unsafe assumption.
• SaaS entitlements belong behind an API boundary so UI state cannot become billing authority.
• Mixed persistence can work when each store has a clearly limited responsibility.

Stack

Next.js, TypeScript, Express, Firebase Authentication and Firestore, PostgreSQL, Puppeteer, Stripe, Docker, Cloudflare, and browser automation infrastructure.