SC2 — Where We Landed: Scraping & Vision

The main sweep

This is the big pull that lists every home for sale in San Antonio — the whole board is built from it. Five services were bake-off tested with real money; one won on cost, coverage, and the fact that it hands us the photos for free. How often we re-pull the whole list — versus just checking what changed — is the Delta question, not this one.

→

On cadence, to be exact: the full "every home" sweep is an occasional job — in the plan it runs about weekly, and it isn't running on a schedule yet. The daily job is only the cheap change-check (what's new / dropped price / gone). We do not re-scrape every home every day.

igola Chosen

The numbers ruled it

igolaizola~zillow-scraper-ppe · ~$0.0009 per home

We pointed all five scrapers at San Antonio and compared. igola alone returned 13,536 of the 13,591 homes every scraper found combined — everything else added only ~55 homes between them. And it swept the entire metro for $11.61, photos included. Photos-included is the quiet winner: it means we never pay a second time to fetch pictures for the AI to look at.

Full metro sweep

12,416 homes

Settled cost

$11.61

Photos / home

~49 free

Coverage vs all sources

99.6%

→

One catch to know: when you ask igola "what changed recently," it gives you an order but no marker saying what changed or when. That's fine for the big daily sweep — it matters for the Delta tab, where it's the whole debate.

Everything igola gives us — the whole board is built from these

40 usable fields per home, measured against the live 13,036-row board. Coverage = share of homes that actually carry a value.

Identity & location

Address ·100%

Street address (2,022 have none → "No address" badge)

ZIP ·100%

Territory / warehouse routing

Cover photo ·100%

The one photo shown per lead

All photos ·~99%

Full carousel — what vision reads

Listing URL ·100%

The Zillow link (the real drill-in)

Zillow ID / zpid ·100%

Stable identity for dedupe

The house itself

Price ·100%

Qualifies the staging tier + fee

Beds ·99.8%

Baths ·99.8%

Square feet ·99.7%

How much furniture it takes

Property type ·100%

Single-family 90% / condo / townhome / multi / mobile

Year built ·99.6%

Older stock often needs staging

Lot size ·73.1%

Acreage / lot area

Price & value signals

Zestimate ·70.8%

Zillow's value estimate

Price vs Zestimate % ·70.8%

Priced under market = motivated seller

Price / sqft ·~95%

Value density, cross-listing compare

Rent Zestimate ·89.4%

Investor-lens value

Tax assessed value ·55.9%

A second value anchor

Tax assessment year ·55.9%

Timing & freshness

Days on market ·100%

Stale = anxious seller = warmer lead

Listed date ·100%

Real list date (DOM computed live from it)

First seen ·100%

When we first captured it

Last seen ·100%

Most-recent capture — the "still there?" signal

Status ·100%

active (single-valued today)

Lead-lane flags (who's a special case)

FSBO ·88

Owner-listed, no agent — own lane

Open house ·199

A showing is scheduled — timing

New construction ·5,428

Broad new-build flag

Builder plan ·2,588

To-be-built render — hidden by default

Builder spec home ·2,749

Real new house — good lead, marked

Pre-foreclosure / auction ·31

Distress = motivated

For auction ·156

Bank owned (REO) ·19

Listing type ·98.5%

fsba / new_construction / fsbo / coming_soon

Vision class, confidence, the 0–100 score and the call/mail/drip/drop bucket also sit on every row — but those are things we compute from the photos and signals, not things igola hands us. They live in the Vision tab.

How we change this later Swapping the main scraper is a CHEAP · config change — no part of the database depends on which service filled it, and a fresh full sweep is only ~$12. The one thing that would actually hurt: picking a source that doesn't include photos, which would push vision cost up because we'd pay to fetch images separately.

!

Worth your eye: a seat wrote the recommendation and another seat signed it on 2026-07-02 — the numbers are genuinely strong, but you never actually ratified igola. This tab is your chance to.

clearpath Kept as backup

clearpath~zillow-bulk-search-unlimited-scraper · ~$0.0005 per run

Returns the same core fields as igola, plus two things igola lacks: a native "sold" mode (useful for spotting homes that left the market) and real signed price markers baked in — "−$10k on Jul 2" — instead of us working that out in code. Slight staleness in its cache. We kept it as the backup lane and the sold-detector; it also reappears as a live contender in the Delta debate.

RapidAPI "class-C" APIs Emergency net

us-property-market1 (600/mo) · zillow-real-estate-api (50/mo) · free within quota

Two free services with hard monthly call limits — so they physically cannot run away with your money. The quotas are small, so they're no good for daily breadth, but they're a structured safety net: if Apify goes down or a quota's exhausted, we can switch to these and still get listings. $0 as configured.

maxcopell as the main sweep Rejected here

maxcopell returns richer per-home data, but at roughly 4× the price for the breadth job — and igola's photos were already good enough for the AI to judge staging. Rejecting it for breadth saved $36–72 on a full backfill. (maxcopell isn't gone: it's the pick for the narrow Detail job on hot leads, where its one unique field earns its keep.)

MLS / RESO Bridge Never a dependency

You ruled it

The MLS is the industry's official listing feed — free if a broker sponsors us. It's a nice-to-have someday, but you ruled it clearly: "nice-to-have someday, never depend on it" (2026-06-30). So it's not in the plan, and nothing we build assumes it. This one's fully your call and it's on record.

The extra sweeps — for-sale-by-owner

Homes sold without an agent are a separate, cleaner outreach lane — no agent to go through. It's a lane you asked for (confirmed wanted 2026-07-11). We tested twelve scrapers for it; the adapter is built but no scrape has ever run yet — it's ready and dormant.

igola (owner-posted) + maxcopell Chosen pair

The numbers ruled it

The Zillow FSBO backbone. maxcopell gave the highest count (166 metro) at the cheapest price. igola can also pull owner-posted homes — it first looked broken (963 of 1,000 results were actually agent listings) until we found it works cleanly if you explicitly set every other listing-type flag to false. clearpath rides along as a cross-check (careful — its default is rentals, a trap).

These return the same house data as the main sweep — the point of the lane is which homes (owner-sold), not new fields.

solidcode — FSBO.com Chosen · complementary

This is the valuable one. FSBO.com is a different site than Zillow, so it surfaces +41 leads that are invisible on Zillow entirely — and it comes with the seller's name and phone number. That's a warm, agent-free lead with contact info attached. A genuinely additive lane, not a duplicate.

Zillow-invisible leads

+41

Includes

name + phone

swerve — ForSaleByOwner.com Lane 4 · gated

A seat staged it

A third FSBO site yielding +147 unique homes, 84% fresh, independently validated. It's promising enough to keep as a fourth lane — but held behind a freshness audit before we trust it (some FSBO sites carry stale, already-sold listings). A cold backup, xtracto, sits behind it. Whether to actually turn this on is open.

Ruled out — and two that cost us to learn Rejected

tri_angle (Redfin) — burned $4 on unusable data.
realtygrind — $0.15 per result (65× the going rate); burned $1 for 0 usable items.
benthepythondev — returned 0 items, timed out.
memo23 — works but lossy (kept 41 of 63) and mangles fields.
whitewalk / scrapyspider / craigslist / api-ninja / memo23-propwire — no owner filter, wrong geography, or no real addresses.

These two burned-dollar lessons are why every scrape now runs under a hard spend cap first.

The delta — keeping the board current

Once we have the board, we need to know each day what's new, what changed price, and what's gone. The how is settled. The which scraper is the one genuinely open money-fork in the whole mission — and it's waiting on a test we've never run.

Per-ZIP daily pulls + a weekly full sweep Settled shape

The numbers ruled it

The original idea — one big citywide pull, take the top slice — was tested and failed (it caught only 13–39% of real changes). What works: pull each ZIP code separately every day (catching 60–70% of each ZIP's homes), and run one full sweep weekly as the ground-truth backstop. That shape is proven and settled; you tried it on a real ZIP cohort already, which is exactly the "small test on a slice" instinct you have.

→

You mentioned wanting to try a small partial pull (a ZIP or price band) to understand the data better — that's already how this was proven, and it's the cheapest kind of test to re-run any time you want to see it live.

clearpath vs igola for the daily check OPEN — your call

Nobody ruled it

This is the real fork. Both work. It's a quality-vs-cost trade, and the tiebreaker test — a 7–14 day live side-by-side — has never been run. So nothing here is decided; the board literally says "gated on a shadow A/B."

	clearpath	igola
Monthly cost	$68–102 / mo	$27–41 / mo (2.5× cheaper)
Finds changes	~2× shallower in the list — surfaces them faster	deeper down the list
Agent-change events	catches 100%	catches 0%
Price markers	native "−$10k on Jul 2" — less code	order only — we compute the change

Plain version: clearpath is the nicer data and finds changes faster, for about twice the money. igola is cheaper and we write a bit more code to fill the gap. Neither is wrong — it's a taste-and-budget call, and it's yours.

How we settle it Run both in parallel for two weeks on real San Antonio data and see whether clearpath's better change-detection is worth the extra ~$40/mo. CHEAP to test — the loser is just switched off. Until then, MVP monthly spend is $0 either way, so there's no rush.

How we tell a home is gone Designed & proven

A seat set the rule

A house drops off the listings — sold, withdrawn, or did the scraper just miss it? We don't trust a "404 not found" (Zillow fuzzy-matches bad IDs and says "active" — false positives). Instead: if a home is absent from a full sweep, it becomes a "maybe gone" candidate; we do one cheap targeted lookup (~$0.002) to confirm sold/pending/withdrawn; and if it's missing from two sweeps in a row (not one — a single miss can be a scraper hiccup), we mark it gone. A Redfin cross-check ($0.0001) is the independent second opinion.

This answers your "dropped delta" question directly. It's designed and measured (5/5 correct on the real cohort test) — just not built yet. The "two-in-a-row" rule was a seat's call, backed by the test but never ruled by you.

How we change it later The "two misses" threshold is a single number — CHEAP to loosen or tighten if the "gone" calls come too slow or too jumpy.

The detail scrape — do we even need it?

A "detail" scrape fetches rich per-home data one listing at a time. The honest answer: almost never — and it hinges on one question for Brandon.

The walkthrough Hinges on Brandon

A seat scoped it

We checked what a detail scrape gives us that the main igola sweep doesn't. The answer is: almost nothing. Price history, foreclosure status, relisting — igola already carries all of it, byte-for-byte identical. There is exactly one field only the detail scrape provides:

The only unique thing a detail scrape adds

The agent's name & phone number

So the whole question reduces to: how does a rep contact a lead?

If reps click through to the Zillow listing to make contact → we don't need the detail scrape at all. $0.
If you want reps to call the agent from inside our app → we run maxcopell, but only on the top handful of hot leads (~$0.12/week), never the whole board.

?

This is a Brandon question (already queued): do reps call from inside our tool, or click out to Zillow? His answer decides whether this scrape exists. Nothing's built either way.

maxcopell detail scraper Top-leads only, if at all

maxcopell~zillow-detail-scraper · ~$0.0024 per home

If we do need agent contact info, this is the tool — it can target a specific home by its Zillow ID (igola can't) and pulls the agent name + phone 7 times out of 10. The rule we'd set: never on breadth, never on the daily loop — only the top ~50 leads a rep is about to work. At that scale it's about 12 cents a week.

Vision — reading the photos with AI

This is how a home gets its staging read. The pipeline is designed, measured, and adversarially stress-tested — but it still carries status: proposed, because the big run costs money (on trial credits) and waits on your GO. A few method questions inside it are genuinely still yours.

Four independent passes Adopted · stress-tested

Numbers + 7-voice crucible

Your instinct — "triage collage, then advanced collage, then individuals" — was exactly the debate. We tested the chained version (cheap first-look hands only the good photos to the expensive judge) and retired it: it goes blind to digitally-staged homes (10 of 21 fake-staged homes reached the judge with zero usable photos) and it's cheaper on paper but wobbly. What we landed on runs each look independently:

Pass	What it does	Cost / 10k homes
0 · Gates	Free database checks (land, builder-plan, etc.)	$0
1 · Triage	Picks the cover photo, labels render/community shots	~$10
2a · CGI net	Catches digitally-faked / rendered photos	~$29
2b · The judge	The real read: how furnished, how polished	~$117

Per home

$0.0156

Full backfill (batched)

~$78 trial credits

Real dollars

$0

!

The pipeline is seat-decided and crucible-hardened; the authority to spend the backfill is still yours and unexercised. Nothing runs on the 10k homes until you say GO.

Two scores: Fill × Polish Seat-locked

A seat locked the shape

The AI rates each home on two 0–10 scales: Fill (how furnished is it) and Polish (how good is the styling). Every axis is always computed and saved, and Brandon gets on/off + weight toggles later — that store-everything design is your call (2026-06-25). The specific two-scale shape was locked by a seat three days after a review panel warned against exactly two fine-grained scales. It works, but it's a fair thing to revisit — flagged honestly.

How photos are delivered to the AI Measured

The numbers ruled it — after you caught an error

We tested single photos vs. a 2×2 grid vs. a 3×3 grid vs. one big collage. For the CGI/fake-photo catch, the 2×2 grid won (caught all 21 fake homes). The big collage missed exactly the home you'd caught by eye — too much shrinking loses the tell. Worth knowing: an earlier round had concluded "collage wins," graded against the AI's own answers — you eyeballed the errors and overturned it, and we re-tested against a gold set you could check on your phone. That's the cleanest case in the whole mission of you catching a false "settled."

Three vision questions still yours OPEN

All three are $0 to test and genuinely undecided:

Exterior photos in the fake-photo check — keep or drop? Your instinct: "community photos are noise; exterior-of-house is sometimes a CGI tell." Dropping exteriors cuts ~1,100 false flags per 10k homes but misses 3 catches. The framework itself calls this "the decision most likely to reverse." open
Skip builder spec-homes from vision? Would save ~41% of vision compute. Raised to you 2026-07-13, never answered. open
The "mark suspected-fake photos" feature you asked for — turns out it already exists: pass 2a already emits a per-photo "looks CGI" flag plus a plain-English reason for every photo, for $0. Your ask isn't a new AI pass — it's just putting what we already have on screen. a seat closed this — worth re-opening

The full catalog — every service we tested

All 121 candidates that appear anywhere in the bake-off record — winners, backups, and every rejection and burned-dollar dead end. Search by name, or filter by role and by what we decided. Each carries its Apify link, cost, data points, and verdict. This is the complete reference; the other tabs are the guided read.