Commit Graph

7 Commits

Author SHA1 Message Date
f74e9bcfb0 refactor: split ssr.py into package, enrich OG Online detail pages, fix travel upsert
- Split src/adapters/ssr.py (2160 LOC) into ssr/ package grouped by CMS:
  realworks.py, sure.py, schiedam.py, denhaag.py, overige.py
- Add _og_detail() to api.py; all OG Online scrapers now fall back to
  detail page fetch when energielabel/bouwjaar are missing from the API
- Fix run() to recalculate travel times for existing listings where
  fiets_mark IS NULL; upsert() now writes travel cols on existing rows too
- Update tests/cache.py to patch fetch_soup in every ssr submodule
- Update docs to reflect new package structure and mark API enrichment TODO done

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 23:39:35 +02:00
1011d9cf87 feat: add scrapers for vandriel, vanherk, vanoord
- vandriel (Schiedam): OG Online API, filtered by city=schiedam
- vanherk (Schiedam): SURE WordPress plugin (card-house), detail page kenmerken
- vanoord (Delft + Schiedam): Elementor WordPress, two filtered listing URLs, rw-object-features-list detail parsing
- makelaars.md: mark all three as done, add TODO for API scraper detail page enrichment

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 19:31:08 +02:00
75c5b6f26d add scrapers: 88makelaars, Borgdorff (SSR) + Elzenaar, DOEN (OG Online API) for Den Haag
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 23:30:06 +02:00
d310a7a560 add scrapers: Van Daal (API), Van Silfhout (SSR) for Delft
- fetch_vandaal: OG Online API, covers Delft/Rijswijk/Den Haag area,
  includes is_bought→verkocht status mapping
- fetch_vansilfhout: HTML scraper, all listings on single page,
  extracts postcode from embedded JS variable (objectZipcode)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 21:39:02 +02:00
c92ddb5812 add scrapers: Moerman & De Jong (API) and Schieland Borsboom (SSR)
- fetch_moerman: OG Online realtime-listings API (same platform as bjornd),
  includes bouwjaar from dateOfConstruction, energielabel, strips postcode space
- fetch_schielandborsboom: paginated HTML scraper filtered to Schiedam,
  fetches #kenmerken detail page for full specs (bouwjaar, kamers, etc.)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 21:34:58 +02:00
17b35d1997 add some more makelaars, and some more infra 2026-04-03 15:49:42 +02:00
26d9d936f4 first setup, travel works, bjornd api works 2026-04-03 13:53:39 +02:00