Product Data Scraper Build
Budget / SalaryHourly project
TypeFreelance project
LocationRemote
Posted2 hours ago
I need a Scrapy-based spider that reliably collects complete product information across three sources I specify: traditional e-commerce sites, larger multi-vendor marketplaces, and a small set of social-media storefront pages.
For every product it reaches, the crawler must capture exactly these fields and nothing less:
• Product name
• Price (in the currency displayed)
• Full product description
The code should be clean Python 3, using Scrapy pipelines for any post-processing and exporting the results in both CSV and JSON. Please structure it so I can add or swap domains later with minimal edits—ideally via a single config or a domains.py file.
Acceptance criteria
1. One command deploys the crawl for each domain without errors.
2. Output files include all three fields for at least 95 % of product URLs in a provided test list.
3. Your README explains setup, required Python packages, and how to extend the spider to new platforms.
I can run quick tests and give you prompt feedback, so feel free to suggest milestones that break the work into manageable chunks.
For every product it reaches, the crawler must capture exactly these fields and nothing less:
• Product name
• Price (in the currency displayed)
• Full product description
The code should be clean Python 3, using Scrapy pipelines for any post-processing and exporting the results in both CSV and JSON. Please structure it so I can add or swap domains later with minimal edits—ideally via a single config or a domains.py file.
Acceptance criteria
1. One command deploys the crawl for each domain without errors.
2. Output files include all three fields for at least 95 % of product URLs in a provided test list.
3. Your README explains setup, required Python packages, and how to extend the spider to new platforms.
I can run quick tests and give you prompt feedback, so feel free to suggest milestones that break the work into manageable chunks.
Apply on Freelancer →
Project sourced from Freelancer.com. Applications happen directly on the original platform — we never collect your data.