Python Flask Scraping Integration Fix and Improvements

via Freelancer ·

Budget / Salary€250–750
TypeFreelance project
LocationRemote
Posted1 hour ago
Project Overview
I have an existing Python / Flask application using Scrapfly for web scraping.

The application provides three main API flows:

Search destinations
Search hotels for a selected destination. A separate working demo/proof of concept is available and successfully extracts the required data.
Search hotels around geographic coordinates
The current scraping integration stopped working due to upstream antibot and changes.

The main goal is to restore the existing integration and make targeted improvements to session handling, provider architecture, configuration, testing, and documentation.

This is an incremental fix and improvement project, not a full rewrite.

Main Scope
The project includes:

Restore the three main scraping/API flows

Fix request setup, headers, referrer, cookies, and session state where needed

Improve session pooling, reuse, retries, expiration, cleanup, and persistence

Improve provider abstraction / Inversion of Control

Keep Scrapfly behind a clear provider interface

Add configurable language and currency

Default language: English
Default currency: EUR
Add meaningful unit and integration/end-to-end tests

Improve relevant logging and technical documentation

Existing behavior should be preserved unless a change is explicitly agreed.

A detailed task breakdown with requirements and acceptance criteria will be provided.

Development Approach
Development will take place in an existing private GitHub repository.

I will provide:

Support with questions and clarifications
Repository access
The working demo/proof of concept
A Scrapfly API key for development/testing
Changes should be:

Small and reviewable
Clearly committed in Git
Delivered through Pull Requests
Incremental rather than large rewrites
Major architectural changes should be discussed before implementation.

AI-assisted development tools are allowed, but the developer is responsible for reviewing, understanding, and validating the final code.

Expected Deliverables
Working implementation of the agreed scope
Source code delivered through the project repository
Automated tests for important changed functionality
Updated technical documentation
No credentials or API keys
A feature is considered delivered when the corresponding code is available in the repository and can be reviewed and tested.

Proposal
Please briefly mention:

Your experience with Python / Flask and web scraping
Experience with Scrapfly or similar scraping services
Experience with sessions, cookies, retries, and persistence
Whether you are comfortable working through small Pull Requests in an existing codebase
All project communication, milestone approvals, and payments must remain on Freelancer.com.
javascript python web scraping flask reverse engineering software engineering
Apply on Freelancer →

Project sourced from Freelancer.com. Applications happen directly on the original platform — we never collect your data.