Python CSV Automation & Verification

via Freelancer ·

Budget / SalaryHourly project
TypeFreelance project
LocationRemote
Posted1 hour ago
I need a clean, well-documented Python script that can pick up CSV files from a watched folder, process them according to my business rules, and then push the results to a designated output location. The workflow breaks down into three core parts.

1. File I/O handling
• Automatically read incoming CSV files.
• After processing, write the transformed data to a separate folder.
• Rename each source file to a timestamped archive name and move it to an “_archive” sub-directory so nothing is re-processed by mistake.

2. Data processing & algorithmic checks
The CSVs include a fixed header row followed by transactional records. For every row the script must:
• Run field-level validation (type, range, mandatory columns).
• Apply record-level business rules that I will supply as a Python dictionary of lambdas.
• Flag or correct detectable data errors—e.g. wrong date formats or out-of-range numeric values—while logging any changes so I have an audit trail.

3. Reporting
• Create a concise log (plain text or JSON) listing files handled, rows processed, rows corrected, and any unrecoverable errors.
• Exit with a non-zero status if unrecoverable errors are found so my scheduler can react.

Deliverables
• A single runnable .py file (Python 3.10+), built on standard libraries plus pandas if you find it useful.
• A README that explains setup, config variables (input/output paths, rule dictionary), and example command-line usage.
• Sample unit tests (pytest preferred) covering at least the validation and renaming routines.

Acceptance criteria
✔ Processes a test batch of CSVs without manual intervention.
✔ Produces the expected transformed output and audit log.
✔ Renames and moves originals exactly once per run.
✔ Returns correct exit codes for success vs unrecoverable error cases.

If you have questions about the rule dictionary format or edge cases, let me know early so we can lock requirements before you code.
php python data processing software architecture json scripting automation pandas
Apply on Freelancer →

Project sourced from Freelancer.com. Applications happen directly on the original platform — we never collect your data.