CSV Data Validation Maintenance

I keep a collection of text-only records in several CSV files and need a solid validation pass before the next release cycle. The task is to build and run checks that confirm each field adheres to agreed formats, catches missing or duplicate entries, flags unexpected characters or line breaks, and produces a concise report of every issue found.

You are free to use Python (pandas, Great Expectations, or similar), Bash + awk, SQL import routines—whatever gets reliable, repeatable results. Everything must run headless on Ubuntu 22.04 and be callable from the command line so it can slip into our existing CI pipeline.

Deliverables:
• A runnable script or notebook with clear setup instructions
• A summary report (CSV or Markdown) listing all validation errors and their locations
• Brief documentation showing how to rerun the checks and adjust rules in the future

I am only focusing on validation for now; any cleaning or schema changes may follow as a separate phase once the error landscape is clear.