Java & Python Data Pipelines

I’m putting together a data-processing pipeline that blends the strengths of Java and Python. The goal is simple: pull data from several sources, cleanse and transform it, perform lightweight analysis, and push the results downstream in a form that is easy for other services to consume. Whether the flow ends up running in real time, on a schedule, or as classic batch jobs is still open for discussion, so versatility with both approaches will be appreciated.

You will design and code the core pipeline components, set up automated tests, and leave behind clear documentation so the in-house team can extend the work without a hitch. Modern tooling is expected—think Maven or Gradle for Java, virtualenv or Poetry for Python, Git for version control, and whatever lightweight orchestration or CI you prefer.

Deliverables
• Well-structured Java and Python source code
• Automated unit tests covering key logic
• A runnable demo (local or containerised) showing the pipeline end-to-end
• README and deployment notes that someone new to the project can follow in minutes

Acceptance criteria
• Sample data moves successfully through every stage with no manual intervention
• Tests pass on a fresh checkout of the repository
• Basic performance and error metrics are exposed for monitoring

If you have a track record of building reliable data flows and enjoy switching between Java’s robustness and Python’s flexibility, I’d like to hear how you’d approach this project.