MLOps/DevOps Engineer for AI Inference Platform

DevOps / MLOps Engineer Needed for OpenVINO AI Inference Platform

I have a completed OpenVINO + FastAPI AI inference application and need an experienced DevOps/MLOps engineer to build the deployment, CI/CD, monitoring, and scalability layer around it.

The AI/inference code is already functional. The focus of this project is to make it production-style, observable, reliable, and scalable.

Required Work

1. Docker

- Production-ready Docker setup for the OpenVINO CPU inference service
- Environment-based configuration
- Health checks and proper container practices

2. CI/CD
Build a GitHub Actions pipeline:

Git Push → Tests → Docker Build → Security Scan → Image Registry → Deployment

Use tools such as GitHub Actions, pytest and Trivy.

3. Prometheus + Grafana
Implement application and infrastructure monitoring, including:

- Request rate
- Error rate
- Inference latency (P50/P95/P99 where practical)
- Inference errors
- CPU / memory usage
- Service health
- Container/pod metrics

Create a professional Grafana dashboard.

4. Alerting
Configure alerts for:

- High inference latency
- High error rate
- Service unavailable
- High CPU/memory usage

5. Kubernetes
Deploy the application on Kubernetes using manifests or Helm.

Include:

- Deployment
- Service
- ConfigMap/Secrets where required
- Health/readiness probes
- Multiple replicas
- HPA/autoscaling

6. Failure & Load Testing

Demonstrate real DevOps impact through controlled scenarios:

- Traffic spike → latency increases → alert → scaling → recovery
- Pod failure → Kubernetes recreates it → service recovers
- Application failure → monitoring detects and alerts

Use the existing benchmarking tools to measure actual performance. No fabricated metrics.

Preferred Stack

Docker, GitHub Actions, Kubernetes, Prometheus, Grafana, Alertmanager, Helm, Trivy

Argo CD/GitOps is an optional plus.

Cloud deployment is not mandatory; the core setup should be reproducible locally using Docker/Kind/Minikube or equivalent.

Deliverables

- Docker configuration
- GitHub Actions CI/CD
- Prometheus configuration
- Grafana dashboard
- Alert rules
- Kubernetes/Helm deployment
- HPA
- Security scanning
- Architecture diagram
- Setup and troubleshooting documentation
- Failure/load-testing demonstration

The goal is not simply to install monitoring tools, but to demonstrate how DevOps improves deployment, observability, reliability, incident detection, and scalability of an AI inference service.