Restore News Site Crawling

My custom-built news aggregator has suddenly lost almost its entire crawl rate—indexing is down by roughly 99 %. Pages that were previously picked up within minutes now sit unindexed, crawl speed has plummeted, and server crawl logs are peppered with errors. No theme, code, or content changes preceded the drop, so the cause appears to be technical rather than editorial.

The job is to track down the exact bottleneck—whether that lies in robots.txt, sitemap.xml, response headers, server-side redirects, canonical tags, duplicate URL parameters, rate-limiting, or anything else that can throttle Googlebot and Bingbot—and then implement the fixes. A full technical SEO audit, log-level analysis, and hands-on remediation are expected, followed by validation through Google Search Console and similar tools to confirm the crawl budget rebounds.

Deliverables
• Concise report detailing root cause(s) discovered
• List of code/server configurations changed, with before-after snippets where relevant
• Confirmation screenshots from Search Console or comparable logs showing crawl and index rates recovering
• Preventive recommendations so the issue does not reoccur

Timely indexing is mission-critical for a news site, so I need the crawl flow and visibility restored quickly and reliably.