Web-Scraping Info Chatbot
Budget / Salary$10–30
TypeFreelance project
LocationRemote
Posted5 hours ago
I’m looking to have an AI chatbot built whose sole purpose is to answer user questions by pulling facts from publicly available websites in real time.
Core requirements
• Develop the chatbot engine—natural-language understanding, dialogue management, and response generation.
• Implement a crawler or API-based scraper that fetches content from selected public websites, cleans it, and stores it in a searchable format (vector store, database, or other approach you recommend).
• Design an information-retrieval pipeline that ranks and cites the most relevant passages so answers can be traced back to their sources.
• Provide a simple web interface or API endpoint so the bot can be embedded later in my site or app.
Key considerations
– Accuracy is critical; I want the bot to return verifiable answers, not guesses.
– Latency should stay comfortably below a few seconds per query.
– The solution must respect robots.txt and any site usage policies.
– Preferred tech stack is Python with libraries such as LangChain, FastAPI, or similar, but I’m open to what you do best.
Deliverables
1. Source code with clear docstrings and in-line comments.
2. Setup guide and requirements.txt (or equivalent) so I can run it locally.
3. Short video or written walkthrough demonstrating the chatbot answering sample questions.
4. Optional: suggestions for scaling the scraper and adding more sites in the future.
If you have prior experience building retrieval-augmented chatbots or working with web scraping/NLP pipelines, I’d love to see examples. Let’s create a helpful, information-rich assistant together.
Core requirements
• Develop the chatbot engine—natural-language understanding, dialogue management, and response generation.
• Implement a crawler or API-based scraper that fetches content from selected public websites, cleans it, and stores it in a searchable format (vector store, database, or other approach you recommend).
• Design an information-retrieval pipeline that ranks and cites the most relevant passages so answers can be traced back to their sources.
• Provide a simple web interface or API endpoint so the bot can be embedded later in my site or app.
Key considerations
– Accuracy is critical; I want the bot to return verifiable answers, not guesses.
– Latency should stay comfortably below a few seconds per query.
– The solution must respect robots.txt and any site usage policies.
– Preferred tech stack is Python with libraries such as LangChain, FastAPI, or similar, but I’m open to what you do best.
Deliverables
1. Source code with clear docstrings and in-line comments.
2. Setup guide and requirements.txt (or equivalent) so I can run it locally.
3. Short video or written walkthrough demonstrating the chatbot answering sample questions.
4. Optional: suggestions for scaling the scraper and adding more sites in the future.
If you have prior experience building retrieval-augmented chatbots or working with web scraping/NLP pipelines, I’d love to see examples. Let’s create a helpful, information-rich assistant together.
Apply on Freelancer →
Project sourced from Freelancer.com. Applications happen directly on the original platform — we never collect your data.