Project overview
Building an MVP & Need Guidance on Choosing the Right Tech Stack We're looking for advice in regards to which path we should take for an MVP that we are working to build. Here is the concept: workflow We're pulling data from various e-commerce sites and then we want to email some of that data to our subscribers based on the preferences they set up in their WordPress profile. Right now we're close to a functioning prototype but I can't help but feel like there's either 1. A completely different and easier or better way to do this or 2. We're missing some things in our tech stack that would make this run more efficiently (like MongoDB or an entirely different system from AWS etc.) The current setup is as follows: Scrapers built on scrapy library & managed using scrapinghub Data goes from there to S3 User data is stored in the WordPress database and goes from there to S3 Lambda is housing some queries and redshift is filtering down the database tables to match the exact preferences of the user/subscriber Then the idea is that the user data would go into S3 and then get formatted using some code and sent out to subscribers via SES. Just to recap: Ecom data that matches user preference data would be stored, formatted, and sent out to subscribers via email.