I have an app that a user can upload an excel sheet of analytics data to S3. I am wanting to trigger a Lambda function on upload to do some data processing and then write the analytics to a client’s organization’s database (I am using Aurora). Eventually we will be capturing live clickstream data but for now we are just using generated reports.
My question is is this best practice to just have all the analytics in a database with thousands and thousands of events in as many rows? I can see that your table supports a maximum of … a little over 4.29 billion rows. but does that mean I can just pile them into a giant table until then? If I am potentially getting 50k rows per month should I just not think worry about it until I see a performance hit (if I even ever see one)? Or am I, as a newbie, just worrying over nothing?
Ideally I don’t just want to make this thing work, I want to learn how to make something that lasts and scales. And reading the docs on Aurora it sounds like this shouldn’t be an issue but I don’t know if I am just not seeing something that will become an issue.
Thanks for any advice and feedback!