I´m thinking about to write an application will have to store a small amount of records per user (<300) but hopefully will have a lot of users (>>1000).
I did some research for a platform that allows starting small and scale if there is a need to do so and got stuck with App Engine, but I´m not sure if it is the right tool for it, especially the datastore.
How will I get it to scale if I have a User entity and a Message entity and store all users and messages in that entities? I think the amount of records in the entities will grow really big and filtering i.e. for all messages of a user will get expensive. Is that a problem or will Google handle that ? Do I have to introduce multitenancy and create a namespace for each user so I only see the records in the entities that relates to the user? Is there a limit for the number of namespaces ? What would be the right approach for modeling the data in the datastore?
I do not really have a clue how to handle the App Engine datastore and if its the right tool for me.
The App Engine datastore is explicitly designed to handle this kind of scalability. Queries execute in time proportional to the number of records returned, so fetching all a user’s messages will take the same time regardless of how many users there are in the system.