When should I use AWS and when not

I have a question which is in my mind since a long time now. I want to start a new project and will also try to attract business attention with it just to say that it is no private or internal company product in the end.

Before starting to develop I am also thinking about the environment on which I will run my application later on. In the past I always got a VPS server and just deployed my app + database + elasticsearch/redis/whatever there.

Recently I got more into AWS and pretty much like the idea of it. But on the other side it seems more expensive.

Let's say I deploy my app on a EC2 instance with a RDS instance as a database and S3 bucket for my static files.

That would be 3 or 4 times the money I would pay for a standard VPS server which is a lot starting out as a project where it is unsure if it will gain public attraction or not.

But also I know in case the app starts to grow it is much easier to handle it with AWS compared to a VPS.

Which do you guys choose in which situations? Is it advisible to start using a VPS and migrating to AWS later on?

Thanks for you answers.