When the two then move upstairs, they are old enough to sleep through the night or to manage stairs alone at night
I have a 6-year-old at home.
The way he wanders through the apartment at night when he has to go to the bathroom, it will take a few more years before he could manage stairs.
It will only take 2 more years until both children are old enough to sleep upstairs, until child 1 hits puberty and chats with friends on weekdays until 11 pm and then comes up with the idea "oh, I still have to shower. And then blow-dry my hair." You’re sure to be annoyed if you want to sleep.
3 years later they come home drunk from the town wine festival and have to go down the stairs in that state when they want to go to the bathroom.
A living floor without a toilet would be a deal breaker for me.
Otherwise, I’m not a fan of "But think about the resale value!", but here please bear in mind that many 4-person families will immediately rule out your house.