Well, the thing is actually really simple and it has already been said: if building and everything that goes with it is fun and now - to put it exaggeratedly - seen as a "hobby" (that’s the case with us, others can’t comprehend it at all), then go for it! But if it’s only about "bigger" and "more beautiful," without any real desire behind it, then I would definitely tend to improving the existing house. And please: I would like to see the house from which you couldn't get anything else out! Get yourselves a - good! - architect if you lack the imagination.