Are the apartments properly separated (e.g. in terms of access), so you don’t have to go through one living unit to get to the other? For renting out, this is basically an absolute must. For "we want to use this as a family as 'more space' with our 6-year-old and (let’s say) 2-year-old child" it’s rather impractical. Do the parking spaces somehow fit? Do the tenants also somehow have an outdoor area or do you say "no, there just isn’t one. Period." (Not so unusual for rental apartments nowadays, even if most tenants would probably wish for at least a small terrace or balcony if they had the choice – but there are also plenty of rental apartments without one)
Personally, if you actually get along reasonably well with the tenants, I’d also feel a bit guilty about giving them notice due to personal use (and it would sort of be like "actually the 100 sqm we have is enough…") But of course, it always depends. It can also simply happen that they want to look for something new anyway (e.g. because of family growth) and that would not be a problem at all.
Otherwise, this is roughly how I grew up: an old farmhouse where at some point the barn was converted into living space. It was then basically 3 living units and over the years we (or at some point just my parents) moved around a bit (at first the family lived in one, then more or less two and made a connecting doorway. Meanwhile the connection is closed again, the two living units are now rented out and my parents have moved into the third). You can do it, but somehow it’s always a bit stressful. Since it wasn’t all planned that way from the start, there was always something to organize and since it is ultimately an old big house, quite a bit has gone into renovation and renewal over the years. Which is still always a bit more than with a smaller single-family house.
Having a living unit for ourselves as teenagers was definitely a huge luxury, but it also meant that we kind of became the youth hangout for the neighborhood. And I’d say the consequences of that were not always positive, without wanting to go into all the youthful misdeeds of my brother and me here ;)