Assuming you now take the smaller house so that you have paid it off in about 10 years and can then build a single-family house, I assume that you will also start family planning during this period, and then it will get tight in the house. Also, always remember that you have to get rid of the house again, and the question arises whether this size with the layout is in demand in your area, and if so, what you will get for it (after 10 years it usually starts that you have to invest something again for renovations).
I would not build just to secure the currently low interest rates; rather, consider whether to take out a higher building savings contract to secure the interest rates and thus also build up equity until it is time to build in 10 years, and in the meantime move into a more suitable rental apartment.