New single-family house construction through partial mortgage swap or let it be?

I see no problem. The situation with the properties is a bit more complex, but for the bank certainly no problem. I would do it. If the interest rate starts to fall again in 2024, the currently available plots of land will quickly come back on the market and you will regret it! Do it!

That should not be the case with regard to anytime. Because that only applies to object exchange or collateral exchange. If there is no exchange, that is a new object and you settle the termination early, then the VFE applies.

At an interest rate of 1.7%, the prepayment penalty should be manageable. I am only surprised that there is no appointment for the collateral exchange.

Short answer to your questions.

The bank is more relaxed with me because there are still a few capital investments financed there. Additionally, the loan is still being serviced and the money is in a daily allowance account with them, which is blocked. So they do have 100% security. The VFE is actually currently 0 or a small fee that is negligible, but of course this can change if interest rates fall.

We would sell our house without a real estate agent, we always do. We have really taken the lower range of used properties and calculated rather conservatively. Agents would probably go for 900-950k, but I don't see that at the moment, so rather 750k. If it is more, all the better, it gives you more buffer for the construction.

At an interest rate of 1.7%, the prepayment penalty should be manageable. I'm just surprised that there is no date set for the pledge exchange.

The OP had mentioned something about a date. Somehow within 1 or 2 years.
I can well imagine that the pledge exchange is given a period of 2 or 3 years, provided the loan continues to be serviced. That is the case here.
And with €220,000 that will also be an amount that can weigh heavily on you.

The bank is more relaxed with me,

Hmm… how can you tell that?

The prepayment penalty is actually currently zero or just some fees that are not worth mentioning,

At the moment there is none either, since you have not terminated.
The worst part comes at the end.

You are mistaken. We had it calculated beforehand, of course, and at the moment it would only be a few euros processing fee, so which big problem do you mean? How do I know this? Because I still have 3 other properties as investments with the bank, so the process is more relaxed, they know me as a customer, and the procedures are much shorter.

At the moment, none is due either, since you have not terminated. The worst is yet to come.

The prepayment penalty applies if the bank cannot re-lend the loaned money at an equivalent rate. Since interest rates have risen, every bank is happy to receive 4% interest instead of 1.7%. At the moment, only a few € in fees are really incurred.