4 years after construction, the office also demands land acquisition tax on the house

Off to the lawyer. The circumstance, as you described it, is not exotic!

From my point of view, no real estate transfer tax should be incurred. An offer for house construction does not establish a tax liability.

But the way to the lawyer is likely to be expensive as well. We are talking about 10k with a dispute value of maybe 300,000 to 400,000 euros if the legal dispute goes to court.

I have always assumed that tax is only due for what is also notarized. But things are getting wilder here in this country.

This meant I was free to decide WHEN (I could have also built only two years after the neighbor) and HOW (there were significant changes to the design) I build my semi-detached house on it.

But the when and how do not matter, but with whom. What only confuses me is that it was actually a private seller who initiated by saying, buy, but then build with GU y. I have always understood a linked transaction to mean that the GU initiates something, i.e., offers the house construction on plot x, so the process must go the other way around, meaning the offer must be available.

The OB is more difficult to answer. Theoretically, I could have sold the property again after x years without building.

Does that change anything about your obligation to build with GU Müllermeier? Get yourself a lawyer. An initial consultation is not expensive, and afterwards you can still decide to waive the objection due to lack of prospects. I can understand that you are frustrated. But this clearly constitutes a tied deal.

With that, I was free to decide

No. You bought a property with the explicit and unchangeable condition that you MUST build with construction company X. No choice. The question of exactly what the house looks like and when you build it does not matter here.
You were not free to decide when signing the contract. The purchase was linked (this is where the term tied transaction comes from) to the mandatory obligation to commission construction company X.

The only way out of this, at best, is some legal trick that has nothing to do with the actual facts. My opinion.

You have presumably also built with the contractually named [GU]. Then it is de facto a tied selling transaction.

Theoretically, I could have sold the property again after x years without building.

Then the tax office would surely not make any claim against you.