4 years after construction, the office also demands land acquisition tax on the house

You won't be able to do anything about that. The crux of the matter is as follows:

We built a semi-detached house. The other half was built by the former owner of the entire plot who divided his land and sold half to us.
Since the land seller wanted to build a semi-detached house, the purchase agreement for the land at that time stated that we commit to constructing a semi-detached house with a predetermined general contractor (who is not further connected to the land seller).

You practically acquired both together and the plan existed from the very beginning. There could have been no plot without a semi-detached house, even the general contractor was already fixed, so you acquired both together. If that were not stated in the land purchase contract, one might still argue against it, but as it stands, you can no longer do anything about it.
It was very similar for us. We specifically concluded the construction contract and the land purchase contract separately, but in the very first exposé, both were addressed together. Thus, there was a connection for the tax office and we had to pay for everything.

Now one could of course blame the tax office again, but you know such things beforehand. The information is only a Google search away. Others have already listed the laws on this here.

Just a question about that: You signed a notarial contract to commission a company to build a house, without having any prior contract/binding plan with them?

Even I can't believe that. How is the tax office supposed to believe that someone would do something like that?
The construction company could have completely taken advantage of you.

But the situation is already sufficiently explained: you had notarized that you are buying a plot of land and building a semi-detached house with a specified company.
And thereby entered into a tied transaction.
([No legal advice])

Just a question: You signed a notarized contract to commission a company for a house, without having a prior contract/binding plan with them?

I can believe that. The seller of the half-property certainly got a small discount for imposing his general contractor on the other semi-detached house as well. It doesn't hurt him anyway ;)

Just a question about that: You signed a notarized contract to commission a company with a house, without having a contract/binding plan with them beforehand?

Even I can't believe that. How is the tax office supposed to believe that someone would do such a thing?
The construction company could have completely ripped you off.

That was actually the case and was due to timing, as my neighbor wanted to start construction promptly.
My planning including the price was only finalized weeks after the land purchase.
Therefore, my case is somewhat different from the others mentioned.

Yes, the general contractor could have ripped me off. But he didn’t.

I find it quite extreme to see this as a tied deal now and wouldn’t have expected that.
At this level, one might assume that every land purchase is a tied deal, since the builder is usually never really free (due to zoning plans or compulsory building after x years).

Yes, the general contractor could have ripped me off. But he didn’t.

You were lucky there!

I find it pretty shocking to see a tied transaction here and didn’t expect that.

Why is that?
You actually only wanted a plot of land and already had to buy a semi-detached house from a general contractor in the notary contract.
Just because you were so “naive” not to define the details beforehand doesn’t change that fact.

At this level one could assume that every land purchase is a tied transaction, since the builder is usually never really free (because of zoning plans or building obligations after x years).

It’s quite different whether a zoning plan requires me to build a gabled roof or if a contract states that I have to build with company X. ;)

The crucial question for the tying is: Could you have built with another general contractor?
And you already wrote the answer.

Seeing a tied deal here now I find pretty extreme and would not have expected it.
At this level, one might assume every property purchase is a tied deal, since the builder is usually never really free (because of the development plan or building obligation after x years)

The development plan gives you complete freedom to choose your building partner when concluding the purchase contract. The building obligation does as well. These are urban development instruments.

Maybe you just haven’t understood the tied deal yet. It means that when concluding the purchase contract, you are not free to choose your building partner. It has nothing to do with how you build, how expensive it will be, or when you build. It simply means that you buy the land with a building by partner XY. And that is also the basis according to the Real Estate Transfer Tax Act.
“If the acquisition process extends to a building yet to be constructed... the value of the land, deviating from § 157 paragraph 1 sentence 1 of the Valuation Act, is determined according to the actual circumstances at the time of completion of the building.”
==> Yes, you bought a plot of land with a house still to be built. That is what your contract says. Buy the land on the condition of building a semi-detached house with partner XY.
==> Yes, the real estate transfer tax is then based on the finished complete package land + house. But that is only determined at the very end – as in your case. If you sell the land beforehand, then “at the very end” means only the land purchase is taxable.

Could you have bought the land if you had not wanted to build with partner XY? That is the only question that matters. That has nothing to do with it being extreme.