Neighbor has erected a 180cm high fence without final coordination

He is also planning a garden shed. However, when I look at the boundary development, the garden shed triggers setback areas and must legally be placed at least 3m (?) away from the property line, since he already reaches 15m of boundary development, right? I have roughly sketched it, proportions are not correct.

Almost correct. The garage with 9m already requires the entire boundary development to the street. However, he can still place 6m at another boundary. So the garden shed would have to be away from the street, but may still stand at your boundary.

Basically, the shed doesn't bother me since it's neither by a window nor visible from the terrace. I'm considering making a deal with him; his garden house doesn't bother me, and in return, his climbing frame with a tower, which I would place directly on the boundary, doesn't bother him. Or would I be at any disadvantage because of this?

The problem will be that 1. You are not the only one who has to agree then, but all neighbors with boundary constructions. 2. As far as I know, these neighbors have to take over the setback areas on their property. So yes, it should not be done.