Equipment shed on a non-buildable boundary. May I do it?

From this response from the office, I do not read any prohibition on the use of your circus wagon. It refers solely to the base plate; you would therefore have to remove it. Such a circus wagon or similar stands on wheels and is not connected to the ground. It "rests on the property due to gravity," but as long as you can move it without technical aids, it is allowed, as the office writes. A donkey, for example, is not a technical aid, nor are four men pulling the thing away, and thus it would be exempt from approval according to their official letter. Since they have named exactly this as the reason for the rejection, you can also refer exactly to this. They will not admit it, and they will certainly be annoyed, and they might even order you to dismantle the base plate. But it is certain (according to their justification) that you may set up a similar wagon there as long as it can be moved without technical aids. You cannot be forced to demonstrate it, but it is apparent that it could be done by animal or human power. Abroad we have seen oxen pulling small trucks or strong men pulling steamrollers. If you have the nerves for it, go ahead, but be sure that he will be a sore loser. I also do not believe that the man from the office was there by chance...... Consider whether you want to reply to him that you are now acquiring such a wagon which meets his required conditions, namely that it can be moved without technical aids... and thank him for the approval. Then get such a part and have the seller confirm that this wagon can also be moved without technical aids. The wording from the office is "can". So you are allowed to tow it there with the car, it just has to be possible to do it without technical aids. I am curious to see how it will continue.....

So the prohibition was communicated via email.

Hard to say whether this already constitutes an administrative act against which legal action can be taken, especially since you mention several emails. Legal advice would be recommended here.
Depending on the content and wording of the emails, a final request with a deadline might suffice, in which you refer to the legality of your project according to §23, para. 5 of the Building Use Ordinance and insist on naming the specific public or neighboring interests. Depending on your willingness to take risks, you proceed with your project after the deadline expires.

Development plan as PDF is online

I found it, but indeed no references to ancillary facilities. Apparently, the authority is displaying royal Bavarian authoritative behavior. In some municipalities, it seems they have not yet realized that Bavaria belongs to the Federal Republic of Germany and is therefore subject to federal laws and the resulting regulations. A discretionary decision is not dependent on the mood of a clerk but requires balancing public, neighboring, and builder interests. If ancillary facilities outside the buildable areas are not desired, this must be mentioned in the development plan, and the balancing process must have taken place during the preparation procedure and been reflected in the justification for the development plan. If this is not the case as here, very strong arguments are needed to deny approval of an ancillary facility.

Smart, fits much more anyway.

Maybe. Maybe not, since a pool is also an ancillary facility according to §14 of the Building Use Ordinance, and the dismantling is presumably much more complex than the removal of the base slab.