whether I have to involve the neighbor here from a legal perspective
You definitely have to involve him. However, if it is no longer possible to determine who planted the hedge, and there are no agreements known between the former and current neighbors, and you are sure that the trunks of all plants belonging to the enclosure are entirely on your property, you do not need the neighbor's consent to remove it.
So, as long as the boundary does not go through the trunk, it would not be a hedge lying on the border?
Yes, unless one or the other plant belonging to the enclosure lies entirely on the neighbor’s property.
If it’s 100% mine, then it will come down
If it is taken down, then pursuant to §14 of the Hessian Neighbor Law
the owners of the two properties are mutually obligated to cooperate in the construction of the (new)
enclosure.
If removal occurs without consultation, fulfilling this obligation could become a burden.