Noise in the outdoor area with controlled residential ventilation - noise regulation regarding time?

basically, any point can be the reference point

The locations of the relevant immissions sites, to which the immission guideline values are to be applied, are clearly and unequivocally defined in the TA Lärm. There is nothing more serious to say about this.

For comparison: 45dB(A) prevail INSIDE a quiet(!) apartment.

Well, it’s not really quiet, on the contrary. I have already conducted expert measurements in luxury hotels and apartments where the residents were disturbed by noise from a nightclub operation. Here, the measured levels (not assessment levels!) were well below 30 dB(A), but nighttime rest was out of the question because of the impulsiveness of the noise. In a quiet apartment, nighttime levels should be about 25 to 30 dB(A), but not significantly higher.
Back2Topic: If the neighbor feels disturbed, he must provide proof that it is actually too loud. This proof is not provided with any smartphone apps, but with the involvement of a §29b measurement site.