So I would also tend to go more towards white or something in the lighter gray range color-wise and maybe apply that somewhat to the existing building. Right now, the colors only match to a limited extent! Sure, a wooden cladding or other cladding could also work well, but then you would definitely have problems with the window.
A connecting element could theoretically be the brown of the windows, but painting the annex brown? I don't think that's perfect either.