Building expert is right. Why would you need a building permit if the municipality/city can still come up with further requirements after it has been issued? You can also surely get the gentleman to respond by requesting his objections in writing. He then has to justify them and explain why he demands such things, even though the building was approved differently.
In cases of serious violations, a building permit can certainly be revoked, but because of color design!? That already sounds like boredom...
Best regards
Dirk Grafe