That's exactly the point. I am well aware that such behavior is not environmentally conscious. However, I wouldn't categorize this aspect under comfort. Let me try with a different wording: Are there other advantages of a ventilation system with heat recovery besides recovering heat, compared to a system without heat recovery?
Please try sitting in front of an open window at -5 degrees. Do you seriously think a turned-up heater can compensate for the loss of comfort caused by the open window?
There's a draft, it's cold, you catch a cold, etc. With a controlled residential ventilation system, the air exchange rate is lower, which reduces the effect.
But ice-cold air still enters the house.
My contribution was meant ironically...