The cup of coffee in the morning is, in my opinion, an incredibly bad example for smart homes, because that is something for which I need a lot of sensors for little gain (is there a coffee cup under the machine, is water and coffee powder available) and I either have to clean and refill it myself anyway or it is enormously complicated (technically). It is possible, but those are automations that, in my opinion, are far from the first steps.
If you don’t have a regulated daily routine, then your shutters in the bedroom are simply operated manually (via a switch). But all the others in the house are regulated in a way that makes sense based on the weather and saves energy. Not having to reach for the light switch anymore is pleasant (if well configured). And with a single button press, the night scene could be activated – so that the lighting doesn’t go to 100% when you wake up at night but is pleasantly dimmed, especially if you go to bed at different times every day and cannot control it by time. With Keyless-Go, the car locks when you walk away from it and unlocks again when you approach and operate the door handle. Why shouldn’t that also work in the house?
And is your daily routine really so inconsistent, or are you convincing yourself of that? My mother is a nurse with shift work. She cursed the automated shutters... until she had to live with the crank again for a while. Does it really make so much fun to regularly water the flowers on the balcony, and reminders related to the house (like the garbage collection, the chimney sweep, or maintenance work) are only helpful for us; nobody else needs them, or you keep them in every resident’s calendar – my car also tells me when I need a service?
There are more than enough regular routines in almost everyone’s everyday life. NO, I do not have and will not automate everything, but the possibilities are sufficiently given for everyone. But there are meaningful applications that make life easier. Whether it is worth the money to someone and whether one is willing to engage with the new is everyone’s own decision.
With us, it is not the "fear" of something new and unknown, but rather the feeling that the house (or rather the automation) basically rules over us.
But as you write, it is a feeling that this now triggers in you. You don’t know whether the feeling would remain if you lived in an automated house and whether the feeling is justified. So maybe it really is the 'fear' of the unknown after all.