The wall smells chalky/stone-like - clothes too - need advice

Hello everyone, I am currently very desperate and hope that someone here has advice. We moved into our old building apartment in the summer, I already had the feeling from the beginning that there was a strange smell and blamed the smell on the previous tenant. Someone also came and examined one of the 4 rooms for mold – everything was below the threshold. The house is very old but completely renovated and very well maintained. One room has herringbone parquet, the others have floorboards. All rooms except the living room have new plastic windows, on which water also collects at 0/-1 degrees.
My problem: the exterior walls are stony/calcareous, unpleasant, I can't describe it better. I traveled a week ago, unpacked my clothes, and noticed that they had totally absorbed the smell. That was really disgusting! I could barely wear them and it’s driving me crazy. Does anyone have an idea what this could be and how I can fix the problem? I feel really uncomfortable when I wear an item of clothing that has hung there on the clothes rail. I have meanwhile moved most of it into wardrobes, but that hasn’t solved the problem. When I stand in the room and smell the clothes, I don’t notice the smell, but when I take them onto the balcony, breathe in deeply three times, I smell it again very strongly. I have also set up a hygrometer everywhere, humidity is at 55-60%, mostly around 57%.
What could this be?
Thank you to everyone for an answer,

I would suspect that due to the new windows there is no longer sufficient air exchange. Often a problem in renovated houses. Then a musty smell develops. I would try using a dehumidifier and airing out a lot. Were the exterior walls checked regarding their water (im)permeability?

Hello Rick, thanks for your reply.

We air out at least twice a day crosswise and the humidity is always below 60%. The dehumidifier arrived yesterday, we wanted to try that now. Would something like that be enough once or does it have to be used repeatedly? I don’t know this smell either and chalky/stony describes it approximately. The basement smells different again. Moldy fits there. When I enter the room where the clothes racks are, I don’t notice the smell at all. But the clothes smell completely. That is the strange thing. In another room I smell it again. But I can air out as much as I want there. According to the property management, everything is fine. The owners also live here in the house and the house is very well maintained. The walls in the hallway don’t smell. In the other apartments (where I’ve already been) there is no smell. Are there walls that “just smell,” or is that rather unusual? What really pushes me to the limit is that my clothes smell like that...

Were the walls possibly replastered or something? As long as the problem is not found, you will probably need the dehumidifier repeatedly. Ventilating twice a day can be too little in airtight houses. Since it doesn’t appear in other areas of the house, I now rather suspect a fresh wall covering/paint. An absolute last resort would be odor removal with ozone. But I would still postpone that by a long way. Even with that, it will come back if the cause is not gone.

We already used an ozone device in the summer when the apartment was still empty. However, only in the room where I also notice the smell when entering. I also considered removing the wallpaper and just painting the bare wall. But then I decided against it when I saw the colorful walls under the wallpaper. I know that the new windows have been in for about 5-6 years. When drilling, I come across various substances. Nothing works without a stone drill. Usually, red masonry appears at some point. Do you think it can get better with even more frequent ventilation? I still remember the summer when I blamed the smell on the previous tenants. But back then, I also had the windows open all the time and still smelled it. It has definitely gotten better.

To treat an apartment with ozone, you really need proper equipment. It might actually be behind the wallpaper. But then it is more likely musty. Difficult.