Is it enough if I only do this on the ground floor? Or should I better also remove the screed on the upper floor and attic?
You only need to insulate against the floor slab and the soil on the ground floor.
You can also do it upstairs, but more so because with a conventionally installed underfloor heating you can keep significantly smaller distances than with milling. From an insulation point of view, it is not that important, since under the warm upper floor there is also a warm ground floor.
Have you had a professional room-by-room heating load calculation as well as the installation of the underfloor heating calculated? With your specifications, e.g. regarding the desired room temperature or supply/return temperature of the heat pump?
If not, that is the first step anyway (e.g. from the engineering office Heckmann).