Switching from gas to solar / photovoltaic with / without heat pump

Yes, of course, photovoltaics is simply power supply. Whether it then powers the heat pump or your computer doesn't matter ;)

Can I actually start with the photovoltaic system and install the heat pump and storage afterwards? We have the problem with the heat pump that we have a very small garden and it would be "difficult" to implement outside. Also because of the appearance. Inside I only have one corner that would fit. But that still has to be clarified

That is always possible. The heat pump has already been answered. Here only in detail to clarify: Are there cheaper heat pump tariffs and is it worthwhile then (with approx. 10 cents delta) regarding cascade connection, to obtain the roughly 2/3 heat pump consumption that is not covered by photovoltaics more cheaply? Would there then be a meter place for that or would a conversion be necessary? In the last case it would be too expensive. If there is little space and no outdoor air-water heat pump is to be used, an expensive deep drilling would be possible with approval. Being fancy costs. ;) But that would also have a good annual performance factor compared to air-water heat pumps, which means lower consumption. That is, more of the demand could be covered by photovoltaics.

Regarding the storage: Possible, but it can also be prepared. There are hybrid inverters where a storage input can be unlocked. If you buy the storage at the same time, you can reclaim the VAT as with the photovoltaic system and with DC-side storage you can also depreciate it (if intended or the system is very large anyway). And pay attention to any available funding programs!

If there is little space and no outdoor air-to-water heat pump is to be installed, an expensive deep drilling would be possible with approval. Being fancy costs. ;) However, it would also have a good annual performance factor compared to an air-to-water heat pump, which means lower consumption. That means more of the demand could be covered by photovoltaics.

Our garden has about 80 sqm of green space.

How much space is needed for a deep drilling? Actually, only for the equipment, right? Ground heat collectors are out of the question.
Also, a deep drilling would be a huge mess. My better half would kill me ;)

You don't really need much space and you can build over it again later. However, you must maintain a certain distance from the boundary. And of course, you need to be able to get there with heavy equipment.

Would it also be possible to do the deep drilling in the driveway?

I think so. Mine is planned in a strip between the house and the boundary, depending on the exact placement either the garden house will be there or a paved path. Suitable locations can certainly be suggested to you by a drilling company. More important is to clarify whether you are allowed to drill at all. In the water protection area, this is not permitted.