You can dismiss the idea that the gravel or crushed sand under the stones gets washed away as an urban legend, otherwise our houses would also eventually stand crooked, since the same gravel is underneath (no crushed sand). However, our paved areas are bordered by curbstones, so nothing flows out, at least no substructure. What can happen with gravel in the joints will happen... the stupid ants carry it away over time... what color is the leveling material under your stones and what color is the joint sand? Crushed sand is gray for me, gravel yellow. I compacted an additional parking area with a 60 kg compactor from Bauhaus, everything works, you just can’t compact a 30 cm layer all at once. But nothing else than what you yourself have already suggested works, either remove everything and start again or do patchwork...