Hi, the issue with water must definitely be kept away from the house. Otherwise, you will get problems with mold, etc. If it only works with a drainage layer of gravel, then you should listen to the specialist.
If the expert advises you to go for such a solution, then I would do exactly the same. Because you are building for the first time, but he knows much more about it.
I would rely on what the architect and the structural engineer say. They certainly know better than we do here and they probably don't want anything bad for you.