Do you both earn exactly the same amount to come to 2x 320 euros? Or how does the advisor arrive at this amount? Please have that explained to you once. I believe that he may be overshooting the mark a bit there.
One piece of advice applies so often in life: as much as necessary, as little as possible. In this sense, in my opinion, you should only pay into both building savings contracts as much as needed to achieve the maximum possible benefit (allowance and tax consideration). Paying in beyond that only makes limited sense (and leads back to the usual issue with repayment substitute services and the associated long period of high interest payments on the full amount).
The maximum Riester subsidy is 2,100 euros per person and year, from which the allowances are deducted. If you tell me both of your annual gross incomes (separately) and existing children, I can go into more detail. Calculating with an example now is not expedient.
If you each pay 320 euros monthly + basic allowance for 10 years, you will have about 2* 40,000 euros together after 10 years, which is somewhat more than the minimum you would have to have. Depending on the building savings contract, 40-50% of the building savings sum, so only about 60-75,000 euros. With this construction, you are aiming significantly too high. The saved part could be put into a higher repayment of the loan, which has the highest interest costs.
Actually, the bank should consider the building savings loan as subordinated within a one-stop financing. At this point, the terms should already improve significantly, since the building society usually goes into subordination. Definitely check this and inquire there about how the collateral structure between the bank and the building society looks in detail!
With the KfW, this cannot be stated precisely. Usually, the KfW is secured by the same land charge as the bank loan. The loan is also granted via this bank. Generally, banks therefore assume the same rank of loans, which then leads to this loan not being considered equity capital. Exceptions with banks may exist, but you could probably count them on one hand nationwide. It makes more sense to negotiate a discount on the KfW conditions. Although this makes significantly less of a difference, some banks sometimes give 10-20 basis points, but classic house banks tend to have a hard time with this. You usually find that with one or another direct bank or loan broker.