If I understand correctly, you have a hybrid between a pure general contractor and individual awarding. You make a basic contract with a house construction company, but the individual billing takes place directly between the subcontractor and the client. Something similar was also offered to me by a house construction company at the time.
If that is the case, the question is, was the acceptance between you and the site manager or also between you and the "subs"? The former suggests that the invoices are correct, the latter suggests that the invoices would need to be changed. However, we are all not familiar with the contractual arrangements.
This is definitely no longer a layperson's question suitable for a forum. Legal advice on tax issues is reserved solely for tax advisory professions.
If you cannot get the invoices changed, in my opinion, you should consult a tax advisor or a specialist lawyer for tax law. Since this probably does not involve small sums for you either, I could imagine that the additional effort might be worthwhile.