NSW Nationals Leader Gurmesh Singh has fiercely rejected allegations he had questions to answer about his knowledge of modern slavery risks on berry farms.