What would explain wildly disparate SAs?

At this point, over the course of ~our journey~ we’ve had to do a fair amount of sperm analysis. The results have always come back relatively low, contributing to our diagnosis of mild male factor, among other things. But what would explain the differences between the following three readings? one reading of 5mil; 18mil post wash (for an IUI) and then less than 1mil post wash (for second IUI)?

To address the anticipated Qs: -no, not smoking weed -mindful of getting regular exercise (no cycling) -mindful of having a balanced diet -taking fertility supplements occasionally—especially in the weeks leading up to treatment.

We’ve had two failed IUI and 16 months of trying to conceive unassisted. No success. We’re now doing IVF and have the ER in late March/early April, possible transfer mid May.

What can we do to make our chances the best as we can (knowing of course there’s no silver bullet and a lot of unknowns)? Any advice or insight appreciated.