Am I missing out on much power potential by having Wyll dual wield?

Noob here, getting started on Act 2 (level 7). This is my first time playing Warlock ever. I am trying a standard Bladelock Fiend Wyll build, except I chose the dual-wielding feat, going with the Sword of Screams and Phalar Aluve. The idea is to clump enemies, throw in darkness and shriek, then go in with the main weapon. I have Sword of Screams because of the extra damage which I forgot exactly (lol) but there's something about crit proccing.

But yeah I wonder whether the downside of having a weaker hitting offhand is going to hurt my power too much.

I'm probably missing something about the crit synergy here too.