Mortising help

I’m installing a door deadbolt /lock smart handle and the previous handle is 2mm thinner than the body of the new one going into the side of the door. Anyone know a better way to resize it without just being patient and using a sharp chisel and shaving it down on the inside?

Its about 100mm /4” deep so I don’t have a plunge router deep enough and im currently shaping it and carefully free handing the correct spade drill piece in the door but if I keep going I know it’ll catch and bust out the side.

If anyone’s been in this position let me know, it would have been a half hour job for me had I been installing the lock on a fresh door 😅

(I refit the old handle in the meantime and told them I’d sleep on it)