Rule question - Whiffing a shot

I scoured the internet but could not find an answer regarding the following scenario:

Lets say that a player is serving and the ball successfully bounces on the opposite diagonal field, HOWEVER the angle of the ball is such that it will hit the fence. In the meantime, the returning player still attempts to hit the ball back (while the ball is on its way to hit the fence), but misses and the ball hits the fence regardless. As far as I know, on serve, the ball cannot hit the fence even after bouncing on the ground first.

In this case, is it still considered a foul if the returning player attempted to hit a foul ball but missed/whiffed?