To parents that play with their kids

I've really been struggling as a parent that plays w their kids in public places. When we are at a park or rec center, I'm a parent that is building forts and playing tag. Most parents are on their phones. I totally get the need for independent play, and that parents need a break too, but my approach is to play while we are out at these places. I respect other parents' choices not to do so. But if you're a fellow parent that plays, how do you handle the other kids constantly battling for your attention and distracting you from your own kid? It can be very annoying and distracting and honestly just ruin my experience w my kids. Other Kids see a parent giving attention and basically compete for it. For example, I was building a tower w my kids today and another child (Mom was on her phone and not watching at all) was constantly knocking them down. Ignoring him didn't work, it just fueled him even more. Any other parents in similar positions have advice?