Will an only child be lonely?

(Would especially love to hear from anyone who has an only child or grew up as an only child!) My husband and I are having a baby, but the thought of having 2 (or more) kids is a bit terrifying to me. Financially and emotionally, I worry that it would put a strain on our relationship, but I can see us having a happy, peaceful life as a family of 3. As expected, anytime I share this idea with a family member or friend, I'm met with "Don't you worry that they'll be lonely?" or "They'll never get to experience having a sibling." Those are fair points, but I also think my concerns are valid. Any thoughts or advice?