I think the problem is just the word, "sleepover". It immediately evokes thoughts of something that children would do in junior school. Even as a female who went to sleepovers at aged 16 I would still feel a little foolish saying that I'm going to a "sleepover". "Staying over at so-and-so's house with a few friends" means exactly the same thing but doesn't somehow sound as childlike.
There's nothing strange about it anyway.