I've had three of these turn up since yesterday. I'm thinking they came in on my Xenia. Spent two hours on seaslugforums but couldn't get a perfect match.
The picture isn't the best but I would say it's Phyllodesmium crypticum or a species of Phyllodesmium. They prey specifically on Xenias so yes they are a pest.