IMO for the average reef hobbyist, there is far more benefit in all the harmful things being exported via continuous use of a skimmer than any downside from loss of trace elements. I think the great number of amazingly healthy reef systems out there these days that are using very powerful skimmers 24/7, speaks pretty well to that.
We also have to consider that many modern salt mixes far exceed natural seawater levels with many trace elements, so some extra exporting of these elements may actually be more of a benefit than a problem.
Also, skimmers are a powerful aeration tool that can help stabilize pH levels. In some systems, turning it on/off can increase daily pH swings - not a good thing.
As for phytoplankton, if you're dosing phyto it's probably a good idea to turn your skimmer off for an hour or two afterwards. But as for the notion of 24/7 skimming reducing phytoplankton levels in home aquaria, I think it's a non-issue. I don't think we have much (if any) sustaining levels of phytoplankton in our home systems to remove.
JMO