I skim. I have plently of pods in the main tank even though I skim, the fuge has to go through a pump to the tank, and I have a mandarin.
So I don't think skimming removes too much, but the skimmer is right before the refugium on my system, so the water has to go from the fuge, to the main tank, through another tank, before it gets to the skimmer.
Maybe there would be more stuff in the main tank if I didn't skim, but I am happy with the way things are.
I also feed phyto, so maybe that compensates for someof the skimmed out stuff.