I think there is a difference in terms of legality of bringing things home from abroad and bringing stuff home from the beach that you live near... or maybe it's not legally different, but i think you're less likely to get caught committing local "theft."
On the other hand, the impact on the environment may be the same in either case - we're taking from the ocean, and it is simply not a zero net effect.
On the other hand, take something that doesn't "belong" where you find it, and let's say it lives for 8 years in your system when it would certainly have perished wherever you found it..... well, it lived a better life, but stuff has been drifting that way for millenia (perhaps), and in taking it, you have depleted the local system of some "food?" ...perhaps....
the ocean is all connected - it's something to think about.
The good news is that you don't have to buy 100 lb of live rock - buy 10 lb live and lots of great shape dead, give it time and all of it will be live.