The "rarity" argument regarding shards is pretty hollow when almost no one is actually reading shards, you know? I agree with your argument, generally speaking, but there needs to be more done to make players aware of when they've found a rare shard and when they haven't.
As it stands, you just pick up all of them, but it's a huge hassle. Meanwhile you have thousands of some and actually zero of others. If a shard dropped that I currently have zero of, I probably wouldn't even notice.