I guess I am the unlucky one. About six years ago or so I had all of my gear stolen when I was about 10 miles into the backcountry. I was out all day scouting with just basic stuff with me, was gone from camp about 6 or 7 hours and when I returned all of my nice gear was gone. However, the A-hole did leave all their crappy stuff behind(walmart special stuff). This camp was about 150 yards off of a trail. Lost easily about 2k worth of gear. They basically just dumped their crappy gear on the ground in a pile and loaded all of mine up. I was a ways off the trail, but i think what allowed them to find me was my TNF mountain 25 tent, its basically bright yellow. Now I use all earth toned and camo colors and it blends in about 1000x better, hell sometimes i even have a hard time finding my camp again haha.
Nowadays I pack it all up and it comes with me if I am anywhere near a trail. If I am off trail and no where near a trail then I don't worry about it as much. I have also had a vehicle vandalized at a trailhead before. They didn't take anything but broke all my windows and slashed a tire.
It can happen, i agree its more likely to happen when your close to a road though. If I do leave gear anywhere now, i make sure to hide it thoroughly. So don't think it can't happen, and don't think that it wont happen because they will have to "pack it out" because the chances are they are already packing stuff themselves and if your stuff is nicer/better they could just help themselves.
Not saying its a frequent occurance. Moral of the story is to not camp in plain sight if you can help it, try to hide your camp to the best of your ability, and use earth toned colors. Hope no one ever has to go through what i did, because it sucks something fierce. Just now recently have managed to replace all of my gear with the high end type stuff i had before.