Sounds more like operator error to me.
I have the same set-up, Mossberg 12ga. and 50cal M/L for the 500 platform.
It shoots as well as I do, so I'm guessiong your "neighbors" need to work on sight-picture, sight-alignment, and trigger squeeze.
i have no doubt that its user error. they are just the type of hunters who dont pratice or sight in there gun prior to season. i remember the first year they hunted there i ask how it shot. one of them said "we didn't see any this morning"