It isn't trespassing if you have the permission of the land owner, which is appears he may have had (though it was years ago, the owner doesn't remember any more, and no one thought to get it in writing).
It is a shame, though. Odds are, the natural progression of life and death would have handled this situation within a decade or so. It's not like the owner was using that land...
What is the point of rules that are not enforced?