http://zone4garden.com/growing-peach-trees-in-zone-4/
google "fruit trees zone 4" or something like that. This is one link which came up with the author saying he grows PEACH trees in zone 4. He says:
"Yes it can be done. It takes a lot of care. I live close to Lake Superior which has a tendency to create a micro climate. Very seldom does it get below 20 degrees below here. I have had some success with the Reliance peach variety. The peaches are a little on the small side. But what a treat!
Plant the tree in the spring. When fall comes around and the ground freezes place a layer of straw for mulch around the tree a couple feet, and about 3 inches deep. In the spring don’t remove the mulch. It will help the tree break dormancy a little late so the blossoms don’t freeze out.
Henry Fields has another Peach Tree out this year that caught my eye. I am going to give it a try. It is also good for zone 4. It is called Contender. I have listed the link below."
The authors of my major two book set on building an edible forest garden are based out of Keene NH, btw. I will repeat what I said before: what others in your area are growing is probably a very poor indicator of what CAN be grown by you. You have to do your own research and try some things.
I have come to realize that men are not born to be free. Liberty is a need felt by a small class of people whom nature has endowed with nobler minds than the mass of men. -Napoleon