John Potter built his house entirely with his own hands; he made every bit of carving, the nails, latches, and hinges. He started the house before the Revolutionary War, building it over another house in which he lived. The only parts of the older house that were torn down were those which hindered the progress of the new construction. He finished building when he returned from the war. The house stood in the `North Precinct,’ the part that was later set off as the town of North Brookfield. It was in the center of the village – near the meeting house, the schoolhouse, and tavern. There was no public or steeple clock for the town, so Captain Potter made one with three dials which he placed on the outside of his house for all to read the time as well as the day of the week and phases of the moon. The Potter House was moved to Storrowton in 1929.