This house was built by Nelson and Elizabeth Virden Fancher in the summer of 1854. It is the oldest still occupied house in Black Hawk County.

Nelson Fancher left Dubuque in 1850 for the California gold fields. He returned in 1853 and, along with his brother, he moved to Waterloo with a wagonload of general merchandise. They set up Waterloo's first general store in a long cabin at what is now the corner of Park Avenue and Commercial Street, site of the Waterloo Public Library.

Fancher apparently did well in California. He was able to purchase a 400-acre tract of land along the river south of Elk Run Heights. He was also financially able to marry soon after arriving in Waterloo.

During their first year of marriage, the Fanchers lived in a log cabin on this partially wooded tract. In 1854, they built this spacious house out of local walnut that was hauled to Waterloo for sawing.

The house was extensively remodeled in 1939, when the large columns were added onto the front entrance.