Construction on the T.B. Scott Mansion began in 1884. The original owner, T.B Scott, owned the Warren Mill and was instrumental in getting the railroad to Jenny. He said that his house would be fit for a lumber king and he doubted if the house had an equal anywhere in the state. Unfortunately, Scott died before the house was finished.

Here begins the "curse" on the mansion. There is a tale that white men killed a indian chief's daughter. With the words, "Let this ground be sacred to my daughter's memory, and let it never do any white man any good," he put a curse on the hill where the mansion now stands. Scott was just the first of many owners to die. Countless other misfortunes, such as bankruptcy, fraud, and disappearance, have plagued the occupants of the mansion.

In 1919, the City of Merrill bought the Mansion and the surrounding 39 acres of property for $8,500. Soon after, the Holy Cross Sisters from the Merrill community moved in and remained there until 2006.