"At about 8:30 p.m. on Christmas Eve, 1864, a huge fire engulfed the new Starr Hall dormitory leaving only the walls standing by the next morning." (Stameskin page 155). Ezra Brainerd was the first one of the scene, but later he called some other students among whom was Albee Smith (who later on left an article in the Undergraduate describing the events). Five people managed to savemost of the personal belongings and the furniture from the north side, the south side being impossible to go into. A strong wid threatened the Old Chapel as well, but nothing else happened. Ezra Brainerd in his article from the Middlebury College News Letter (Summer 1965) describes the events:

"But after three and a half years of occupancy the building was burned, leaving only the bare walls. It occured during the winter vacation, when but a few persons were occupying the building and when these few were all down at a Christmas evening service, of special interest as that year Christmas fell on Sunday. Soon after entering the campus on my return from town I noticed sparks flying from the south-west chimney of Starr Hall and soon after detected in the air the smell of burning paint. As I entered the south hall I noticed that the fire had eaten through the partition along the stairway. [...] The fire had evidently resulted from the falling of burning soot from an open door in the base of the chimney. I saw that I was absolutely helpless. I made the best use I could of a single pail of water from my room in the north entry. [...] The structure burned to the ground and the goods of all students who roomed in the south entry were totally lost".

Albee Smith also wrote an article on the fire:

"Soon ready hands were carrying out goods. My three companions had gone directly to the hose house, and they had a good, hard, tug to bring the cart through the snow, and there was considerably loud talk and orders given with little authority to back them up, save such authority as strong purpose and pressing emergency gives."

The trustees immediately voted the rebuilding of the dorm and president Labaree started a fund raise of $10,000 for this project. Surprisingly, the ones to respond were again the members of the community who made it possible to have Starr reopened in the Fall of 1965. "Middlebury was still at a large extent a town's college" (Stameshkin, page 156)