The Penfield Friday Club was formed by nine women who first met in 1939 as a social and literary group. The club was the nucleus of the group which met in 1941 to form the Penfield Free Library. The grand opening of the Penfield Library was held on February 25, 1942 in a tiny house on Liberty Street. On June 20, 1952, the Penfield Library became the first suburban library in the county to join the Monroe County Library System. Until this time, all Penfield Library workers had been volunteers.

By September of 1985, the library had outgrown its space multiple times and had moved to the renovated Baird Road School, where it currently resides. In 2000, an addition of more than 10,000 square feet created a children’s area, a story time room, quiet study rooms, and increased space for the adult collection.