The PhiladelphiaYoung Friends Association was founded in 1888 by members of the Hicksite branch of the Society of Friends to provide a forum for study and discussion of Quaker history and principles. Later, in the early years of the 20th century, their objectives broadened to include development and maintenance of a center for Quaker activity and fellowship in Philadelphia.

The organizational meeting was held in October 1888 at the home of Robert M. Janney in Philadelphia. Among those present were Janney's sister, Mary, Isaac Roberts, William W. Birdsall, David L. Lukens, Charles Paxson, Elgar M. Townsend, and other Hicksite Friends. One of the stimuli for its formation was the recent publicity surrounding the Young Peoples' Society for Christian Endeavor. The Quakers felt that a similar group was needed in the Society of Friends; in fact, the success of their effort in turn prompted the formation of over thirty similar “Young Friends Associations” in and near Philadelphia by 1893.

The Young Friends Association elected Janney the first President. The name of the organization invited much spirited discussion, but finally William W. Biddle's suggestion was approved because it was felt that the word “Young” would serve to attract younger Friends, while older Quakers would not be limited by age in their right to membership. A simple constitution provided for the forming of four committees: History, Literature, Discipline, and Current Topics. Although the group was not officially recognized by Philadelphia Yearly Meeting, meetings were held in Race Street Meeting House. The Association met monthly, and at each meeting a paper on some topic of interest was read by its author. Large meetings were held during Yearly Meeting. In 1895, the YFA of Philadelphia joined the General Association of Young Friends Associations, which merged with Friends General Conference in 1900 (See: RG4 Friends General Conference).

After a few years, it became apparent that a larger site was needed, for the Association as well as other Friends' activities. Through the generosity of individual Friends like Anna T. Jeanes, property was purchased at the corner of 15th & Cherry Sts. and a five story building was erected in 1899. In 1904, the structure of the Association itself was changed, and the old committees dropped in favor of a Sectional Committee having the power to undertake any projects which “might at various times seem advisable.” As this work expanded, it became necessary to become an incorporated body. This was accomplished in 1912. The first action of the new corporation was to take down the original building, and to erect on its site a new eight story structure, later called the Whittier Hotel. This location provided increased accommodation for Friends activities, including an auditorium, several committee rooms, offices, a restaurant, and a number of rooms for long-term residents and others visiting the City.

The Sectional Committee began to conduct pilgrimages, or visits, to local meetings, and to undertake service projects. By the late 1920's, however, this Committee asked to be discontinued because their work was being duplicated by a number of other Quaker organizations, including the Young Friends Movement. This was not immediately accomplished, but the focus of the group began to shift more and more to a social function and to the management of The Whittier Hotel. In 1945, the name of the group was changed to The Whittier, Inc. By the 1950s, the building was out-dated and no longer competitive as a hotel or social center. It was leased to Jefferson Medical College in 1955-1959 for use as a dormitory for nursing students, and thereafter to International House. In 1957, the group changed its name to The Whittier Association. In 1970, International House moved to West Philadelphia, and the Whittier was sold to the Philadelphia Redevelopemnt Authority in 1971. In 1974, its assets were merged with Trustees of Philadelphia Yearly Meeting, and the funds were used in part for financing the building of Friends Center.

The Young Friends Association was similar in function to the Friends Institute, an Orthodox organization, founded in 1880.