Philip Milledoler Brett was born in Newark, a graduate of the New Brunswick Theological Seminary, and the recipient of an honorary degree from Rutgers in 1916. A descendant of the Reverend Philip Milledoler, president of Rutgers College from 1825 to 1840, Brett was graduated from Rutgers in 1892. As an undergraduate he was the winner of the sophomore and junior oratorical prizes and was an active participant in athletics. He was captain of the football team in 1891, a member of the basketball team, and doubles champion with Clarkson Runyon Jr. Receiving his degree with honors, he was awarded an LL.B. from the New York Law School in 1894.
At the time of his selection as acting president, Brett was a partner in the Manhattan law firm of Nevius, Brett and Kellogg. In 1906 he became a member of the Rutgers College Board of Trustees, and served as chair of the committee on honorary degrees, a member of the executive and preparatory school committees, a trustee of the Epsilon Chapter of Delta Phi fraternity, and a member of its board of governors.