Caroline County was the home of statesman Edmund Pendleton and the childhood home of brothers General George Rogers Clark, the Revolutionary War hero; and William Clark, of the Lewis and Clark Expedition.

During the Civil war more than 1 million men marched or camped in the county. Confederate troops under General George E. Pickett fought Union troops near Milford in 1864. Confederate General Stonewall Jackson died in Guinea after being shot by his own troops at Chancellorsville.

John Wilkes Booth, the assassin of President Lincoln, was allegedly shot by federal troops in Caroline County.

John Taylor, another Caroline native son, was a celebrated Revolutionary War soldier, a member of the Virginia House of Delegates, three times chosen U.S. Senator, and a Jeffersonian Democrat.

Caroline County is also the birthplace of the great Secretariat, winner of the 1973 Triple Crown and record holder for the Kentucky Derby and Belmont Stakes.