The vulgar idea that "riding the goat " constitutes a part of the ceremonies of initiation in a Masonic Lodge has its real origin in the superstitions of antiquity.
The old Greeks and Romans portrayed their mystical god Pan in horns and hoof and shaggy hide, and called him "goat-footed." When the demonology of the classics was taken up and modified by the early Christians, Pan gave way to Satan, who naturally inherited his attributes, so that to the common mind the devil was represented by a he goat, and his best-known marks were the horns, the beard, and the cloven hoofs.
Then came the witch stories of the Middle Ages and the belief in the witch orgies, where it was said that the devil appeared riding on a goat. These orgies of the witches, where, amid fearfully blasphemous ceremonies, they practiced initiation into their Satanic rites, became to the vulgar and the illiterate the type of the Masonic mysteries; for, as Dr. Oliver says, it was in England a common belief that the Freemasons were accustomed in their Lodges "to raise the devil." So the "riding of the goat," which was believed to be practiced by the witches, was transferred to the Freemasons, and the saying remains to this day, although the belief has long since died out.
- Source: The National Freemason - 1873
The notion of riding the goat is still used by many masons in kidding a potential candidate especially when that candidate is a personal friend. So common is this inside joke that it has been used in at least one Masonic poem: