In a statement Sunday, the caucus asked the Frankfort lawmaker to resign immediately and removed him as minority whip.
Spectrum News of Louisville reported Saturday the alleged incident happened at the man's home. Carroll, who denied the allegations, didn't immediately respond to a request for comment by The Associated Press.
After the alleged incident, the man went to state police and played a taped conversation of the meeting. No charges were filed.
There's no evidence of groping on the tape, which shows the man refusing Carroll's sexual proposition and Carroll then backing down.
Carroll was Kentucky's governor from 1974 to 1979.
