Matt Holliday drew a leadoff walk to start the winning rally. He advanced to third on Matt Adams' single. After Peter Bourjos ran for Holliday, Jhonny Peralta walked to load the bases. Hoover (1-10) then plunked Jay to force in Bourjos with the winning run.
Pat Neshek (6-0) pitched one inning of scoreless relief as the Cardinals won for the fifth time in six games and remained 2 1/2 games behind Milwaukee in the NL Central.
The Reds have lost four in a row.
Jay scored the tying run in the eighth on pinch-hitter Daniel Descalso's RBI double.