This letter considers a reconfigurable intelligent surface (RIS)-aided indoor visible light communication system, where a mirror array-based RIS is deployed to assist the communication from a light-emitting diode (LED) to multiple user terminals (UTs). We aim to maximize the sum-rate in an entire serving period by jointly optimizing the orientation of the RIS reflecting unit, the time fraction for the UT, and the transmit power at the LED, subject to the communication and illumination intensity requirements. To solve this high-dimensional non-convex problem, we transform it as a constrained Markov decision process. Then, a soft actor-critic (SAC)-based deep reinforcement learning algorithm is proposed with the goal of maximizing both the average reward and the expected policy entropy. Simulation results prove the effectiveness of the proposed SAC-based joint optimization design in improving the sum-rate and long-term average reward.