The efficient scheduling of electric vehicle (EV) charging is a critical challenge for modern power grids, characterized by its NP-hard complexity. This work addresses the Electric Vehicle Charging Scheduling Problem (EVCSP) with the objective of maximizing the number of satisfied charging demands under realistic constraints of charger availability and grid capacity. We propose a novel bi-level optimization framework. At the upper level, the assignment of EVs to chargers is determined using a Hybrid Genetic Algorithm (HGA) that is enhanced with a Q-learning module. This hybrid approach dynamically guides the mutation operator, enabling an adaptive balance between exploration and exploitation. For each candidate assignment generated by the HGA, the lower level solves an integer linear programming model to ensure optimal energy allocation without violating grid constraints. Computational experiments on benchmark instances demonstrate the superiority of the HGA, which consistently outperforms a classical Genetic Algorithm and a state-of-the-art Simulated Annealing method, especially as problem scale increases. The results confirm that the integration of reinforcement learning provides a significant advantage in solving large-scale, complex instances of the EVCSP.