Learning Without Critics? Revisiting GRPO in Classical Reinforcement Learning Environments

Workshop Paper · November 2025 · 1 min read

Theoretical and empirical study revisiting GRPO in classical RL environments, comparing it against PPO and analyzing the role of the critic.

Paper ⧉ Code ⧉

Learning Without Critics? Revisiting GRPO in Classical Reinforcement Learning Environments

We revisit Group Relative Policy Optimization (GRPO) in classical reinforcement learning environments, comparing it against Proximal Policy Optimization (PPO) and analyzing the role of the critic in policy gradient methods.