Automated Code Smell Detection And Refactoring Using Reinforcement Learning Techniques

Abstract

Maintaining code quality in complex software systems is a persistent challenge, largely due to the prevalence of "code smells"—sub-optimal design choices that increase technical debt. Traditional static analysis tools for detecting these smells are often rigid and lack the semantic understanding to provide context-aware refactoring suggestions. To address these limitations, this paper proposes a lightweight, automated framework for code smell detection and refactoring using Reinforcement Learning (RL). The proposed system employs a Q-learning agent to analyze Python code snippets by leveraging structural metrics derived from Abstract Syntax Trees (ASTs), including Lines of Code (LOC) and cyclomatic-like complexity. The agent learns an optimal policy by interacting with the code, selecting actions such as EXTRACT_METHOD or RENAME_VARIABLE to mitigate detected smells like Long Method and God Class. The agent's decisions are guided by a reward function designed to maximize improvements in code quality and reduce smell severity. Experimental results demonstrate the framework's effectiveness, showing a measurable increase in the average code quality score from 0.768 to 0.795 post-refactoring. Furthermore, the system successfully resolved 100% of the identified issues in the test scenario, with the RL agent achieving a 60% success rate on individual refactoring actions. This study validates the feasibility of using RL as an adaptive and intelligent approach for automated software maintenance, paving the way for more sophisticated, self-improving software engineering tools.