Dynamic Traffic Assignment (DTA) aims to improve network performance by enhancing the flow of vehicles throughout a transportation network, thereby effectively smoothing traffic. Traditional theories of traffic assignment are based on inaccurate assumptions that do not reflect actual traffic behavior. Additionally, the measurement of network performance using travel time has limitations that hinder real-world applications. In this study, the performance of a transportation network is measured by applying a macroscopic fundamental diagram (MFD). This is achieved through conducting simulations that accurately reflect real traffic behavior. Based on the MFD, understand the state of the network under different traffic assignment methods, with a particular focus on the concept of network heterogeneity. Show that the impact of network heterogeneity on network performance can be represented by the variance of the network's volume-to-capacity ratio (V/C). Taking this into account, this study examines approaches for enhancing network performance. A heuristic algorithm and a reinforcement learning model were developed to mitigate network heterogeneity. As a result, a 6.5% and 8.5% increase in network capacity, and a -61.2% and -58.7% decrease in total travel times were observed, respectively. Specifically, the reinforcement learning (RL) model utilized the concept of marginal utility, which determines the extent of network heterogeneity reduction per vehicle, to facilitate efficient traffic assignment. Simulations assuming a connected vehicle (CV) environment confirm that the effectiveness of the strategy is maintained in a limited environment, enabling the application of the developed model to the real world. In particular, since the RL model was developed based on a heatmap that reflects the appearance of the network, it is expected to be easily applicable to an expanded network in the future. All things considered; this research is expected to contribute to the development of efficient traffic assignment strategies.