The responses of tropical anvil cloud and low-level cloud to a warming climate are among the largest sources of uncertainty in our estimates of climate sensitivity. However, most research on cloud feedbacks relies on either global climate models with parameterized convection, which do not explicitly represent small-scale convective processes, or small-domain models, which cannot directly simulate large-scale circulations. We investigate how self-aggregation, the spontaneous clumping of convection in idealized numerical models, depends on cloud-radiative interactions with different cloud types, sea surface temperatures (SSTs), and stages of aggregation in simulations that form part of RCEMIP (the Radiative-Convective Equilibrium Model Intercomparison Project). Analysis shows that the presence of anvil cloud, which tends to enhance aggregation when collocated with anomalously moist environments, is reduced in nearly all models when SSTs are increased, leading to a corresponding reduction in the aggregating influence of cloud-longwave interactions. We also find that cloud-longwave radiation interactions are stronger in the majority of General Circulation Models (GCMs), typically resulting in faster aggregation compared to Cloud-system Resolving Models (CRMs). GCMs that have stronger cloud-longwave interactions tend to aggregate faster, whereas the influence of circulations is the main factor affecting the aggregation rate in CRMs.