El Nino Southern Oscillation (ENSO) is one of the most important climate variabilities on an inter-annual time-scale. We aim to find out whether ENSO frequency will change in a changing climate. We analyse two ensembles of General Circulation Models that participated in the Coupled Model Intercomparison Project Phase 6 (CMIP6) and an initial-conditions ensemble of the MPI-ESM-LR model. We identify the uncertainty caused by natural variability by comparing 120-year time-series of the pre-industrial control and the 1-percent CO2 simulations for the CMIP6 ensembles. We found that the multi-member mean for all ensembles predicts almost no change in ENSO frequency, but the uncertainties are large, and that most of the inter-member variability can be attributed to natural variability. This means that the impact of inter-model differences might have been overstated in previous studies. This makes it impossible to make reliable predictions of changes in ENSO frequency based on 120-year simulations.