Perfect foresight hydroeconomic optimization models are tools to evaluate impacts of water infrastructure investments and policies considering complex system interlinkages. However, when assuming perfect foresight, management decisions are found assuming perfect knowledge of climate and runoff, which might bias the economic evaluation of investments and policies. We investigate the impacts of assuming perfect foresight by using Model Predictive Control (MPC) as an alternative. We apply MPC in WHAT-IF, a hydroeconomic optimization model, for two study cases: a synthetic setup inspired by the Nile River, and a large-scale investment problem on the Zambezi River Basin considering the water-energy-food nexus. We validate the MPC framework against Stochastic Dynamic Programming and observe more realistic modelled reservoir operation compared to perfect foresight, especially regarding anticipation of spills and droughts. We find that the impact of perfect foresight on total system benefits remains small (<2%). However, when evaluating investments and policies using with-without analysis, perfect foresight is found to overestimate or underestimate values of investments by more than 20% in some scenarios. As the importance of different effects varies between scenarios, it is difficult to find general, case-independent guidelines predicting whether perfect foresight is a reasonable assumption. However, we find that the uncertainty linked to climate change generally has more significant impacts than the assumption of perfect foresight. Hence, we recommend MPC to perform the economic evaluation of investments and policies, however, under high uncertainty of future climate, increased computational costs of MPC must be traded off against computational costs of exhaustive scenario exploration.