This study investigates the performance of ensemble machine learning weather prediction (MLWP) models for the medium-range forecasting of atmospheric rivers (ARs) over the north Pacific Ocean basin. With the proliferation of MLWP models, the need for process-based evaluation of these novel models becomes crucial. ARs are a useful lens through which to analyze MLWP models because they are filamentary features exhibiting large gradients in thermodynamic and kinematic fields that may not be well represented within the machine learning loss function framework. ARs are also motivating to study because they have large impacts to both water availability and flood management. We compare forecasts of ARs and their associated precipitation impacts across GenCast, NOAA’s Global Ensemble Forecast System (GEFS), and ECMWF’s IFS ENS during the winter season of 2022-2023. This work features grid point-based and novel, object-based verification of MLWP and NWP ensembles’ representation of AR systems. We also investigate the sensitivity of the ensembles’ precipitation forecasts to changes in synoptic features at earlier forecast lead times to investigate evolution of the forecast around key synoptic drivers. While GenCast performs well in many metrics, especially those relating to location errors, it struggles with representing the size and intensity of ARs, particularly strong ARs. Additionally, GenCast is much less dispersive than GEFS and a higher amount of its ‘dispersiveness’ comes from a small number of its ensemble members.