Short-term forecasting—from minutes to days ahead—is essential for electric power grid operations. Numerical weather prediction (NWP) models form the backbone of load and renewable generation forecasts used by grid operators. One of the most used NWP models by US grid operators is NOAA's High Resolution Rapid Refresh (HRRR) model. However, NOAA has been developing a next generation 3-km NWP model known as the Rapid Refresh Forecast System (RRFS). There is limited information available regarding the performance of RRFS for energy applications. Here we evaluate forecasts of global horizontal irradiance (GHI) from NOAA's HRRRv4 and the experimental RRFSv1 against observations from the SURFRAD network in the Continental US during the spring and early summer of 2024. We find that HRRR produces more skillful GHI forecasts than the RRFS Control at all SURFRAD sites during overcast days due to high-biased GHI forecasts. Conversely, RRFS forecasts on partly-cloudy days are generally more skillful than HRRR forecasts, while the two models perform similarly in clear conditions. Analysis of RRFS's 5-member perturbed physics ensemble reveals that: (1) All RRFS ensemble members share the control member's overcast bias, (2) The RRFS ensemble is under-dispersed, missing both high and low GHI events, and (3) Despite shared biases among RRFS forecast products, an ensemble-mean of the six RRFS variants and the HRRR reduces GHI forecast error by 10-20% across all sites relative to the single HRRR deterministic forecast. Finally, an ensemble mean using dynamically-assigned weights determined by the forecasts themselves should further reduce forecast error.