Agricultural price fluctuations create significant challenges for farmers and policymakers, making accurate price forecasting vital for agro-economic stability. This study introduces an Explainable Artificial Intelligence (XAI)-driven Random Forest model for forecasting weekly modal prices of agricultural commodities in Bangladesh and India. The model utilizes the Agricultural Commodity Price Forecasting Dataset, containing 23,093 weekly price records across Indian and Bangladeshi markets. Following rigorous preprocessing and feature engineering, multiple models were evaluated, among which the Random Forest achieved superior performance (RMSE = 397.07, MAE = 100.96, and R² = 0.9931), outperforming XGBoost, CatBoost, MLP, and LSTM. Integration of SHapley Additive exPlanations (SHAP) provides interpretability by identifying key influential factors such as Max Price, Min Price, Market, and Commodity Type. The proposed XAI-based Random Forest framework ensures both high predictive accuracy and transparency, offering valuable insights for data-driven decision-making in agricultural market forecasting.