Understanding the impact of vegetation on organic matter content in sediments is essential for sustainable reservoir management and water quality protection. This study examined the relationship between land cover, erosion processes, and organic matter accumulation in the sediments of four small water reservoirs in the Republic of Serbia. Organic matter content was quantified and analyzed in relation to basin characteristics, including land-use composition, absolute and mean flow gradients, and sediment grain size distribution. Field sampling was conducted across the catchments of four small water reservoirs—Duboki potok, Resnik, Ljukovo, and Sot—with sediment samples collected from main tributaries and accumulation basins. A multi-method approach was employed, combining remote sensing for vegetation-cover assessment, granulometric analysis, organic matter evaluation via loss-on-ignition at 350 °C, and statistical correlation analysis to assess the influence of land use and hydrological gradients on sediment composition. The results revealed a strong correlation (R = 0.892) between forest cover and sedimentary organic matter content, confirming the significant role of vegetation in stabilizing sediments and promoting organic matter deposition. Reservoirs with higher forest and shrub cover (e.g., Sot and Duboki potok) exhibited greater organic matter accumulation (5.79–5.98%), while the agriculture-dominated Ljukovo catchment (76.85% agricultural land) recorded the lowest organic matter content (3.89%) due to increased sediment displacement and reduced erosion resistance. These findings underscore the critical role of vegetation in regulating sediment dynamics and enhancing organic matter retention in small water reservoirs. To mitigate excessive organic matter deposition and improve water quality, sustainable watershed management strategies—such as vegetation buffer strips, afforestation, and erosion control measures—are recommended.