Genomic imprinting is observed in endosperm, a placenta-like seed tissue, where transposable elements (TEs) and repeat-derived small RNAs (sRNAs) mediate epigenetic changes in plants. In imprinting, uniparental gene expression arises due to parent-specific epigenetic marks on one allele but not on the other. The importance of sRNAs and their regulation in endosperm development or in imprinting is poorly understood in crops. Here we show that a previously uncharacterized CLASSY (CLSY)-family chromatin remodeler named OsCLSY3 is essential for rice endosperm development and imprinting, acting as an upstream player in the sRNA pathway. Comparative transcriptome and genetic analysis indicated its endosperm-preferred expression and its likely paternal imprinted nature. These important features are modulated by RNA-directed DNA methylation (RdDM) of tandemly arranged TEs in its promoter. Upon perturbation of OsCLSY3 in transgenic lines, we observe defects in endosperm development and a loss of around 70% of all sRNAs. Interestingly, well-conserved endosperm-specific sRNAs (siren) that are vital for reproductive fitness in angiosperms are also dependent on OsCLSY3. We observed that many imprinted genes and seed development-associated genes are under the control of OsCLSY3. These results support an essential role of OsCLSY3 in rice endosperm development and imprinting, and propose similar regulatory strategies involving CLSY3 homologs among other cereals.