Tea residues are rich in dietary fiber, most of which are insoluble dietary fiber (IDF). However, soluble dietary fiber (SDF) is reported to show a better health-promoting effect. In this paper, the Eurotium cristatum (E. cristatum) fermentation method was employed to prepare SDF from tea residues. The results showed that the yield of SDF in fermented SDF (FSDF) was higher than that in unfermented SDF (USDF). Meanwhile, an increased proportion of galactose and a looser microstructure were observed in FSDF. In addition, FSDF has more advantages than USDF in relieving colitis symptoms. FSDF is more effective in reversing weight loss, colon shortening, and tissue damage. Meanwhile, it has a better regulatory effect on the level of inflammatory factors (IL-6, IL-1beta, TNF-alpha, and IL-10) and oxidative stress (CAT, T-SOD, and MDA). FSDF treatment more effectively restored gut microbiota composition toward normal parameters compared to USDF by upregulating Akkermanisa and Lachospiracae_NK4A136_group and downregulating Helicobacter and Alisitipes. In conclusion, fermentation treatment with E. cristatum contributed to the preparation efficiency and bioactive effect of SDF from tea residues. This study will provide a theoretical basis for the development and utilization of tea residues.