Intestinal CD4(+) T cells that are specific for self-, diet-, or commensal-derived antigens are critical for host tolerance but must also be tightly regulated to prevent aberrant activation and conditions like inflammatory bowel disease (IBD). However, it is unclear how the antigen source and location dictate the intestinal TCR repertoire. Here, we hierarchically classified self-, diet-, or microbiota-dependent TCRs using TClibeta TCRbeta transgenic mice. This demonstrated that microbiota had a greater influence than diet on CD4(+) T cell responses throughout the intestine at homeostasis. Complex bi-directional interactions between microbes and diet were also observed. In the context of murine colitis as a model of IBD, we showed that antigen-free diet substantially altered the microbiota and associated T cell responses, which ameliorated intestinal inflammation. Collectively, these findings suggest how deconvoluting the gut immune interactome may facilitate identifying primary microbial and dietary drivers of T cell responses during health and disease.