It's not just about having enough food; it's about having diverse food. Citizens who eat a varied diet (e.g., berries, meat, fish, vegetables, fruit) will have higher health and be less susceptible to illness. This reduces the strain on your Herbalists and Hospitals and keeps your workforce productive.
Actively work to establish all types of food production: Gatherer's Huts, Hunter's Cabins, Fishing Docks, Farms, and Orchards. Use the Trading Post to acquire new seeds or livestock to diversify your food sources if your map lacks certain options. A healthy population is a happy and productive population, directly impacting your town's long-term stability.