Open your world map (default key 'M') and locate the town the merchant mentioned. Click on the town to set a path. Your party will automatically move towards it. During your journey, you might encounter small groups of bandits. In the early game, it's advisable to avoid large groups or retreat if you are outnumbered. If you do engage, try to use your starting weapon effectively and protect your few recruits.
Keep an eye on your party's speed, which is affected by the number of troops, their mounts, and your party's inventory weight. Faster travel means less chance of being caught by larger bandit parties. If you run into trouble, remember you can always press 'Tab' on the world map to retreat from an engagement before it begins, though this will cost you some morale.