To unlock the Pester Ball, you need to achieve a certain score threshold in the Beach course. Focus on getting good, centered shots of the Pokémon you encounter. Aim for close-ups and try to get them in interesting poses.
Specifically, try to get a good shot of the Pikachu on the surfboard at the end of the course. Throw an Apple near it to make it jump. Also, look for the Doduo and Vulpix near the beginning. Once you've accumulated enough points (usually around 25,000-30,000 total across your best photos), Professor Oak will grant you the Pester Ball, opening up new possibilities for interactions.