In either case, you'll eventually notice problems.
Watertight Seal
Caulking around certain joints and fixtures creates a seal that prevents water that flows from faucets and showers from seeping into cracks and crevices. If allowed in, this water can cause damage that's costly to repair. This is why installers often caulk around a kitchen faucet, for example. The caulking prevents water from entering between the faucet and countertop, where it can cause damage not only to the counter, but also to the cabinet and floor underneath the sink. Without caulking around a bathtub, water would damage the wall and floor.
Lower Energy Costs
There are numbers of spots inside a house where caulking prevents drafts and hot or cold air loss to the outside. Prime spots include the borders around windows and doors, and along wall baseboards. If these cracks and openings are not caulked, valuable indoor cool air escapes during hot months and warm air escapes during cold periods. In response, you increase your home's heating and cooling usage to make up for the loss, and increase your home's energy costs. The drafts that are created also make it uncomfortable to stay near walls and windows inside the house.
Protection From Outside Elements
Along with preventing air leaks, caulking around windows and trim provides protection from outside elements seeping inside, around the window frame or through other gaps. Dust and dirt enter the structure more freely without caulk to provide any blockage. When the moisture from rain or snow enters, not only does it damage the structure--causing rotted, warped wood and peeling wallpaper--it also leads to dangerous mold and mildew. Water and mildew stains appear on the walls, and also in the carpet.
Other Benefits
Caulking provides a seal against small insects that would otherwise have open routes to enter a house or building. Also, without caulking, insects have access to more hiding places once they enter a structure. In addition to creating a more professional-looking appearance around surfaces and materials, caulking also prolongs the lives of those surfaces. Because it protects against outside elements and thwarts inside water damage, caulking also helps protect against erosion.