This build-up is known as scale and can affect the integrity of your pipes, slowing water flow and making it easier for blockages to occur. If you live in an area with hard water, you can install a house water softener to remove the minerals.
Filters and Chemicals
There are water filters that can strain out most mineral particles from the water, and you can install these in order to prevent hard water from staining clothes or creating scale in your bathroom and kitchen. However, these filters are installed at taps where water is directly used, not where the water actually enters your house. This means that the water will still be able to form scale in your pipes and tanks, but not in sinks or bathrooms.
Chemical water softeners are designed specifically for laundry machines, and you should use them only to affect the water you use for washing clothes. These chemicals are designed make magnesium and other minerals in the water inert and unable to bond to your clothes, but they also make the water unsafe to drink. These chemicals cannot be used in any other system and will not stop scale build-up. Use them only for washing machines.
Ion Exchange Systems
The most successful whole house option for softening water is the ion exchange filter. These filters are much larger than other options, cost much more and require steady maintenance, but they effectively soften water for the entire house. These systems use a resin made from a mixture of salt and water and a light electrical field to exchange the ions of minerals like calcium and magnesium with sodium ions, rendering the minerals harmless. Most ion exchange filters will soften a certain amount of water every night when no water is being used. They make use of a brine tank where salt and water are combining to replace the resin. You will need to keep these tanks constantly filled with the right mixture of salt and water.
When choosing an ion exchange system, look for one that has the features you want. Some allow you to soften water whenever you want. Others are automated, like thermostats, and soften water as needed. Some let you program specific times for softening water. You may need to buy separate filters to clear out the sodium in the water and make it fit for drinking.