It's easier to do construction work when the weather is dry and relatively cool. The cool air will make it harder for you to overheat, and of course things will be much more pleasant if it's not raining or snowing on you. Depending on where you live, different times in the year are more likely to consistently produce favorable weather. In general, spring and autumn are the best times of year to build.
You can build a house for a lot less money by waiting until you can get a good bargain on land, building materials and labor. These prices routinely fluctuate depending on market factors and resource availability.
For example, the housing crisis of the late 2000s caused a sharp contraction in homebuilding activities, leading to a drop in prices across the board. Land that had been bought up for development suddenly become economically impractical, leading developers to sell it on the cheap. Demand for homebuilding commodities such as wood, copper and steel all dropped significantly, making the materials cheaper to buy. And less development meant fewer jobs for construction workers and contractors, forcing them to lower their fees to remain competitive. If you don't plan on selling the house for several years after you build it, a housing contraction is usually a good time to build. If you do plan to sell soon after building, then the best time to build is usually at the beginning of housing boom, when prices are still low but the market is picking up steam.
All else being equal, it's better to build when you can afford to do so with few or no loans, and without totally depleting your savings. Building a house will tie up a lot of your money, and once you commit to it, it could be disastrous for you to have to abandon construction at some point due to unforeseen financial problems. You can largely avoid these risks by having money to build and money to spare right up front at the start of the project.