I've often wondered why life must be so hard, and I think I am getting a handle on it. I am in the midst of severe and scary financial difficulties right now, so I think about it a lot.
I think that there is even more, and that the essence of it is that for anything to be freighted with glory, there must be real, potentially bad consequences for failure. I notice it most in the live performance of music with my band. In our rehearsals, I can only reach to a certain level of axemanship on my electric guitar. But in live performance, because the stakes are so high, and the consequences so severe, one of two things almost always happens: I play better than I have ever played in rehearsal, or else I fall on my face and mess up really badly. So, I think that the close proximity of real, tangible failure is necessary to keep us "on our game", so to speak. I cannot be bothered to play brilliantly in practice, just exactly because I know that my bandmates will overlook any errors and forgive me immediately if I fail. No, to really soar, I need to be working in front of a stern judge, who will weigh every note and bend in the balance, and perhaps find me wanting.
Of course, I learned this basic principle from G. K. Chesterton long ago:
If our life is ever really as beautiful as a fairy-tale, we shall have to remember that all the beauty of a fairy-tale lies in this: that the prince has a wonder which just stops short of being fear. If he is afraid of the giant, there is an end of him; but also if he is not astonished at the giant, there is an end of the fairy-tale. The whole point depends upon his being at once humble enough to wonder, and haughty enough to defy. So our attitude to the giant of the world must not merely be increasing delicacy or increasing contempt: it must be one particular proportion of the two--which is exactly right. We must have in us enough reverence for all things outside us to make us tread fearfully on the grass. We must also have enough disdain for all things outside us, to make us, on due occasion, spit at the stars. Yet these two things (if we are to be good or happy) must be combined, not in any combination, but in one particular combination. The perfect happiness of men on the earth (if it ever comes) will not be a flat and solid thing, like the satisfaction of animals. It will be an exact and perilous balance; like that of a desperate romance. Man must have just enough faith in himself to have adventures, and just enough doubt of himself to enjoy them.
Or, again:
And the perils, rewards, punishments, and fulfilments of an adventure must be real, or the adventure is only a shifting and heartless nightmare. If I bet I must be made to pay, or there is no poetry in betting. If I challenge I must be made to fight, or there is no poetry in challenging. If I vow to be faithful I must be cursed when I am unfaithful, or there is no fun in vowing. You could not even make a fairy tale from the experiences of a man who, when he was swallowed by a whale, might find himself at the top of the Eiffel Tower, or when he was turned into a frog might begin to behave like a flamingo. For the purpose even of the wildest romance results must be real; results must be irrevocable. Christian marriage is the great example of a real and irrevocable result; and that is why it is the chief subject and centre of all our romantic writing. And this is my last instance of the things that I should ask, and ask imperatively, of any social paradise; I should ask to be kept to my bargain, to have my oaths and engagements taken seriously; I should ask Utopia to avenge my honour on myself.
Those quotes are from Chapter 7 of Orthodoxy: The Eternal Revolution. And when I read them, all the seemingly endless difficulty in my life comes clear. There is a reason for it. I have made certain irrevocable decisions (e.g., to buy things I could not afford, and to give free reign to my coveteousness in other ways), and my Utopia is "avenging my honour on myself." It would not be Utopia else. It cannot be any other way.