Fair enough. And you might get run over by a bus tomorrow, but that doesn't mean you shouldn't plan your next holiday.
If you don't stick with it, you'll never know. Yes, the stars may never line up in your favour, which is why you should enjoy the journey, but there isn't a timetable we all follow. Some are high flyers with their first novel, and go onto greater things while others flounder to achieve the same success. Some keep on writing with no success until suddenly they leap to prominence - Clare Morrall's Booker Prize shortlisted success Astonishing Splashes of Colour comes to mind, the publishing deal arriving 20 years after she started writing.
I had supper the other day with an author friend. We first met about eight years ago before either of us were published, although at the time she had an agent. In a few months I too had an agent, and also a publishing deal, something which eluded her. She kept on writing. Her agent dropped her, another took her on, then also dropped her. She kept on writing. Her persistence has been rewarded with a three book deal in both the US and the UK (and a new agent), and I hope the novels will be mega-successful; she deserves it.
But what if she'd given up, even a few years ago? We hear a lot about overnight successes, and some people do get published fairly speedily, but for some - most? - people success comes slowly not quickly. Don't be too hard on yourself, but relax, enjoy your writing and above all give yourself time.