Writing is all about communication. Grammar and spelling matter because they make communication easier. When grammar and spelling are correct the reader reads exactly what you intended them to read. There are no ambiguities, no confusions. The reader also reads without effort because they don't have to keep stopping to work out what the meaning might be and effortless reading makes reading a pleasure.
It could be argued that that's what editors are for, and it is - up to a point. But publishing is being squeezed at the moment and there's less money to spend on editing. If there is a choice between two manuscripts, one which will require lots of editing and one which won't, then they will choose the one that will be cheaper for them.
And you can't by-pass the issues of poor grammar and spelling by choosing to publish yourself. because readers can download sample chapters. Would you choose to buy a book with lots of errors? Even if the book was free, you'd be unlikely to want to spend your valuable time on it.
Good grammar and spelling matters, regardless of the quality of the story telling.