You need it.
But I have two things to comment on with regard to setting.
1. Don't get so caught up in the story that you forget setting. That's easy to do. I know. I've done it.
But the reader will appreciate a little reminder once in a while.
Key words here: once in a while.
You don't need to shove setting down the reader's throat - but a little sprinkled here and there serves to keep the image of the setting in the reader's mind while other stuff is going on.
Remember the weather.
Remember the smells.
Remember what time of year it is (so...what are people doing, seeing, hearing, etc.)
2. If you don't have a VERY clear image of the setting in your mind as you write, the reader won't either. AND - that image must stay consistent throughout the story (unless the weather changes, or whatever.)
Here's what I do to help with that - I DRAW the setting.
Now, bear in mind that I am the WORST artist ever. I'm talkin' stick figures here, folks. So my drawings are embarrassingly pitiful. But they are for my eyes only (although I did send one to a copyeditor once and could practically hear the snickers....).
In Me and Rupert Goody, I drew a sketch of the inside of Uncle Beau's store. I needed to see where the old couch was, where the cash register was, where the door to his room was, etc.
In Taking Care of Moses, I drew a sketch of Randall's neighborhood - where his house was, where the church was, etc.
In How to Steal a Dog, I sketched the town - where the school was, where Carmella's house was, where the abandoned house was, etc.
In my upcoming novel, Greetings from Nowhere, I sketched the motel, numbering the rooms and marking which characters were in which rooms, where the swimming pool was, where the flagpole was, etc.
Those little sketches were a valuable tool to keeping the setting consistent and for helping me to remember to drop in some references to setting from time to time.
They helped me see if the movements of the characters were logical and if all the action "works."
They are also much appreciated by copyeditors (even if they snicker).