A good workbench should be the starting point. Buying them is expensive and they're usually not well built when pre-made or ready to assemble. I prefer building one ground up. Vices seem to always be handy when I need to beat on something with a BFH (all of my projects seem to require this at some point). I also like having a grinding wheel mounted to the bench. Pegboard behind the bench for the most commonly used tools and those that are hard to store elsewhere like squares and levels. $200-$300
Get enough lighting. A couple 4' workshop lights can be had for less than $100. Add an LED worklight on a stand for another $100 for portable light.
Then keep an eye on Craigslist and yard sales for cabinets and tools sold by angry wives. Golf clubs are found this way, too, lol. It's hard to find good cabinets at low prices. Don't bother with "value" cabinets like the Husky line at Home Depot. A decent one is going to eat up your whole budget. Either get a good one, or find a way to go without for a while. I'm currently without.
The cost of shelving is surprising. Cheap shelves have burned me a couple times. Depending on how much you need, this could easily take up your entire budget, even for just one decent shelf.
I keep nails and screws on shelves made with cheap L-brackets and a board across them. If nails/screws/bolts aren't in a correctly labeled, store bought box, I toss them. I've spent too much time searching through buckets and drawers looking for the right screw/bolt just to end up driving to buy a new box anyway so I don't keep stragglers around anymore. One 4' shelf holds all of the ones I keep on hand. This has helped me keep organized more than anything else, really. I have to make a conscious effort not to hoard fasteners.
Most of my power tools are on a shelf I built into the bottom of my workbench. Socket sets are in the case they came in and usually either on top of the bench or wherever I last used them, usually against the wall in the garage. A couple small Craftsman toolboxes house the remaining tools like screwdrivers and such.