Granted I don't have my 5 BW yet, but here's my personal take: If you have a 2nd "fun / sports car", it changes the equation.
For about 10yrs now I've had a sports car (Honda S2000 >> now Lotus Elise) for true twisty road driving, paired with a V8, manual sports sedan (E39 M5 >> E90 M3 >> soon 5 BW... I have a Civic Type R to tie me over in the meantime).
For the sports car, I aim for something that's quick enough to be entertaining but not so much that you'll lose your license flooring it for more than 3 seconds. These can be light, agile, and no-compromise since you're not looking for luxury, refinement, long-distance comfort, practicality, etc. Clearly I went "all the way" with the Lotus as it's very much a pared-down, Point-A to Point-A type of car, and does that exceedingly well.
That leaves the daily you'll be using for probably 90% of your driving, if you're like me. By nature, you'll probably want this to be at least fairly comfortable, spacious, etc., which usually means "size and weight" and less of a razor edge. You probably won't be caning this on backroads as much, because that's what the sports car is for.
So how do you make daily driving a bigger, heavier, more compromised car as fun as possible? In my book it's noise, immediate and plentiful torque, a good manual trans, enjoyable driving dynamics, and certain creature comforts. The 5 BW fits that bill better than the 4 BW IMO and maybe better than any other car (also I have a kid so the back seat and trunk of the 4 seem rather cramped).
I believe that the 4 is more fun than the 5 for true twisty roads driving, but I have the Lotus for that. If I was only able to have one car, the 4 might be a better solution (and also cheaper!).