This is a personal question - my opinion isn't right for everyone. What else do you own or have you owned in the past? Does your garage have a 60s muscle car with a 4 speed or a Lotus Elise in it? (The manual trans experience in the 5 is a lot more like the former than the latter).
The only reason I'm sharing my thoughts is that you specifically are looking for a manual trans experience with the Blackwing. To cut to the chase, I own a 5 Blackwing, but think the 4 is a better manual trans experience.
As background, I currently own 9 cars with manual transmissions; I've never bought a car without a manual transmission; I've daily driven manual trans cars since high school (30 years), including a 66 Mustang with a 4 speed in stop and go traffic in downtown Los Angeles on the freeway (very heavy clutch pedal). I never, ever would have considered a blackwing (or any other car) with an auto. In short, I am a manual trans snob. Having said all of that, I think the 5 BW is a better car with the auto. There is absolutely nothing wrong with the clutch / shifter / trans in the 5 - they are executed perfectly. But rowing the gear box in the 5 does not bring me joy the way it does in my other cars. And the auto is faster and more "effortless" - I think it is a better match for the engine.
I have an S2000 and 90% of the pleasure of that car is working the clutch and shifter and winding the motor out to 9,000 RPM. Because it only has 240 horsepower, you can go to a canyon road and wind out second gear on every straight before shifting to 3rd and then heel and toe downshift to second for every corner. I also have a 1930 model A Ford, and the pleasure in driving that car is double clutching every downshift because the trans doesn't have any syncros. I also have a couple 90s Civics and the clutch is so light, and the shifter so fluid that it's completely effortless to heel and toe for right hand turns in city traffic - and again the power is low so you can wind out all the gears and heel and to every downshift and not go to prison. In all of those cars the shifting is natural and effortless (after 30 years even the double clutch in the A is subconscious)
The 5, while a magnificent piece of engineering, just doesn't make me want to shift - ever. The clutch pedal force is totally manageable, but it isn't what anyone would describe as effortless. And, there really is no need to shift on a twisty road. Leave it in 3rd (which is good for 115 mph) and it will be faster than anything you'll ever encounter. The fraction of a second faster the car would be if you downshifted to second on every corner adds no pleasure and you'll pay for it with a sore leg the next day. In fact, the car will pull away from a stop in 3rd gear more easily than any of the other cars listed above will do in 1st. There really is no reason to shift - it's completely optional.
As for the 4 vs 5 - the advantages I see of the 5 are the engine sound and the larger back seats. I think the engine sound of the 5 is worth 30 grand, but excepting the back seats, I prefer pretty much everything else about the 4. The biggest downside of the 5 in my opinion is the added power. It's not usable on the street - you have to track that car to take advantage of the power. To use full throttle in the 5 you need warm tires, a warm, perfectly smooth, and empty road. Even then, you can't use full throttle for more than 2 or 3 seconds. So while the sound of the 5 is among the best sounding engines / exhausts of all time (boy, does that sound bring me pleasure), you can't hear it at full throttle for more than a couple of seconds. The 4, in contrast, has less power so you can actually row the gears and use full throttle in a lot more situations on the street. Also, something about the 4 makes heel and toeing subconscious, while I never do it on the 5. (I wonder if the trans tunnel isn't wider on the 5 and encroaches on the pedal box.)
As a final aside, I did spring mountain twice - once in a 4 and once in a 5. With the 4, the instructor asked why I was on the throttle under braking at the end of the straight - I said I didn't know as I never left foot brake. I reviewed the PDR footage, and I heel and toed EVERY SINGLE downshift on the 4, but I had no I idea I had done it even once - it was completely subconscious and effortless. Even at 1100 miles on the 5, I still struggle to heel and toe in it - not sure why. I, personally, prefer the MT experience in the 4 over the 5. If only they would put a NA V8 (for the sound) in the 4 I'd be the first in line with a deposit.
Note that there is a big price difference for the 5 vs the 4. Could you keep your current 5 non BW for daily use and use the $50K it would probably cost you to upgrade to the 5 BW to buy a Miata, GR86, Lotus Elise, used Boxter, etc for fun weekend use and keep a car you are otherwise happy with?
I owned a few Camaro 2SS 1LEs that only cost $45k loaded up with Recaro seats, HUD, Brembos, mag-ride, e-diff, etc. so I struggle with the Cadillac tax.
