It has long been a source of confusion to players what doubles mean, especially when it comes to high level contracts. We are governed by a couple key principles. Lets work our way to those principles.
In the above auction, what does double mean?
Starting assumption - we will almost never be dealt a "penalty" hand in such situations. East usually has 8 good diamonds for the bid, our chances of having say 4 or 5 good diamonds is maybe 0.5%, basically never (and we will talk about what to do in such situations later).
Reality: We will frequently be dealt a hand with 0, 1 or 2 diamonds, without a clear bid. For example, lets give you cards in that auction
What do you do here? Just play in the 5-2 spade fit? This was the full hand
6♥ and 6♣ both made, whereas 4♠ goes down!
Doubles on the 3 and 4 level need to convey the meaning - I want to bid, our side has enough points, but I don't have a very clear bid. I.e Takeout.
The funny thing about it all - A lot of the time we leave high level doubles in, despite their meaning.
Take a look at this example.
The opponents have a 10 card fit for sure to be bidding (overbidding) so high, in the face of a 2NT opening. Our double is "takeout" by meaning, showing points, too much to just let the opponents play undoubled (given partner is a 2NT opening). Yet, we were 100% sure our partner was going to pass our takeout double, why?
It is much easier to defend on the 5 level than to declare. We will almost always "leave in" a takeout double.
The 4 level is related to the 5 level, not quite as extreme - but often we want to leave 4 level doubles in also.
Partner doubles 4♦, we are not thrilled about defending 4♦, especially with our values sitting under the 4♦ bidder - but it is similar to the rules of gravity - how can the opponents make 10 tricks when we have enough points for game? Mostly they will not.
The alternative to passing is to take on the burden of declaring, and now having to make 10 or 11 tricks, with your balanced hand and no fit with partner.
It is better to defend on the 4 level than declare without a fit. Also, balanced hands often indicate that its time to pass.
Lets close on another example
Your 2♠ bid was a cue raise, showing 3 card support and 11+ points (see Cue Raise in Bridge: A Strong Raise of Partner's Suit). What now - and what is partner's double?
Think about this logically, the opponents have say a 10 card fit to bid like this, they aren't bidding on points. Partner probably has 0 spades, his double is not "penalty", it just shows a good hand with extra points that wants to do something.
It conveys this meaning: Please do something logical, I would like to keep bidding.
Since you have 5/12 of your points in the opponents suit (see Points opposite Shortage: Part 1 - Fundamentals), it indicates that it is time to defend rather than bid on. Whereas, if for example you had this hand
Here you have 0 points in partner's shortage, and it is an excellent time to bid on. Some people might say, well you are balanced, shouldn't you defend? Partner is highly distributional here, with a void. You don't both need to have a distributional hand to move, for your hand its more about whether your points are well placed or not. Yes it would be great if you had some distribution as well, but you cant have all the stars aligning every time..
What to do in the rare event when you have an actual penalty hand?
Suppose this rather unlikely hand comes up
Here you would be cursing this article because you would want X to mean penalty. What can you do? Firstly pinch yourself, you're probably dreaming, its once every few years this might come up. But, okay say it does.
When you have a pure penalty style hand, just PASS.
Passing is the correct idea. Why? Firstly you risk the chance that it goes all pass, and 4♦ is the final contract. Firstly that is not so bad, you will get a Plus score, which is already better than doubling and partner correctly thinking it is takeout, and being on the 4 or 5 level going down, rather than defending 4♦.
But much more likely, partner will have 0 diamonds and contribute a takeout himself, for example North may have
This should be a classic takeout style hand - north should also be aware to the possibility that you have a stack of diamonds (he has none), and contribute a double on this hand, usually expecting you to pass it. (Worth noting, the double, as always, is still takeout style).
Summary of principles
1. High level doubles are takeout. Takeout style hands will be about 100x more frequent than penalty style hands.
2. We tend to leave in high level doubles, especially on the 5 level, and also on the 4 level - especially when we have values in the opponents suit.
3. When the opponents bid and raise a suit, showing say 9 or 10 card fits, there is no use for a "penalty" double, it rather means "takeout" but asks your partner to decide (they should often prefer to defend if their points are in the opponents' suit)
4. With a pure penalty style hand, Pass and hope/expect partner to double most of the time, for takeout, and you can leave that double in.
Get my #1 tip for improving your scores straight away.