Most cruisers have problem when they need to take it down in more wind than it when up in. That's mostly a practice thing. With practice you can take down a kite in ANY breeze. But until then keep a eye on the wind. As you go downwind you do not feel the wind building nearly as much.
Racers will have problems wiping-out in big breeze. This can be a simple round-up (ok), a round-down (bad), or a nose-dive (disaster).
It gets more complicated if you are trying to work out the details of snuffing the chute in a puff when the leeward mark is only 200 yards from the dirt bank. Then, it definitely gets exciting!