It's easy to accidentally shaky a baby too hard if you juggle it, especially when you are tired or frustrated so generally it's a safer bet to hold the baby pressed to your body, so any swinging the baby does is swinging your body is doing too (swiveling and bouncing), protecting your baby from shaking.
Also adults should never take out their own frustration/anger on a baby. That’s not what we are talking about here.
It’s not that easy to accidentally shake a baby too hard. Babies are quite robust, and new parents (that I have seen anyway) are almost all extremely cautious. But make sure the baby is fully supported; you don’t want their head flopping around.
Go watch the nurses/midwives/doctors in a maternity ward sometime and you will see them manhandling newborns in ways that make new parents nearby feel pretty anxious... but the babies are just fine.
https://www.youtube.com/watch?v=sB2Fw3MFlhk (The original Happiest Baby video has a bunch more examples with live crying babies and makes it a bit clearer. I don’t know if those are online anywhere though.)