So much of performing improv is encompassed in that simple statement. Saying Yes is acceptance of your partner's offer - what another person says on stage now becomes "true" for the scene, and everyone has to play it that way. It might not have been the choice you would've made, but it's been made, so move forward with it. Saying and is what moves everybody forward. You're making an offer yourself - you're adding to what's been established and giving everyone more to play with. You're not leaving it all up to your scene partners to do the work, but that you're willing to take a risk and contribute to the scene.
It's a tough thing to do, especially for beginning improvisers. There's a lot of fear in performing something that could go in any direction. How are you going to know what to say? Why can't everyone in the scene just do what I want, because then I'll know where I'm going and what I'm saying?
Well, that's what this site is about. Stay tuned... you'll learn how and why.