Do you have to declare this each action?
For example, Doom wants Invisible Woman's defense value. When he is attacked does he have to declare the defense as his wildcard action or does he get it if you want it?
What I'm asking, really, is can he wildcard another team ability like Skrull and use the replacement defense also.

only once per action.

If during your opponent's attack you declare Skrulls and you miss your roll, you can't then decide to use the AFF TA.