I was on one side of a ladder and my enemy was on the other side. We were stuck like this beating each other for several turns. I know that you can count those squares that connect elevated terrain as adjacent for close combat, but he was also trying to poison me too. Does this work? I said it was fine for the sake of keeping the game going (and I was kicking his ass,) but I'm still unsure.

Any feedback appreciated, cheers.

Squares of different Elevation are not adjacent, so your opponent would not have been able to Poison you.

The only reason you can make a close combat attack on a "ladder" is because we are specifically told so:

Characters occupying the two squares through which a character can change elevations can make close combat attacks against each other as if they were adjacent.

From the 2014 Rule Book

Since Poison is not a close combat attack, it could not be used against the player on different elevation (elevated nor grounded).

However, Poison can be activated even if there is no adjacent character on which to use it.

http://win.wizkids.com/bb/viewtopic.php?f=3&t=8491