I had a situation where I met a new civ by sending 2 troops to their land. Now the game is deadlocked because I cannot finish my turn. It refuses to allow deletion or any movement whatsoever a real game breaker. How did this get missed in beta testing of this game? 

Can you share a saved game ?

Sure, this is due to no open borders, I could not move, thus not able to finish my turn! https://www.dropbox.com/s/zra32d3m4qte9ah/Foxs%20Mauryans%20Turn%20462.ctr?dl=0

You should be able to give a move order outside the other empire territory to get out.

Worth noting, deleting your units also provides population to the territory you're deleting them in.

No-one else has said this yet, but it's really important information and an anti-frustration measure of sorts for reclaiming your pops / using your early tribes in a different way.

DrunkenChoko wrote:

You should be able to give a move order outside the other empire territory to get out.

Yup!  Just give your army directions to move to a tile that's outside of closed borders and they should take the shortest route.  The worst situation they can be in is "hemmed in" in neutral territory and unable to escape through closed borders.  This happens to my naval units on occasion.

Also, you could "Skip" the army's turn (there's a button, or press "K") and you should be able to end your turn, even with some unfortunate scouts trapped in hostile territory.