Here are some potential fixes to try:

Start a new Protected-mode game alone, and then save on turn 1, then load that game again in Protected mode lobby, then invite, you may be able to connect to each other.

Restart computer. Host game with public mode, Send invite to only one friend friend. After he connects, then othes can.

Go to -> Steam - Settings - Downloads - Download region and make sure everyone joining game is in same download region.

When you host the game use a unique lobby name you have never used before.

Also worth a shot, to remove and reinstall your anti-virus software.

The only thing that was no tried is starting a game saving and inviting. Someone availabe so we can try this workaround ...?

The best way to find someone live to play or test Endless Space is check the Amplitude Discord in the lfg (looking for group) channel. Here is the official Amplitude Discord. If you scroll down you will find a channel named #lfg-endless-space, ask there.

Was this issue addressed in the recent patches as I have not tested after the patches if its rectified ? 

Hi all,

The issue is fixed now. For anyone that is having this issue or will have it in the future, the game is not the problem. 

The problem was the ISP. 

What I have done is I have changed my ISP provider as I have changed my living location and decided to use a different ISP. They have as well changed the router. 

So if anyone is having this problem that I had for years, its 99% the ISP. 

Cheers!