How do you do that? Even if I use ships with siege and show all my patience to lower enemy system manpower to zero it need to be invaded, and even with guerilla tactic somewhere around 7-8 buildings are destroyed? Is there a way to lessen losses? 

No other options, guerilla tactic is the way to go. Improvment destruction and ownership are there to avoid that military oriented major factions can easily snowball early game by conquering other systems.