The public Multiplayer Part need a multistage reputation system.

So when you create a MP game you can set the minimum Reputation a player need to join or let the game master choose if the player can join.



Play-Reputation:

When a player leaves within 10 rounds, reputation will be degreased.

When you stay more than 10 rounds, you got a point for each round.



Finish-Reputation:

For each finished game, increase the value (when play min. 50 %).

I don't believe this is possible in Steamworks without having a custom matchmaking service...

Yeah, I don't think so either. Not even Valve's own games really use any kind of system to punish bad behavior (outside of banning for cheating).

Kind of comes with the territory when you play public that you'll get a mixed bag of players. Best thing you can do is befriend good players and try to play with them again.