I don't expect builds to last against the enemy. Griefers still take too much from the game -- even with votekicking. They trash team builds faster than the enemy can, and they waste team player slots because they're not in the fight.
Griefing seems to be worse when more non-English language players are on; this may be due to of the votekicking language barrier --not because other non-english players have more griefers.
A GUI votekick system could solve all of this, along with a better notification (audio & flash visual) of an initiated or in-progress votekick. An icon system could also allow for multiple concurrent kicks, as there are often multiple concurrent griefers between both teams. Something like a non-optional thin sidebar could do the job, and a mouse pointer could be given focus in that bar by holding a key down (Tab, Q, or E key, something close to the WASD set). "Start new votekick" button with an according icon, a few buttons for the reason (destroying buildings, teamkilling, inactive, mod/hack, Other, etc), a "cancel votekick" button, a dropdown of players to select from, and then a vote button (kick or stay) for each of any current votekicks.
I know it would eat a sliver of screen real-estate, but if it took even more during a votekick, the system would be more responsive for kicking griefers, greatly improving gameplay.