The enemies are allotted by preselection. The selection is based on the main buildings of both players. 

Note: If a suitable preselection has been made and the player executes many PvP battles in succession, it may frequently lead to multiple attacks by the same player.

Hint: You can take revenge on the attacker through reports (left side menu-> Reports).