Count og Join problem

Jeg har følgende query:

SELECT LeadersAndMembers.ID, LeadersAndMembers.Name, Sum(LeadersAndMembers.NumOfMembers) As NumOfMembers
FROM
(
    SELECT Groups.ID, Groups.Name, Count(*) As NumOfMembers
    FROM Groups
    LEFT JOIN Members ON Members.GroupID = Groups.ID
    GROUP BY Groups.ID, Groups.Name
    UNION ALL
    SELECT Groups.ID, Groups.Name, Count(*) As NumOfMembers
    FROM Groups
    LEFT JOIN Leaders ON Leaders.GroupID = Groups.ID
    GROUP BY Groups.ID, Groups.Name
) AS LeadersAndMembers
GROUP BY LeadersAndMembers.ID, LeadersAndMembers.Name
ORDER BY NumOfMembers DESC

Jeg skal have alle grupper, uanset om der er Leaders eller Members tilknyttet og så hvor mange members/leaders der er. Dette virker fint, ud over at gruppen "test" som er oprettet, vises med NumOfMembers = 2, men der er hverken en leader eller member tilknyttet. Hvordan fixer jeg den?