Followers
Mirza Ghulam Ahmad had a group of semi-literate friends and admirers early on (early 1880s) who eventually increased their distance as his claims started to become less credible and he gained notoriety.
After he started to formally accept mureeds (devotees) in 1889, he started to attract mostly adventurous and superstitious people from the general body of Muslims. These grew to a few hundred by the mid-1890s. More on numbers below.
Confidants who Openly Denounced
Original 313 (327)
This comes from a list that he published in one of his books. An estimate by Munir D. Ahmed shows that about two-thirds of these had left over the next decade or so.
Notable People/Families who Left Publicly
The majority of people who left Mirza Ghulam Ahmad during his lifetime and his community after his death would just melt away into the mainstream Muslim society.
However, some chose to expose the cult-like environment and the duplicity inherent in the proselytizing activity of the Ahmadiyya. See the list.
Exact Numbers and Exaggeration
From around the Indian Census in 1901, the number has been ambiguous and has fluctuated wildly. H.A. Walter's The Ahmadiya Movement also describes this exaggeration.
In the 1990s, the Ahmadiyya leadership went on 'doubling' there numbers every year until it became untenable, by which time they had hit The 200 Million Figure. The official number has come down to 'tens of millions' since then, but the best estimates put them at somewhere between 500,000 to 2 million worldwide.
For more information on numbers, also see Ahmadiyya as a Cult