It takes a while for the yeast to pack down like that. A couple days after pitching into a starter, it is still very loose. Also don't forget to count the non-yeast percentage as well.
The proper way to do a count for a starter is to measure the total liquid volume very precisely and mix the yeast evenly throughout the solution before taking a sample. Make sure not to add in any air and make sure there are no CO2 bubbles. The sample should be degassed. Then take a precise sample, dilute to the correct concentration. You don't want too many cells or two few cells across the counting grid. You want nice, even distribution. Then count and do your math.
Trying to take a sample from some yeast on the bottom will always result in bad numbers, because you will never know what the density is until you do a proper count.
Now if you do that and still don't have the right numbers, I would think either your viability of the initial yeast was really bad or there are errors in counting.
