I've kept various livebearers in my 10 gallon for a few years now, livebearers are awesome and I love seeing new waves of babies appear.
I've gone up to 6 adult platys in the ten gallon, although that's pretty full, so for a 40b I would say ~24 adults is the absolute max. I think the ideal would probably be around 10-16. For guppies I think you could double that figure. Also I second shrimpaholich, I wouldn't do any crayfish or lobsters, though there are lots of really cool species of shrimp you can try. I love bamboo shrimp with their neat little bristley fan hands.
Also I've always kept my tank heavily planted with live plants, which helps a LOT with water quality and I feel like it makes a huge difference in the survival rate of the fry. Lots of plants create lots of hiding places for the fry to prevent them from getting eaten, and I believe there is also lots of microscopic life living on the plants that provide fry with food to eat in the very early stages of life. Something quick growing to fill up the tank is best, I've had good luck with hygrophila, java moss on driftwood is also great, and also definitely consider floating plants like anacharis or riccia moss, it'll give the fry lots of places to hide out and snatch food from the surface when you're feeding. For feeding I like to crush some of the flake food between my fingers to create a fine powder.
The majority of my experience has been with variatus platys, which grow pretty slowly, they can take months to reach adult size, but that's a plus for me because it keeps the tank from becoming overcrowded too quickly, and I get to enjoy the tiny babies for a long while. Heavy feeding can increase growth but that's just like, more unnecessary maintenance imo lol. I've just got a new batch of dalmation molly fry, and I'm not sure how quickly they're gonna grow, but I assume it'll be about the same rate as the platys. Guppies, however, breed and grow way faster, I often hear of people having their guppy population get out of control very quickly haha.
Speaking of mollies and guppies, they can hybridize, so you wanna be careful about that, don't try putting a big aggressive molly male with a bunch of female guppies, things are not gonna end well for those females! Male guppies and female mollies do fine together though. Platys (for sure variatus, but maybe maculatus too) can interbreed with swordtails, and I've HEARD that mollies and platys might be able to crossbreed, but the jury's still out on that one. I've got a male variatus platy in my tank with mollies though so if they start making babies with his sort of markings i'll know for sure!
Also, consider looking into endler guppies, try to look for pure lines for males and females, I know it's pretty hard to find pure endler females in LFS's, but I think they're awesome and I'd definitely consider keeping a big school of those guys. Killifish are another option and they too look really neat but I don't have any experience with them.
