First off, there appear to be more then one type of swarm, or at least more then one thing a swarm may be doing. A bait ball is a circle of beings all circling in the same direction and staying close together but not converging to a point. A large, mostly stable swarm may appear to grow limbs or briefly split. A moving swarm may be more like a flock then a standing one. Perhaps there are other forms we may wish to create that are swarmlike?
So, the rules. A standing swarm that does not change shape seems to need only two rules- move randomly and stay inside a sphere. Luckily these are both O(n) time operations for a swarm of size n, so we can get away with an efficient little swarm if we allow individuals to move through each other. To get the shape changing effect I would guess we could create invisible attractive or repulsive forces and move them around, orbiting the center of the swarm and possibly reversing their effect occasionally. Another possibility is temporary leaders, either with new rules or with the same rules as others, that are followed by close by individuals. Another possibility is that individuals always sort of follow each other but the strength of that influence varies.
For a bait ball I would guess we would need separation from a central point, random movement, and a strong need for alignment. I'm not sure if this is enough, but I imagine the alignment would keep them in check and the random movement (possibly also a need to move forward) would keep them circling.
Finally the flocking behavior of a moving swarm may simply be the above standing swarm rules with a force pulling the individuals to a certain location. This may be enough to create a nice looking moving swarm.
Thats all I can think of for now. Any thoughts? Other types of swarms? Other rule sets that seem plausible?