I noticed the little fan worms will populate and die back in cycles. I also noticed that when I quit running my skimmer for a time I had a regular explosion of them. Worms seem to breed when there's lots of feed (ok, I even groaned at myself over that one!)
At one time I had so many in my tank that the kids called it "tube worm city" LOL