One of my co-workers is a sgt in the marine corps reserves, and is the armorer for a recon unit. He's been to Iraq a couple times now and is probably gonna be going back sometime during the summer. Because he's with a recon unit he tends to see just about every weapon that the US military uses and a couple of foreign weapons.
Anyway, to get to my point, the biggest problem that the current generation of M16's and M4s have is bad magazines. The magazines springs tend to be crap and lose tension when they are kept loaded for more than a week at a time, once they weaken they are bad forever, this is probably the biggest reason people think the M16 is a piece of crap, which is unfair because they run almost flawlessly with proper magazines. The reason the magazines are crap is simple, it's due to a "fair" business practice clause that all M16 magazines have to be manufactured by small business's in the US. This means that there are a lot of manufacturers out that that build varied quality magazines, usually the problem is bad springs.
The other big problem the M16/M4 has is that soldiers aren't really instructed in how to clean and oil, either that or they ignore the lesson, a lot of soldiers seem to think that the gas ports are oiling holes, and the gas system on ANY gas operated gun should be bone dry and free of debris.
Another big problem which is also magazine related is broken or bent followers in the magazine. When a soldier does a combat reload, either in practice or in a real fight, he simply drops that magazine on the ground, well the heavy part of the magazine is the top so they land on the follower and bend the crap out of them which ruins the magazine till an armorer can fix it.
The problems that M16's have are mostly user error, the L85 is just as prone to the same problems as the M16. I firmly believe that the M16 has a bad reputation for two reasons.
#1 it was a piece of crap in vietnam, and it still suffers from that stigma even though those problems have been fixed
#2 it's used by huge military force and is only 2nd to the AK-47 for production numbers, so there are going to be the occasional defects like anything mass produced, and it only takes a couple soliders bitching about one gun to get a rumor going that all the guns are pieces of crap.