Over on Warseer Natura posted this question:
Something's been bugging me for a while now, because it just doesn't seem to make any sense. In every Codex or Fantasy Army, there are units that the vast majority of people won't take. This is almost always due to the unit being perceived as overpriced or underpowered. In rare cases the model itself may detract players from using it, but the reverse is also true: the model itself may be so awesome that regardless of it's usefulness, you buy one. You're unlikely to buy a whole swag of them however, and any sales of these models will be vastly outweighed by people buying units perceived as powerful or useful.
Now here's the part that confuses me. Games Workshop write the rules and produce the models. It's in their interest to sell as many models as possible, obviously. So why, when writing the rules, would you create intentionally bad units that people will almost never take? Surely it makes far more sense to make units as equally attractive as possible. I say intentionally here because I simply cannot believe that the rulebook writers honestly thought all their entries were good.
I'm not suggesting that GW make every unit an uber-death-dealing monster. I just can't understand why they'd shoot themselves in the foot like this.
This was my view on the matter.
In order think about why GW might create poor units you first need to ask why GW produces army lists and the units that populate, then you can put them into some sort of context.
GW releases army lists to allow people to play games of different types including tournament play, pick up games at a GW shop or club, casual basement games at home against regular opponents and campaign games.
Then each army book/codex includes units for their in game effectiveness, to highlight a theme or background to the army, for legacy reasons (because a unit was included in the last three lists) or for aesthetics because a cool model exists.
As has been pointed out, tournament and pick-up game players are the most likely to complain about poor units because they have to pick units without knowing which army they are facing in a competitive environment. They have to look at the army list and make a judgement about which units will be most useful. All other considerations are secondary.
But that is not the only way of playing the game.
Players in a casual setting are unlikely to spam a forum with complaints that a unit is poor because they know who they are facing every week. This shifts the balance of army books quite significantly. For instance I have recently been playing a lot of casual games against a Chaos Daemon player using my Daemonhunters. Because I know what I will come up against every week I know I can load up on some units that are great against Daemons but would be classed as poor against everything else.
Another good example is my WFB games, which I have not been playing for very long. My only opponent is a Dark Elf player and I play Warriors of Chaos. He picks a Hydra every game and thinks it is a poor choice because my Knights and characters beat it up in many of my games, but I hear that it is a bargain in competitive play. His view is entirely coloured by the types of games he plays.
And what about background driven/scenario driven/campaign play? At least if GW includes an entry for a Techmarine it gives players an option to use one for such games.
Plus you have armies, sub lists and units such as the Eldar Craftworlds, Ork clans, IG regiments which GW cannot drop. If GW were redesigning 40k as a game with a totally clean slate some of these ideas would be dropped or changed considerably and the game would be easier to balance but GW has 20+ years of history to adhere to. They have tried to incorporate these armies into the main lists recently but it is obviously very tricky to balance everything together.
How can GW write a codex which allows an Eldar player to choose freely between all units and balance that against a player just picking from Iyanden units? A separate book for each is one option but at one point there were something like 60 different army lists available for 40k, not including Forge World. Balance is then even harder to obtain especially as editions of the core rules change.
So, just to sum up, I think that it depends entirely upon how you play the game.