I think the time commitment, patience, and fire control skills are the things that make brisket seem difficult. I've found ribs and pork butts are more forgiving if the smoker runs a bit hot, but brisket can dry out very quickly if you don't keep a close eye on your temps. The long temp plateau that you get with brisket also throws a lot of people off. I know a few people who pulled their first briskets at 160 or so since the temp hadn't budged in hours. Needless to say those weren't good briskets.
I have had really good luck with brisket, but as others have said it may take 8 hours one time and 14 the next, and it can be hard to block out a chunk of time that big. It also makes it difficult to cook for a large event - "we'll be eating sometime between 5 and Midnight" doesn't usually go over well.