Writers obsess over a single word, architects obsess over a single line, athletes obsess over a single motion. So what should drummers obsess over once we have a command of the basics?
Oddly, perhaps ironically, what we need to work on most is the most basic function of all: time, or more correctly, timing. I often see players -- young and old, accomplished and otherwise -- who have picked up the notes in a general sense, but while all the bits appear to be there, for some reason it doesn't seem to work. Just as the composer looks for notes that fit, the drummer should be looking for the strokes that fit. Sometimes drummers will play a lick that may not be suited to the music. Perhaps it's a matter of personal style, but as often it's a disconnect between what can be done and what should be done. But if the lick is appropriate to the style but still doesn't work, then it's usually a question of timing or 'micro-timing'.
The goal is to play with others … as an ensemble. And what holds the ensemble together is that irresistible core of time. If everyone plays the time -- all the time -- then it will work. Now comes the challenge. When you want to add an embellishment to the music, you still are responsible for the time. Everything you do that is not 'playing time' is an interruption of the time. When appropriate and well executed, these interruptions in the flow can be awesome. And when not, they can really toss a wrench into things.
A drummer who tends to speed up or slow down during a fill will throw off the band and the music. Someone who plays the 'correct' notes for a fill but doesn't fit them seamlessly into the framework of the time also makes a bit of a mess. I hear a lot of drummers playing figures that don't work because they are simply un-true to the actual note values. The patterns seem to be correct sticking-wise, but the internal timing is off. The result is that the tune never quite clicks because the drums keep interfering with the time.
So what's the answer?
First of all, listen. And when you choose to depart from the time, listen even more. Try to make your patterns fit tightly, especially with the bass line. listen to how the great drummers to see how their fills fit in. Study, practice, and maybe even take lessons to learn the various note values and to understand how they all fit together.