I used to think like that, and it's a fair assesment, but one problem is that beings like dragons are powerful enough spells bounce off of them, and Gandalf is more magical than dragons, at least the more common ones, we'll leave Ancalagon the Black out of that.
Now it's not unreasonable to think that Gandalf just doesn't get that, but those are my thoughts.