Absolutely can be done, and can be done well.
I hate when fantasy is confined to abide by certain rules. The entire genre exists to push those boundaries away, not adhere to them.
If you want a fantasy epic completely without magic you should check out A Land of Our Own. There is some myth in the backstory but it does not even appear in the story itself. It reads like a classic adventure narrative set hundreds of years ago in a far-off world.