Using a phone for navigation has a few issues
Vibrations can kill the camera.
I've heard of this happening on iphones, probably happens on others too. Some companies such as Quadlock and UltimateAddOns make cases that damp the vibrations and are safe for your phone. After 1000s of miles using an UltimateAddOns case with my cheapo Samsung, I've had no issues with the camera.
The phone can overheat and shut off.
A combination of high summer temperatures, the heavy processing required to do navigation, shutting the phone away in an airless case that heats up an oven, mean that this can happen. It's happened to me a couple of times on hot afternoons. Never been a problem, as I've been riding with others who have proper sat navs.
No data at the bottom of your valley.
Relying on data being available everywhere might leave you riding around looking for the nearest signal But some nav apps, such as MRA and I think google maps, allow you to download maps so they're always available.
No dedicated hard buttons.
It's not that easy to poke around at a phone screen when you're stopped at a red light, even if you have magic conductive gloves. A few hard buttons you can activate easily with gloves might help you out.
It's not easy to use the phone.
When the phone's all cooped up in its case, it's one faff to many to get it out just to take some pics or whatever you do with your phone. It tends to stay there until there's a proper lunch or coffee stop.
It's up to you whether these are deal-breakers or not. I've gone the phone route because I got fed up with Garmin and BaseCamp and longed for something that was just a bit simpler to use. Ended up with MRA on the phone, and been happily using it for a couple of years now. Not thinking of going back to sat navs.