I might get some flack for saying this, but for me it has to be WoK. I read Stormlight Archive last of any of the full books, and its a good thing I did; it took me nearly 500 or so pages to get invested in the story. I honestly would've abandoned it had it not been a Sanderson novel. Admittedly, this is probably because it WAS one the last of the Cosmere books for me and I refused to be spoiled until I was done Ars Arcanum, but I just had to put it down too many times. I was perhaps just a bit too distracted by trying to piece everything together and screaming about shards, especially since I had just finished Bands of Mourning. I even had to sit the book down to read Warbreaker first against my friends advice to calm myself down.
Going back, I reread it all and enjoyed it immensely. But as I read it originally? I admit it's sort of ruined the first half of the book for me, far more than any of the other books people enjoy saying they hate or dislike in the Cosmere, like Well of Ascension, Elantris, and Warbreaker. Especially since I rather enjoy all of them a great deal, particularly Elantris.
WoR luckily didnt have that problem for me, considering I knocked the whole thing out in a day.