It’s kind of like how JRR Tolkien has written many books, though the majority of them take place in Middle Earth. The word “Cosmere” describes the overall setting of most of Brandon’s books, but not all of his books take place in this setting. One such book is The Rithmatist.
And along with what Luamor explained, the coppermind wiki (a Wikipedia for Brandon’s books) says this about the Cosmere:
“The cosmere is a fictional shared universe where several of Brandon Sanderson's books take place. As a result, books set in the cosmere share a single cosmology and underlying rules of magic, and some characters from one world will make appearances on other worlds. Despite the connections, Brandon has remained clear that one does not need any knowledge of the broader cosmere to read, understand, or enjoy books that take place in the cosmere.”
Hope that helps!