Arcanum Search Guidelines

[Argent he/him]

Welcome, users of Arcanum! As you peruse this archive of knowledge about all things Brandon, you will likely notice that the underlying search engine can sometimes produce unexpected results. We tried to hire some researchers from the Palanaeum to do all the work for you, but they charge so storming much! So until we figure out how to find a few thousand broams you’ll need to do things on your own. This short guide aims to make this process a little easier for you.

Basic Search

The most direct way to search for content is the omnipresent search bar. Enter the search terms and the engine will come back with results sorted by relevance - the most relevant on top. The details of how each result is ranked is complex (and may, in fact, change over time), but there are some things you may want to be aware of:

• Tags and exact matches are highly ranked. If you search for a term that matches an existing tag (e.g. Hoid or RAFO), entries tagged with the appropriate tag will show up near the top. Similarly, if your search term doesn’t have a matching tag (e.g. Oracle), entries that contain this specific term will tend to float higher.
• The software powering Arcanum - Palanaeum - supports the ability to find entries that contain words similar to your search term, though generally they will be ranked lower than exact hits. For example, searching for Cultivate will bring up results that include the word (or are tagged with) Cultivation.

Searching for Multiple Keywords

As you might expect, Arcanum supports the ability to search for multiple keywords at the same time. If you use this feature, it is important to remember that, by default, Arcanum will aggregate the results of your “X Y” query with the results of an “X <OR> Y” query - i.e. you should still expect entries containing both of your search terms to float near the top, but following them you will find some entries that contain only one of your keywords. For example, searching for Honor Cultivation will, predictably, put this entry containing both words near the top, but as you scroll down (and to later pages) you will see this entry, which contains only the word “honor”, as well as this one, which contains only “Cultivation”.

If you want to circumvent this default behavior and search for “X <AND> Y” - i.e. if you want only results that contain all of your search terms - you can replace the spaces ( ) in your query with either plus signs (+) or ampersands (&). For example, to search only for entries that contain both Honor and Cultivation, search for Honor+Cultivation or Honor&Cultivation. 

Your browser’s search function (for example, Ctrl+F / Cmd+F) can be very helpful in quickly sifting through variations of terms. Let’s say you’re searching for WoBs on Steelpushing, which does not have a tag and could be referred to by many different phrases (Steelpushing, Ironpulling, Steelsight, Coinshots, Lurchers). You can do a search for “Allomancy,” and then use your browser’s search feature to jump to each of those phrases in turn.

Searching for Exact Phrases

Sometimes searching for specific keywords is not what you are looking for. If there is a specific phrase you want to search, e.g. because you are hunting for a specific entry you remember only partially, you can surround the search phrase with quotation marks. For example, if you want to find out something about the Second Ideal of the Knights Radiant, you might want to search for "Second Ideal".

Advanced Search

For the hardcore WoB hunters among you, Arcanum offers some advanced searching/filtering capabilities. 

• Search in date range: show only entries whose date is within the specified range.
  Note that this is the date an entry was recorded, not the date it was entered in the archive. In other words, entries from the Arcanum Unbounded Chicago signing will all have an entry date of December 6, 2016 (the event’s date), even if the event itself was entered in Arcanum a week later. This feature is generally useful if you are looking for entries from a specific tour or ones prior to / after a certain book's release.
• Search for speaker: show only entries that include a specific speaker.
  Most Arcanum entries come from unknown sources, so the speaker will often be marked as simply Questioner, but lines coming from known members of the community, as well as ones from members of Team Sanderson / Dragonsteel Entertainment will usually be credited appropriately. So, if you are curious about what Brandon’s art director and assistant Isaac Stewart has said on record, you could search for him specifically.
• Search for tags: show only entries tagged with the specified tags.
  Interested in all questions about Hoid that have been RAFO’d? You can search for those. Please be aware that choosing and clarifying tags is a part of the review process every event goes through; it is possible that some entries have not been properly tagged yet.
• Exclude those tags: show only entries not tagged with the specified tags.
  Interested in all questions about Hoid that have not been RAFO’d? You can search for that too.

Advanced...-est? Search

There is one more trick, deep in our sleeves, but it is only for those who dare brave the dark corners of Palanaeum's backend - PostgresSQL. It is a trick so dark, we can only speak of it in whispers. It is the ability to apply some of PostgresSQL's text search operators directly in Arcanum. We speak of this:

"But Argent," I hear you cry out in dismay. "we know not how to perform these dark rituals! Do we need to sacrifice cats and rats for this power?"

No, friend, no sacrifices are needed. Let me teach you some tricks:

• AND queries together: we touched on these earlier, so this should be of no surprise - you can use the && operator (written as just & in Arcanum's search bar) to search for "X <AND> Y". As mentioned earlier, you can search for Honor&Cultivation and expect to see only entries that contain both search terms.
• OR queries together: these are less useful, as Arcanum treats all spaces as || operators. Still, if you wanted to, you could use the Postgres OR operator (written as just | - that's a vertical bar - in Arcanum's search bar). So you should expect the results for Honor Cultivation and Honor|Cultivation to return identical results.
• Negate a query: here's something new. Those of you with strong GoogleFu will know that you can force Google to exclude certain keywords from its results by prepending a minus sign to them. Well, you can achieve the same thing in Arcanum by prepending an exclamation mark (!). Remember how Honor&Cultivation gave you only results that included both Honor and Cultivation? Well, you can search for Honor&!Cultivation if you want results with Honor and not Cultivation (note the ! in the query) - i.e. results that have only Honor. Remember, spaces are equivalent to ORs, so searching for Honor !Cultivation will actually give you all entries that either have Honor or don't have Cultivation; and since the overwhelming majority of entries don't have Cultivation, you end up with pretty much every entry in the archive...
• Query following a query: I'll confess, this doesn't strike me as too useful, but you can constrain your query so it makes sure some search terms appear before others. You'd do this with the <-> operator. For example, you could search for Bridge<->Four (and get pretty much what you would expect, results where "Bridge" appears before "Four"), or you could search for Four<->Bridge (and find out that the word "Four" has never appeared before the word "Bridge" in Arcanum).
• Query containing / is contained in a query: This is one of those features that feel like they could be useful, but I can't come up with a good use case to show them off. In short, it allows you search for one search term in the results of another, effectively giving you the intersection of two queries. This is accomplished with the @> and <@ operators, where the angle bracket indicates the direction of containment. For example, you could search for Sadeas@>flute if you were interested in the results of a Sadeas query which contain "flute" (or flute@>Sadeas for flute query results that contain Sadeas, which is equivalent). The thing is, both of those seem equivalent to the results of searching for flute&Sadeas, so maybe this only comes in handy with even more complex queries. Best not to worry about this one, I think, unless you know PostgresSQL. Especially since I can't figure out how to make the other direction of this operator, the "is contained in" version (<@) to work...

Event Search

At the moment, the functionality to search events has not been implemented. If you are looking for a particular event, scroll through the Event list. To make this a little easier, you can change the number of events you see per page to (up to) 100, and then use your browser's search/find feature (Ctrl+F / Cmd + F).

[Oversleep]

Good work. This should be made more visible.

[+Extesian he/him]

This is excellent Argent. Nicely done. Taught me a few useful things and the examples and links work well.

[Mestiv he/him]

This guide is so good, I think we won't need to change anything about searching for entries for a long time

[Kanrei]

It might be obvious, but can you mix operators in search?
For example to find all entries which contain Lift and Old followed by Magic
Something like
Lift+Old<->Magic
which isnt working

[Argent he/him]

3 hours ago, Kanrei said:

It might be obvious, but can you mix operators in search?
For example to find all entries which contain Lift and Old followed by Magic
Something like
Lift+Old<->Magic
which isnt working

In theory, it should be possible. I am not sure why it doesn't work, even parenthesized. 

[Farnsworth]

Wait - Paelenium is Rosharian. Did you guys make us our own search engine?

[Argent he/him]

6 minutes ago, Farnsworth said:

Wait - Paelenium is Rosharian. Did you guys make us our own search engine?

It's more of an organizational system than a search engine, but in theory, if another fandom wanted to set up a similar system where they categorize and tag pieces of data, they would be able to use Palanaeum to help them do so. I'll let @Mestiv explain the details of it.

[Chaos he/him]

1 hour ago, Farnsworth said:

Wait - Paelenium is Rosharian. Did you guys make us our own search engine?

Wikipedia is its own thing, but it runs on a fork of the software MediaWiki.

Arcanum is its own thing but we call the software Arcanum runs Palanaeum, and so if other fandoms and communities want to use the open source Palanaeum, they can.

[Mestiv he/him]

2 hours ago, Farnsworth said:

Wait - Paelenium is Rosharian. Did you guys make us our own search engine?

Like Argent said, it's not exactly a search engine. More like software to organize interview data. Yes, we made it, but the search engine part is mostly a ready solution provided by the underlying database software.

And as Chaos said, Arcanum is the name of our website, but the whole system will be available as open source project called Palanaeum next year and anyone will be able to use it, provided they have required knowledge and determination to input all the data.

[Farnsworth]

That’s so cool! I’m just beginning to realize the amount of work you must have put into this!

[Scion of the Mists]

I am having issues when searching the Arcanum for simple words like down, up, or the.  If I type just one of those words into the search bar at the top of the page, I get no results.  If I search the same word using the advanced search, it returns all 8933 entries, whether they contain the word or not.  

Additionally, this impacts AND searches as well.  If I search for hoid+own (I was trying to find the one about him owning the mansion), I get all 524 hoid entries.  

[Mestiv he/him]

19 hours ago, Scion of the Mists said:

I am having issues when searching the Arcanum for simple words like down, up, or the.  If I type just one of those words into the search bar at the top of the page, I get no results.  If I search the same word using the advanced search, it returns all 8933 entries, whether they contain the word or not.  

Additionally, this impacts AND searches as well.  If I search for hoid+own (I was trying to find the one about him owning the mansion), I get all 524 hoid entries.  

Unfortunately, because we use PostgreSQL full text search feature, we have to deal with it's consequences. One such consequence is that there are words that get excluded from searching, because they're too common. A more or less accurate list of those words is here: https://gist.github.com/fnando/8999831. "Own" is on that list, that's why searching for hoid+own is equal to searching for "hoid". But if you try hoid+mansion, you'll get right what you need.

[Scion of the Mists]

2 hours ago, Mestiv said:

Unfortunately, because we use PostgreSQL full text search feature, we have to deal with it's consequences. One such consequence is that there are words that get excluded from searching, because they're too common. A more or less accurate list of those words is here: https://gist.github.com/fnando/8999831. "Own" is on that list, that's why searching for hoid+own is equal to searching for "hoid". But if you try hoid+mansion, you'll get right what you need.

Thanks for the info - I knew it had to be something like that.