Kindle Copy of Wiki

[Cuttleguy]

Is it possible to convert the coppermind(and possibly arcanum) into a kindle eBook that preserves links, usable offline? Is it that different from taking the folder and converting it to EPUB? Does anyone know if this would be possible? Thank you in advance!

[Starwatcher]

@LewsTherinTelescope had a method for doing this for individual pages (using the Rhythm of War summary page as an example, though the page name really only matters for stripping out the edit section links):

Quote

Okay yeah just tested and it's absurdly simple via Calibre, with the caveat that images and links to other pages won't correctly work (I imagine it's fixable but I'd need to spend more time looking into that than I want to right now)

1. Download the webpage.
2. Add it to Calibre.
3. Hit Convert and make sure EPUB is the output format.
4. Under Structure detection, look through both expressions for anywhere it says name()='h1' or name()='h2'. Replace with name()='h1' or name()='h2' or name()='h3'.
5. Under Search & replace, enter <span class="mw-editsection"><span class="mw-editsection-bracket">\[<\/span><a href="\/edit\/Summary:Rhythm_of_War\?section=.+?" title="Edit section: .+?">edit<\/a><span class="mw-editsection-bracket">\]<\/span><\/span> as the expression and leave the replacement text blank.
6. Press OK and wait.

Calibre is genuine magic for any type of book conversions

Doing that for every page would be a bit of a project though, and since pages are continually being updated, it would only be good for so long.

[Cuttleguy]

Thank you! I will see if calibre has a python library of any sort, and see if I can automate this task. Any recommendations on downloading? And do you know the size of the coppermind, and the size of arcanum(I remember arcanum being very large when I previously tried to do this)

[ReverentRevenant]

Some quick thoughts.

First, regarding continual updates… One option is to use the wiki’s Time Machine to archive the pages prior to the book you’re reading. You’ll get a solid database of information and (mostly) avoid spoilers. You’d still face a small problem with outdated wiki pages, however.

As for the scale of the task… You’re looking at somewhere between 5,227 and 21,070 pages to download. (I’m using this as source: https://coppermind.net/wiki/Special:Statistics) The lower bound is if you download solely content pages, while the upper bound includes stuff like redirects, disambiguation pages, talk pages, user pages, etc. This is sizable, but still quite manageable. Way better than trying to turn Wikipedia into an EPUB. However, I’m not sure how well your Kindle will do with searching such a large “book” for text.

I’d recommend doing some tests on a smaller scale first. Download 100 pages (preferably larger, wordier ones), see about merging them all into a single book, and see what works and what doesn’t. Can you effectively search for the text, or do other pages drown it out? (E.g. If you’re searching for Kaladin’s page, are you going to have to wade through a hundred mentions of him on Syl’s page first?) Will you need some kind of table of contents? If so, can that table of contents page scale upwards once you have thousands of pages to deal with? Do tables work or do they break on your Kindle? What happens if you tap on a redirect? There are lots of little questions to address!

This tool, https://github.com/hauke96/wiki2book?tab=readme-ov-file, handles the conversion process for multiple Wikipedia pages into a single EPUB file. I haven’t given it a try, but you might want to take a look at it and see how they’ve managed it. It might give you some good hints and ease the process along.

Best of luck!