I do this and it works great. Ad block on all my devices regardless of proprietary sandboxes. I also use Syncthing over my tailnet IP addresses so that traffic never leaves my “grounds”. I’m slowly building out a whole suite of services I host only within my tailnet, jellyfin, calibre, invidious, it been a great learning experience. I’m about to set up a proper home lab, finally moving everything off an old laptop.
Arkhive
joined 1 year ago
Look up Syncthing and then never stop trying to replace closed source and paid software/services. Like any time you launch something ask yourself “does this hit the same way as when I swapped to Syncthing?” If the answer is no you then put “[name of thing you want to replace] foss alternative” into your search engine of choice. You’ll end up down so many rabbit holes, but you’ll come out the other side a whole lot better at making your technology work for you, not the company that made it, and with a suite of free open sourced tools you are in complete control of.
Here are some tools I use that are super easy to get going.
- Syncthing (cloud storage replacement)
- KeepassXC or Pass if you’re a command line person (locally stored password manager, coupled with Syncthing you have your own private cloud password manager
- Tailscale/wireguard (private VPN that allows you to easily connect all your devices without exposing any of the traffic to The Internet)
- PiHole (a DNS sinkhole that blocks a lot of ads and tracking on your entire network, bonus points if you set it as you Tailscale DNS provider to give all your devices ad block no matter where you are as long as the device was a connected to Tailscale)
Those are the ones that got me going and I personally believe act as a solid core. Most people will find all of those useful. Other services are more user specific, but that’s a lightweight bundle of software that your RPi will handle well. Much more and you might want to look at beefier hardware.
To be totally honest I didn’t look through everything you posted, but I’ve toyed with the idea of intranets myself and have come up with a handful of tools I really like.
- Tailscale can be used when there is a necessity to connect remote locations over “The Internet”. It is a private VPN that provides ip addresses for connected devices that are only usable to other devices within your Tailnet
- Syncthing can be spun up very quickly to distribute and sync files across devices on the local network, within your Tailnet, and yes over “The Internet” if need be. This is not full on web server level of hosting, but it can get some things off the ground quickly.
The way I’ve used this to make an “Intranet” is outlined here. I use Obsidian for a lot of note taking, link storage, and general information gathering and navigating. Obsidian stores all it’s files as plain text in a normal folder structure, but this could also be done with htlm files and a normal browser. I can target any portion of these folders with Syncthing and keep them updated across all my devices through my Tailnet. The broader usage of this begins to get into the idea of an intranet.
Let’s say I meet someone within this community, or maybe from one of the other locations. We get to talking and decide to exchange information from our respective collections. I fire up an ad hoc WiFi network off of say my phone, or a small portable router, add them to my Tailnet, which could even be optional given Syncthing’s built in encryption, add their devices Syncthing ID to the folder I want to share with them. They download a local copy of whatever data I want, and then can return to a hub of their own, maybe a home network, or a larger community wide network. Target the new data they have acquired, and sync it to the hub. We could then remove each other from our Tailnets and Syncthing instances, or leave them so we can automatically update differences when in proximity.
That’s a rough idea of how I think this system could be used for a more “personal” internet. One that focuses more on direct and intentional communication and data storage, where each user or group of users is basically selecting which data they value enough to commit drive space to. I have also researched medium and long range “WiFi” networks using radio or other signals to trickle sync nodes over longer distances. I’ve even been inspired by Factorio’s logistic drone networks and thought to attach portable routers, single board computers, and storage drives to drones or even solar gliders that can trickle sync to nodes they pass over.
Just some ideas that don’t quite fit the different systems you mentioned, but I think are a bit easier to spin up for individual users, and could decentralize the load of what you are trying to do even more. Could maybe post this in some of the other FOSS/networking/linux/privacy communities, but I’d maybe clean up the post a bit and make it clearer what you are after.
Yeah okay. That is the system I tried first. Glad it worked for you so cleanly. My network tab was inconsistent of grabbing stuff for some reason, but Libby also doesn’t seem to work through certain VPNs so seems like it’s all somewhat sensitive. Maybe I’ll give it a shot again!
Woah, what system did you find? Sounds way simpler than what I’ve figured out. I’ve read posts about capturing traffic from the network tab of the web inspector but I got really mixed results. That does work, but it was enough of a pain that I explored other options.
The system I eventually found was on Android Libby store files from audiobook unencrypted. They try to hide them by splitting books up into lots of files with random names and distributing them across random folders within its data folder. It even includes some junk folders and files to try and throw you off. None of these files have files types/extensions and Libby tacks on .mp3 when it comes time to play them.
How this can be exploited: I have an android device but this can also be done with Windows Subsystem for Android, or really any other android emulator. I have targeted the parent folder to all that with Syncthing and set it as a one way sync. This way whenever audiobook files are added they are copied to one of my other devices and because it’s one way sync when my loan expires those files disappear from the Android instance, but persist on the device I’ve copied them to. Next step is filtering out all the junk files. This is shockingly easy as I just click into the windows file browser search bar and hit enter. This serves to show a list of any file of any name of any type in all sub folders. Then I just sort the results by size. Libby doesn’t bother to make the junk files the same size as the “mp3s” (remember they don’t say .mp3 yet) so it’s easy to just truncate the list when the files stop being in the kilobytes and start being megabytes.
What’s left is a list of file that want to be .mp3. I use a command line based batch renaming tool to add all the endings, and then begins the painful task of listening to each file to find where in the book it’s supposed to go. The splits do not line up with chapters, so it’s sometimes handy to have an ebook copy to search for phrases. I put them in order then load them all into audacity, merge and then use the detect silence tool set to between 2 and 4 seconds to try and detect the chapter breaks. I’ll manually clean up any misplaced breaks, export as individual files and then finally use one of the many audiobook binding tools out there to bundle it all back together. Though I mostly do that when I’m sharing the book with others. My préférée audiobook listening platform tales individual files very nicely, so I can save a step if it’s only for myself.
I’m both an iOS and Android user for various reasons. There is a free KeePass front end for iOS, but I paid a one time lifetime license for one that was a little more feature rich. That and the only version of Syncthing for iOS requires a like $4 purchase to allow you to sync folders outside of its default location, which was a pretty necessary feature for me.
Yup. Same system here. I really like it.
I do keypassXC and Syncthing. It’s cross platform with only a couple bucks needed for lifetime access to all all necessary features depending on platform. Besides I use Syncthing for a bunch of other stuff as well, so it fits right into my flow. I’m considering moving to a command line tool simply called Pass, and still syncing with Syncthing, but I’ve yet to pull the trigger on that switch yet.
Automated no. Maybe partially, but fairly easy to get up and running with the added bonus of being as close to “legal” as it gets. Read until the end for a more subversive option.
Libby. I cannot recommend a public library card and Libby enough. For avid readers you can probably just stop there. The loan times vary location to location, but if you are someone that reads a good amount each day you can probably get through whatever book before you have to return it.
Here’s the secret sauce tho. You can specify you’d like to read a book from Libby in a different app. This spits out an Adobe Digital Editions DRM locked book. Once opened in ADE the file is now a “normal” .epup but still time locked with DRM. Enter calibre and it’s plug-in support. My current setup is using an older version and a no longer maintained plugin, but that’s because I’m lazy and haven’t updated things. There is now a new maintainer of the project. noDRM’s DeDRM tool seems like the current standard and includes all necessary info I believe. Once you have the plug-in installed it’s simply a matter of dropping the .epub from ADE into your calibre library and then you have it forever.
As far as making this easy for you family, I would say direct them to Libby for general use, and then if they longer to read something resort to more drastic measures.
Kind of a grey area as I simply think of it as “extending” my loan while still being polite to the people waiting for the book behind me. I mean it’s not actually a grey area, like it’s not technically allowed, but I feel it’s very much in the spirit of public libraries and free access.
As far as automation goes I’m fairly confident I could write a script to get the download from Libby as I can post all the necessary requests from a command line if I want. The problem comes up with Adobe Digital Editions. This does not have any sort of command line interface as far as I know, and unlike calibre a file must be specifically imported through the GUI rather than simply dropped into a folder.
The true cheat code is z-lib, which when set up fully can be interfaced with a telegram bot that can be customized and automated all you want. In my opinion once the telegram bot is set up it’s easy enough for anyone. You put a book you’re looking for into the message box, the bot sends you search results back, tap the file you want and it sends it to you right in the chat. I use it all the time when book recommendations come up in conversation. I’ll search for it on my phone, tap the right one and have the .epub waiting for me on my ereader.
I’ve also got a system for ripping audiobooks from Libby, but it’s way more complicated and marginally more capable of being automated.
Yeah, I agree with other comments. You’ve just reinvented the library, but made it illegal. Libraries are not businesses (honestly thank god) and have no monetization model. A for profit library would be awful. Best come up with a different idea. Could stick with the theme and do like a data aggregation/search/sharing software, but even those I prefer to be foss which makes them not really “business model” friendly.
TL;DR you’ve discovered the library, now go donate to your local one
Syncthing and KoReader. I also have a few android eink devices and this system works great for me. When I need a better interface for organizing/editing metadata of files I use calibre which also has some plugins to help free your files from proprietary epub readers.