diff --git a/README.md b/README.md index b23f68e..cbee951 100644 --- a/README.md +++ b/README.md @@ -1,112 +1,133 @@ -# GTS-HolMirDas ๐Ÿš€ +

+ + Join Zeitfresser Matrix Community + +

-RSS-based content discovery for [GoToSocial](https://codeberg.org/superseriousbusiness/gotosocial) instances. +

+GTS-Federator + +

+ +A powerful RSS-based discovery engine to populate GoToSocial instances. + +

-Automatically discovers and federates content from RSS feeds across the Fediverse, helping small GoToSocial instances populate their federated timeline without relying on traditional relays. +

+ ๐Ÿฐ Website + ยท + ๐Ÿ“ฐ Zeitfresser Matrix Community + ยท + ๐Ÿ˜ Fediverse + ยท + ๐Ÿ’ฌ Supportchat +
+
-Inspired by the original [HolMirDas](https://github.com/aliceif/HolMirDas) by [@aliceif](https://mkultra.x27.one/@aliceif), adapted for GoToSocial with enhanced Docker deployment and multi-instance processing. +## โœจ Introduction -## โœจ Key Features +**GTS-Federator** is a high-efficiency content discovery tool designed for [GoToSocial](https://codeberg.org/superseriousbusiness/gotosocial) instances. It solves the "empty timeline" problem of small or personal instances by automatically discovering and federating content from RSS feeds across the Fediverse. -- **๐Ÿ“ก Multi-Instance Discovery** - Fetches content from configurable RSS feeds across Fediverse instances -- **โšก Performance Scaling** - 20-100 posts per feed with URL parameters (`?limit=100`) -- **๐Ÿณ Production Ready** - Docker deployment, environment-based config, health monitoring -- **๐Ÿ“Š Comprehensive Stats** - Runtime metrics, federation growth, performance tracking -- **๐Ÿ”ง Zero Maintenance** - Runs automatically every hour with duplicate detection +By leveraging RSS feeds from larger instances or specific hashtags, GTS-Federator acts as a lightweight alternative to traditional relays, giving you full control over what kind of content populates your federated timeline. -## ๐Ÿš€ Quick Start +This project is a refined and modernized fork of the original [HolMirDas](https://github.com/aliceif/HolMirDas) concepts, specifically optimized for Docker deployment, long-term stability, and GoToSocial's unique architecture. -```bash -# Clone the repository -git clone https://git.klein.ruhr/matthias/gts-holmirdas -cd gts-holmirdas +## ๐Ÿ›  What this bot is for -# Copy configuration templates -cp .env.example .env -cp rss_feeds.example.txt rss_feeds.txt +GTS-Federator is not a relay in the traditional sense. It is a targeted discovery engine. Instead of receiving everything from a relay, you "pull" exactly what interests you. -# Edit configuration -nano .env # Add your GTS credentials -nano rss_feeds.txt # Customize RSS feeds +It is a perfect fit if you: -# Deploy -docker compose up -d +- run a personal or small GoToSocial instance (especially on low-power hardware like a Synology NAS or Raspberry Pi) +- want to populate your federated timeline with specific topics (via Hashtag-RSS) +- prefer a pull-based discovery over the "firehose" of public relays +- need a "set and forget" solution that runs silently in the background + +## ๐Ÿš€ Core Features + +### Smart Content Discovery +The bot monitors a configurable list of RSS feeds. For every new entry found, it triggers your GoToSocial instance to fetch and federate the post. This makes the content available for your local users and populates your instance's database with new remote profiles and statuses. + +### Efficient Resource Management +Unlike many other fetchers, GTS-Federator is designed with a "Single User" philosophy. It uses a persistent HTTP session and a local JSON-based memory to ensure that your instance isn't hammered with redundant requests. + +### Real-Time Analytics +After every run, the bot provides a clean summary of its activity: +- **Runtime:** How long the discovery took. +- **New Posts:** Number of successfully federated statuses. +- **Instance Growth:** Tracking how many new Fediverse domains your instance has discovered through the process. + +### Adaptive Pausing & Timezone Support +The bot calculates its next run based on your local time. With built-in timezone support, the logs will always show you exactly when the next discovery cycle starts, matching your local wall clock. + +### Production-Ready Dockerization +The entire application is container-native. It runs as a non-root user for enhanced security and can be deployed in seconds using Docker Compose. + +## ๐Ÿง  Design Philosophy + +GTS-Federator follows a few key principles: + +- **Keep it lightweight** โ€“ Written in clean Python, utilizing minimal RAM and CPU. +- **Zero-Maintenance** โ€“ Once configured, it handles rate limits, connection timeouts, and state persistence automatically. +- **Privacy First** โ€“ No external tracking. The bot only communicates with your instance and the RSS sources you define. +- **Transparency** โ€“ Clear, human-readable logs that tell you exactly what is happening. + +## โš™๏ธ Configuration + +Configuration is handled primarily via environment variables in your `.env` file. + +| Variable | Description | Default | +| :--- | :--- | :--- | +| `GTS_SERVER_URL` | Your GoToSocial Instance URL | - | +| `GTS_ACCESS_TOKEN` | Your App Access Token | - | +| `FETCH_INTERVAL` | Time between runs (e.g., `30m`, `1h`) | `30m` | +| `MAX_POSTS_PER_RUN` | Max posts to fetch per feed per run | `25` | +| `TZ` | Your Timezone (e.g., `Europe/Berlin`) | `UTC` | + +## ๐Ÿ“ฆ Installation + +### Preparation + +Before deploying, you need to set up the directory structure. This ensures the bot can save its "memory" and find your configuration. + +1. Create the Project Folder: Create a main directory (e.g., gts-federation). +2. Create the Data Subfolder: Inside that folder, create a subfolder named data. This is where the bot stores its database (processed_urls.json). +3. Place your Files: Put your compose.yaml, .env, and rss_feeds.txt into the main directory. + +### Folder Structure: + +Your final structure should look like this: -# Monitor -docker compose logs -f ``` -## ๐Ÿ“ˆ Performance at Scale +gts-federation/ +โ”œโ”€โ”€ data/ <-- Subfolder for the database (Required) +โ”œโ”€โ”€ .env <-- Your credentials & settings +โ”œโ”€โ”€ compose.yaml <-- Docker configuration +โ””โ”€โ”€ rss_feeds.txt <-- Your list of RSS feeds -**Real Production Data:** -``` -๐Ÿ“Š Runtime: 8:42 | 487 posts processed | 3,150+ instances discovered -โšก 56 posts/minute | 102 RSS feeds | +45 new instances per run -๐Ÿ’พ Resource usage: ~450MB RAM total (GoToSocial + tools) ``` -**Scaling Options:** -- **Conservative:** 20 posts/feed (~100 posts/run) -- **Balanced:** 50 posts/feed (~300 posts/run) -- **Aggressive:** 100 posts/feed (~600 posts/run) +### Note on Permissions: -## ๐Ÿ› ๏ธ Configuration Essentials +Ensure the data folder is writable. If you are using a specific UID (e.g., on a Synology NAS), assign ownership to that user: `chown -R 1026:100 ./data` -### Environment Variables (.env) -```bash -# Required -GTS_SERVER_URL=https://your-gts-instance.tld -GTS_ACCESS_TOKEN=your_gts_access_token +### Compose.yaml -# Performance Tuning -MAX_POSTS_PER_RUN=25 # Posts per feed per run -DELAY_BETWEEN_REQUESTS=1 # Seconds between API calls -LOG_LEVEL=INFO # DEBUG for troubleshooting ``` +version: "3" -### RSS Feeds (rss_feeds.txt) -```bash -# Use URL parameters to scale performance -https://mastodon.social/tags/homelab.rss?limit=50 -https://fosstodon.org/tags/selfhosting.rss?limit=100 -https://infosec.exchange/tags/security.rss?limit=75 +services: + gts-federation: + image: domoel/gts-federator:latest + container_name: gts-federator + #user: 1026:100 # If you run into permission issues with the 'data' folder, make sure to set the correct UID/GID for your system. + volumes: + - ./data:/app/data + - ./rss_feeds.txt:/app/rss_feeds.txt:ro + environment: + - TZ=Europe/Berlin # Change to your local timezone. + env_file: + - .env + restart: always ``` - -### GoToSocial Access Token -1. Login to your GoToSocial instance -2. Settings โ†’ Applications โ†’ Create new application -3. Required scopes: `read`, `read:search`, `read:statuses` -4. Copy access token to `.env` file - -## ๐Ÿ“– Complete Documentation - -For detailed information, visit our **[Wiki](https://git.klein.ruhr/matthias/gts-holmirdas/wiki)**: - -- **[๐Ÿ“‹ Installation Guide](https://git.klein.ruhr/matthias/gts-holmirdas/wiki/Installation-Guide.-)** - Detailed setup, Docker configuration, deployment options -- **[๐Ÿ“ˆ Performance & Scaling](https://git.klein.ruhr/matthias/gts-holmirdas/wiki/Performance-%26-Scaling)** - Optimization tables, scaling strategies, resource planning -- **[๐Ÿ› ๏ธ Troubleshooting](https://git.klein.ruhr/matthias/gts-holmirdas/wiki/Troubleshooting)** - Common issues, Docker problems, debugging guide -- **[โš™๏ธ Advanced Configuration](https://git.klein.ruhr/matthias/gts-holmirdas/wiki/Advanced-Configuration)** - Environment variables, RSS strategies, production tips -- **[๐Ÿ“Š Monitoring & Stats](https://git.klein.ruhr/matthias/gts-holmirdas/wiki/Monitoring-%26-Stats)** - Understanding output, health monitoring, metrics -- **[โ“ FAQ](https://git.klein.ruhr/matthias/gts-holmirdas/wiki/FAQ+-+Frequently+Asked+Questions.-)** - Common questions and answers - -## ๐Ÿค Community & Support - -- **[Contributing Guide](Contributing)** - Development setup and contribution guidelines *(coming soon)* -- **Issues**: [Report bugs or request features](https://git.klein.ruhr/matthias/gts-holmirdas/issues) -- **Contact**: [@matthias@me.klein.ruhr](https://me.klein.ruhr/@matthias) on the Fediverse - -## ๐Ÿ”— Related Projects - -- **[FediFetcher](https://github.com/nanos/fedifetcher)** - Fetches missing replies and posts -- **[GoToSocial](https://github.com/superseriousbusiness/gotosocial)** - Lightweight ActivityPub server -- **[slurp](https://github.com/VyrCossont/slurp)** - Import posts from other instances - -## ๐Ÿ“„ License - -MIT License - see [LICENSE](LICENSE) file for details. - -## ๐Ÿ™ Acknowledgments - -- Inspired by [HolMirDas](https://github.com/aliceif/HolMirDas) by [@aliceif](https://mkultra.x27.one/@aliceif) -- Built for the GoToSocial community -- RSS-to-ActivityPub federation approach \ No newline at end of file