~rwa/geminispace.info

public search engine for gemini

f5ae878 allow robots.txt served as any text/* mimetype

14 days ago
28 days ago

#Gemini Universal Search (GUS)

#Dependencies

Install the following packages

  • Python (>= 3.11) including distutils and python-dev (aka headers)
  • poetry
  • gcc
  • OpenSSL
  • SQLite3

#basic setup

  1. Create a user who should run the service, e.g. gus
  2. Clone this repo: git clone https://git.sr.ht/~rwa/geminispace.info geminispace
  3. Run poetry install in this directory

#Making an initial index

Make sure you have some gemini URLs for testing which are nicely sandboxed to avoid indexing huge parts of the gemini space.

  1. Create a "seed-requests.txt" file with your test gemini URLs
  2. Run poetry run crawl -d (might take a few hours)
  3. Run poetry run build_index -d

Now you'll have created index.new directory, rename it to index.

#Running the frontend

  1. Run poetry run serve
  2. Navigate your gemini client to: "gemini://localhost/"

#Running the frontend in production with systemd

  1. Create a private key and TLS certificate (X.509 v3) for your host
  2. Update infra/gus.service to match your needs:
  • working directory
  • domain
  • certificate files
  1. Copy infra/gus.service to ~/.config/systemd/user/
  2. Run the following commands to automatically start the daemon
  • systemctl --user daemon-reload
  • systemctl --user enable gus
  • systemctl --user start gus
  1. As root, run loginctl enable-linger gus

#Running the crawl to update the index

  1. Run poetry run crawl
  2. Run poetry run build_index
  3. Run systemctl --user restart gus

#Running the crawl & indexer in production

  1. set up a cron job with the following params: 0 9 * * * <path to your working dir>/infra/update_index.sh <path to your working dir>

#Running the test suite

Run: poetry run pytest

#Troubleshooting

  • restarting gus.service fails with an error that the bus has not been found when called in a cronjob --> some env vars might be missing, prefix the cronjob command with XDG_RUNTIME_DIR="/run/user/1000" DBUS_SESSION_BUS_ADDRESS="unix:path=${XDG_RUNTIME_DIR}/bus"