Culture Vulture is an application to present a curated list of cultural events in London.
I have maintained, on-and-off, a curated listing of cultural events in which I'm interested, based on collating things from the various emails and websites of venues like the National Theatre, Barbican, and Sadlers Wells (among many others). At first this was an email list, and then it became a Notion database.
The problem is that trawling these pages is quite time-consuming, and it's easy to forget one. I wanted a website which would do all the trawling and scraping of "What's On" pages and emails for me, and then present a list of events I could review, and approve for display, with all the details pre-filled.
Culture Vulture consists of the following components:
Since scrapers will be run multiple times, there is no way of ensuring that the same events do not turn up twice. Therefore we produce a unique "origination" URI using the following format:
source-key is a fixed string relating to the scraper-parser which
sources the event.
There are two parts to the application: frontend assets (stylesheets and scripts) and the server-side application (which renders HTML or JSON to the client).
Build artefacts are created in the
dist folder, and there are three
rules for doing this:
make scripts: this uses
tscto compile scripts in
make styles: this copies over stylesheets from
make binaries: this uses
stackto build the server and CLI binaries
If you just wish to run culture-vulture, you will require:
You can install GHC and stack using
ghcup which is what we would
You can install the dependencies using
make build-deps, or just build
the entire application using
There is a basic docker-compose file for setting up a postgresql instance for local testing and development. You can run this using:
docker-compose -f docker/compose-local.yaml -p culture-vulture up -d
You will then need to source the local environment variables needed to
connect to it, which you can do by sourcing
Once you have the local environment setup, you can use
which will watch for local file changes and rebuild accordingly, while
also running a development server, or
make dev-server to run a simple
We use docker-compose to run the application in production, so you will
need a host provisioned accordingly. You will need to create a folder
/usr/local/src/culture-vulture to which the user on your host has
read, write, and execute permissions.
docker/SECRETS.md for information on the secrets files you
will need to create in the
docker directory before deploying.
Once this is done, you can simply run
SERVER_ADDRESS=<your server> make deploy, and the application will be deployed complete with letsencrypt
SSL certificates for your chosen domain and an nginx reverse-proxy.
When deploying for the first time, a slightly annoying hack is
needed to work around the fact you have no SSL certificates yet. You
will first need to comment out the second (HTTPS) server block in
and then uncomment it and run
make deploy again once the certificates
have been provisioned.
Unit and integration tests can be found in the
test folder, organised
by module. You can run tests (once you have a local database and
appropriate environment variables) by running
When publishing a release, first ensure that the project builds, especially
Then you need to update the
CHANGELOG.md, and the version numbers in the following files:
Once those changes have been committed and you have verified it all works, use
git tag to tag this commit:
git tag v1.4.0 git push git push --tags
Then make a docker image:
And once it has built, tag the docker image accordingly (note the difference between the git tag
v1.4.0 and the docker tag
1.4.0) and push the image:
docker tag <hash> gtf21/culture-vulture:latest docker tag <hash> gtf21/culture-vulture:1.4.0 docker push -a gtf21/culture-vulture