Culture Vulture is an application to present a curated list of cultural events in London.
I have maintained, on-and-off, a curated listing of cultural events in which I'm interested, based on collating things from the various emails and websites of venues like the National Theatre, Barbican, and Sadlers Wells (among many others). At first this was an email list, and then it became a Notion database.
The problem is that trawling these pages is quite time-consuming, and it's easy to forget one. I wanted a website which would do all the trawling and scraping of "What's On" pages and emails for me, and then present a list of events I could review, and approve for display, with all the details pre-filled.
Culture Vulture consists of the following components:
Since scrapers will be run multiple times, there is no way of ensuring that the same events do not turn up twice. Therefore we produce a unique "origination" URI using the following format:
uri:event-origination:<source-key>:<event-title>:<event-start-date>
The source-key
is a fixed string relating to the scraper-parser which
sources the event.
There are two parts to the application: frontend assets (stylesheets and scripts) and the server-side application (which renders HTML or JSON to the client).
Build artefacts are created in the dist
folder, and there are three
rules for doing this:
make scripts
: this uses tsc
to compile scripts in src/frontend/ts
make styles
: this copies over stylesheets from src/frontend/css
make binaries
: this uses stack
to build the server and CLI binariesIf you just wish to run culture-vulture, you will require:
You can install GHC and stack using ghcup
which is what we would
recommend.
You can install the dependencies using make build-deps
, or just build
the entire application using make binaries
.
There is a basic docker-compose file for setting up a postgresql instance for local testing and development. You can run this using:
docker-compose -f docker/compose-local.yaml -p culture-vulture up -d
You will then need to source the local environment variables needed to
connect to it, which you can do by sourcing local-env.sh
.
Once you have the local environment setup, you can use make dev-watch
which will watch for local file changes and rebuild accordingly, while
also running a development server, or make dev-server
to run a simple
development server.
We use docker-compose to run the application in production, so you will
need a host provisioned accordingly. You will need to create a folder
at /usr/local/src/culture-vulture
to which the user on your host has
read, write, and execute permissions.
Please see docker/SECRETS.md
for information on the secrets files you
will need to create in the docker
directory before deploying.
Once this is done, you can simply run SERVER_ADDRESS=<your server> make deploy
, and the application will be deployed complete with letsencrypt
SSL certificates for your chosen domain and an nginx reverse-proxy.
When deploying for the first time, a slightly annoying hack is
needed to work around the fact you have no SSL certificates yet. You
will first need to comment out the second (HTTPS) server block in
docker/nginx/servers/culturevulture.conf.template
, run make deploy
,
and then uncomment it and run make deploy
again once the certificates
have been provisioned.
Unit and integration tests can be found in the test
folder, organised
by module. You can run tests (once you have a local database and
appropriate environment variables) by running make test
.
When publishing a release, first ensure that the project builds, especially make docker-image
.
Then you need to update the CHANGELOG.md
, and the version numbers in the following files:
Once those changes have been committed and you have verified it all works, use git tag
to tag this commit:
git tag v1.4.0
git push
git push --tags
Then make a docker image:
make docker-image
And once it has built, tag the docker image accordingly (note the difference between the git tag v1.4.0
and the docker tag 1.4.0
) and push the image:
docker tag <hash> gtf21/culture-vulture:latest
docker tag <hash> gtf21/culture-vulture:1.4.0
docker push -a gtf21/culture-vulture