Application maintainers

Set up development environment

Alternatives: Develop with Docker / docker-compose following https://docs.docker.com/samples/django/.

Day to day development

  • Enter WAStD repo, which activates the virtualenv.

  • TODO this does not load vars from .env yet.

  • Ballmer.jpg

  • Run tests with fab test.

  • Deactivate virtualenv with source deavtivate.

Add a new dependency: * poetry add <package> * poetry run pip list --format=freeze > requirements/base.txt

Release

Pre-release

  • Write tests.

  • Write docs (user manual).

  • Commit changes.

  • Build Dockerfile locally and test changes:

    • docker build -t dbcawa/wastd:latest . or fab dbuild

    • docker run -it dbcawa/wastd:latest

  • Portainer is a great UI to run and inspect local Docker images.

Release

  • Edit .env with new WASTD_RELEASE.

  • Deactivate virtualenv with source deavtivate.

  • Activate virtualenv with poetry env use 3.10.4 to read new WASTD_RELEASE

  • Run fab tag to create a new git tag, push commits, then push the tag.

  • GH Actions will build and publish the Docker image to ghcr.io.

Deploy

  • Open Rancher UI and edit the UAT config for WAStD workload to the new version number. This will download the Docker image (which can take a few mins), then hot-swap the images.

  • Apply migrations if any through a shell on the respective workload with ./manage.py migrate.

  • Once running and tested, edit PROD. Since the Docker image is already downloaded, this step will be fast. Run db migrations if necessary.

Useful commands

Create a postgis database on an available database cluster.:

export CLUSTERNAME=sdis
export CLUSTERPORT=5444
export PG_VERSION=14
sudo -u postgres pg_createcluster -p $CLUSTERPORT $PG_VERSION $CLUSTERNAME --start && \
sudo -u postgres createuser -s $CLUSTERNAME -p $CLUSTERPORT -P && \
sed -i "s/#listen_addresses = 'localhost'/listen_addresses = '*'/" /etc/postgresql/$PG_VERSION/$CLUSTERNAME/postgresql.conf && \
sed -i "s/#wal_level = minimal/wal_level = archive/" /etc/postgresql/$PG_VERSION/$CLUSTERNAME/postgresql.conf && \
sed -i "s/127.0.0.1\/32/10.0.0.0\/8/" /etc/postgresql/$PG_VERSION/$CLUSTERNAME/pg_hba.conf && \
sudo systemctl restart postgresql@$PG_VERSION-$CLUSTERNAME restart

Create a virtualenv project, clone this repo and pip install the requirements:

$ mkproject wastd
(wastd) path/to/wastd:$ git clone git@github.com:dbca-wa/wastd.git
$ pip install -r requirements/dev.txt
$ cp env.example .env

.env must contain a working DATABASE_URL, such as (with your respective DBUSER, DBPASS, HOST, PORT):

DATABASE_URL="postgis://DBUSER:DBPASS@HOST:PORT/DBNAME"

See env.example for other useful settings.

Then, migrate the database and start the app:

$ fab migrate
$ fab go

Source: SO. See also this post.

Chown the entire project to the webserver’s user and group and set permissions to 775 for directories, 664 for files:

sudo chown -R www-data:www-data .
find . -type d -exec sudo chmod 775 {} \;
find . -type f -exec sudo chmod 664 {} \;

This lets the server admin run commands and edit files as long as the admin is in the same group as the webserver, e.g. www-data.

Restoring a snapshot of the production database to a development environment:

(wastd)florianm@aws-eco-001:~/projects/wastd$ pg_dump -h localhost -p 5443 -U sdis -Fc wastd_8220 > data/wastd.dump
(wastd)florianm@aws-eco-001:~/projects/wastd$ rsync -Pavvr data/wastd.dump kens-xenmate-dev:/home/CORPORATEICT/florianm
(wastd)florianm@aws-eco-001:~/projects/wastd$ rsync -Pavvr wastd/media kens-xenmate-dev:/home/CORPORATEICT/florianm/wastd/media

# LOCAL
./manage.py dbshell
# drop database wastd; create database wastd owner sdis; \q
(wastd)florianm@kens-awesome-001:~/projects/wastd⟫ rsync -Pavvr kens-xenmate-dev:/home/CORPORATEICT/florianm/wastd.dump data/
(wastd)florianm@kens-awesome-001:~/projects/wastd⟫ pg_restore -h localhost -p 5444 -U sdis -d wastd < data/wastd.dump
(wastd)florianm@kens-awesome-001:~/projects/wastd⟫ rsync -Pavvr kens-xenmate-dev:/home/CORPORATEICT/florianm/wastd/media wastd/media

Double check file permissions after transfer.