Application maintainers
Set up development environment
Install pyenv.
Install shell completions, e.g. fish: https://gist.github.com/sirkonst/e39bc28218b57cc78b6f728b8da99f33
Good pyenv intro: https://realpython.com/intro-to-pyenv/
Use pyenv to install Python 3.10.4 or later.
Clone WAStD repo.
Use pyenv to create a virtualenv followign https://realpython.com/intro-to-pyenv/
pyenv virtualenv 3.10.4 wastd
with3.10.4
being the latest Python version available at the time of writing.pyenv local wastd
Install dependencies through
poetry install --no-root
.
Alternatives: Develop with Docker / docker-compose following https://docs.docker.com/samples/django/.
Day to day development
Enter WAStD repo, which activates the virtualenv.
TODO this does not load vars from
.env
yet.Ballmer.jpg
Run tests with
fab test
.Deactivate virtualenv with
source deavtivate
.
Add a new dependency:
* poetry add <package>
* poetry run pip list --format=freeze > requirements/base.txt
Release
Pre-release
Write tests.
Write docs (user manual).
Commit changes.
Build Dockerfile locally and test changes:
docker build -t dbcawa/wastd:latest .
orfab dbuild
docker run -it dbcawa/wastd:latest
Portainer is a great UI to run and inspect local Docker images.
Release
Edit
.env
with newWASTD_RELEASE
.Deactivate virtualenv with
source deavtivate
.Activate virtualenv with
poetry env use 3.10.4
to read newWASTD_RELEASE
Run
fab tag
to create a new git tag, push commits, then push the tag.GH Actions will build and publish the Docker image to ghcr.io.
Deploy
Open Rancher UI and edit the UAT config for WAStD workload to the new version number. This will download the Docker image (which can take a few mins), then hot-swap the images.
Apply migrations if any through a shell on the respective workload with
./manage.py migrate
.Once running and tested, edit PROD. Since the Docker image is already downloaded, this step will be fast. Run db migrations if necessary.
Useful commands
Create a postgis database on an available database cluster.:
export CLUSTERNAME=sdis
export CLUSTERPORT=5444
export PG_VERSION=14
sudo -u postgres pg_createcluster -p $CLUSTERPORT $PG_VERSION $CLUSTERNAME --start && \
sudo -u postgres createuser -s $CLUSTERNAME -p $CLUSTERPORT -P && \
sed -i "s/#listen_addresses = 'localhost'/listen_addresses = '*'/" /etc/postgresql/$PG_VERSION/$CLUSTERNAME/postgresql.conf && \
sed -i "s/#wal_level = minimal/wal_level = archive/" /etc/postgresql/$PG_VERSION/$CLUSTERNAME/postgresql.conf && \
sed -i "s/127.0.0.1\/32/10.0.0.0\/8/" /etc/postgresql/$PG_VERSION/$CLUSTERNAME/pg_hba.conf && \
sudo systemctl restart postgresql@$PG_VERSION-$CLUSTERNAME restart
Create a virtualenv project, clone this repo and pip install the requirements:
$ mkproject wastd
(wastd) path/to/wastd:$ git clone git@github.com:dbca-wa/wastd.git
$ pip install -r requirements/dev.txt
$ cp env.example .env
.env must contain a working DATABASE_URL, such as (with your respective DBUSER, DBPASS, HOST, PORT):
DATABASE_URL="postgis://DBUSER:DBPASS@HOST:PORT/DBNAME"
See env.example for other useful settings.
Then, migrate the database and start the app:
$ fab migrate
$ fab go
Source: SO. See also this post.
Chown the entire project to the webserver’s user and group and set permissions to 775 for directories, 664 for files:
sudo chown -R www-data:www-data .
find . -type d -exec sudo chmod 775 {} \;
find . -type f -exec sudo chmod 664 {} \;
This lets the server admin run commands and edit files as long as the admin is in the same group as the webserver, e.g. www-data.
Restoring a snapshot of the production database to a development environment:
(wastd)florianm@aws-eco-001:~/projects/wastd$ pg_dump -h localhost -p 5443 -U sdis -Fc wastd_8220 > data/wastd.dump
(wastd)florianm@aws-eco-001:~/projects/wastd$ rsync -Pavvr data/wastd.dump kens-xenmate-dev:/home/CORPORATEICT/florianm
(wastd)florianm@aws-eco-001:~/projects/wastd$ rsync -Pavvr wastd/media kens-xenmate-dev:/home/CORPORATEICT/florianm/wastd/media
# LOCAL
./manage.py dbshell
# drop database wastd; create database wastd owner sdis; \q
(wastd)florianm@kens-awesome-001:~/projects/wastd⟫ rsync -Pavvr kens-xenmate-dev:/home/CORPORATEICT/florianm/wastd.dump data/
(wastd)florianm@kens-awesome-001:~/projects/wastd⟫ pg_restore -h localhost -p 5444 -U sdis -d wastd < data/wastd.dump
(wastd)florianm@kens-awesome-001:~/projects/wastd⟫ rsync -Pavvr kens-xenmate-dev:/home/CORPORATEICT/florianm/wastd/media wastd/media
Double check file permissions after transfer.