How to set up Django tests to use a free PostgreSQL database in Heroku

Testing Django in Heroku
Image: Testing Django in Heroku (License: CC-BY-SA Marcelo Canina)

Test with a database close to production configuration

Published:

Last modified:

Tags Heroku , Django

Sqlite databases are great for testing purposes, they are fast and don't need much setup, but when your Django app grows, you will need to test in an environment similar to production to spot as much errors as possible.

This guide assumes you have a Django app hosted at Heroku working with a PostgreSQL database add-on in production.

We are going to use PostgreSQL Hobby dev plan to get a free database just for testing against it.

This can be thought as a previous step to use a Continuous Integration service.

For example, for an app I am currently developing, tests summary passed from this (using sqlite):

              Ran 251 tests in 294.559s  FAILED (failures=2, skipped=18) Destroying test database for alias 'default'...

to this (using postgresql):

              ---------------------------------------------------------------------- Ran 249 tests in 1882.815s  FAILED (failures=1, errors=70, skipped=18) Preserving test database for alias 'default'...

1. Create testing DB in Heroku

To create the database: heroku addons:create heroku-postgresql:hobby-dev.

2. Get credentials

Get the credentials using the above database color e.g.: heroku pg:credentials:url HEROKU_POSTGRESQL_BRONZE 

                          $              heroku pg:credentials:url HEROKU_POSTGRESQL_BRONZE              Connection info string:    "dbname=d1jjjjjjj23b host=ec2-54-225-222-121.compute-1.amazonaws.com port=5432 user=myuser password=hash sslmode=require" Connection URL:    postgres://myuser:hash@ec2-54-225-222-121.compute-1.amazonaws.com:5432/d1jjjjjjj23b

Pay attention to the above database user and connection url as we will use it in the following steps.

3. Configure Django db settings

In the DATABASE section of yourapp/settings.py, set the DATABASES['TEST']['NAME'] key with the database name of the new db, e.g. d1jjjjjjj23b:

              DATABASES                =                {                'default': {                'ENGINE':                'django.db.backends.sqlite3',                'NAME': os.path.join(BASE_DIR,                'db.sqlite3'),                'TEST': {                'NAME':                'd1jjjjjjj23b',         },     } }                # Heroku: Update database configuration from $DATABASE_URL.                import                dj_database_url db_from_env                =                dj_database_url.config(conn_max_age=                500) DATABASES['default'].update(db_from_env)

Note the usage of the package: https://github.com/kennethreitz/dj-database-url to get the database connection settings from an environment variable DATABASE_URL

4. Run tests with DATABASE_URL env variable

Set the env variable when running the test, and specify to not create the database (Heroku already created it):

DATABASE_URL="postgres://myuser:hash@ec2-54-225-222-121.compute-1.amazonaws.com:5432/d1jjjjjjj23b" python manage.py test --keepdb

Now the test command will use the above database when running tests.

Possible Errors

If you don't set the NAME key for the testing database, then the test command will complain from being unable to create the database with the message Got an error creating the test database: permission denied to create database, that is why we also add the --keepdb parameter to avoid the testing command to try to create the database again:

                          $              DATABASE_URL="postgres://myuser:hash@ec2-54-225-222-121.compute-1.amazonaws.com:5432/d1jjjjjjj23b" python manage.py test              Creating test database for alias 'default'... /home/marcanuy/.virtualenvs/myapp/lib/python3.6/site-packages/django/db/backends/postgresql/base.py:269: RuntimeWarning: Normally Django will use a connection to the 'postgres' database to avoid running initialization queries against the production database when it's not needed (for example, when running tests). Django was unable to create a connection to the 'postgres' database and will use the first PostgreSQL database instead.   RuntimeWarning Got an error creating the test database: permission denied to create database

References

  • https://docs.djangoproject.com/en/2.1/ref/databases/
  • https://docs.djangoproject.com/en/2.1/topics/testing/
  • https://docs.djangoproject.com/en/2.1/ref/settings/#std:setting-DATABASE-TEST
  • https://docs.djangoproject.com/en/2.1/ref/settings/#std:setting-NAME
  • https://github.com/kennethreitz/dj-database-url

    Articles

    • How to create a reusable package and distribute it with PIP June 29, 2021
    • How To Serve Multiple Django Applications with uWSGI and Nginx in Ubuntu 20.04 October 26, 2020
    • How to add favicon to Django in 4 steps September 3, 2020
    • Categories in Django with Breadcrumbs August 30, 2020
    • How To Migrate From SQLite To PostgreSQL In Django In 3 steps August 28, 2020
    • Practical guide to internationalize a Django app in 5 steps. August 24, 2020
    • Disable new users singup when using Django's allauth package September 3, 2019
    • How to add ads.txt to Django as requested by Google Adsense August 30, 2019
    • Have multiple submit buttons for the same Django form July 2, 2019
    • Better Testing with Page Object Design in Django May 1, 2019
    • Generating slugs automatically in Django without packages - Two easy and solid approaches February 14, 2019
    • How to set up Django tests to use a free PostgreSQL database in Heroku
    • Dynamically adding forms to a Django FormSet with an add button using jQuery February 6, 2019
    • Use of Django's static templatetag in css file to set a background image February 1, 2019
    • Activate Django's manage.py commands completion in Bash in 2 steps January 29, 2019
    • Sending Emails with Django using SendGrid in 3 easy steps January 9, 2019
    • Adding Users to Your Django Project With A Custom User Model September 21, 2018
    • Setting Up A Factory For One To Many Relationships In Factoryboy April 17, 2018
    • Generate UML class diagrams from django models March 24, 2018
    • Set Up Ubuntu To Serve A Django Website Step By Step July 3, 2017
    • Django Project Directory Structure July 16, 2016
    • How to Have Different Django Settings for Development and Production, and environment isolation June 10, 2016
    • Django Overview June 2, 2016

    Subcategories

  • Django Forms
    • Adding a Cancel button in Django class-based views, editing views and forms July 15, 2019
    • Using Django Model Primary Key in Custom Forms THE RIGHT WAY July 13, 2019
    • Django formset handling with class based views, custom errors and validation July 4, 2019
    • How To Use Bootstrap 4 In Django Forms May 25, 2018
    • Understanding Django Forms April 30, 2018
    • How To Create A Form In Django July 29, 2016