title: How to upgrade my postgres in docker container while maintaining my data? 10.3 to latest 10.x or to 12.x
body: <p>I have a 10.3 postgres docker container in my production and localhost.</p>
  <p>In a previous <a href="https://stackoverflow.com/a/62789347/80353">question</a>, I had to restore a dump that was archived in 10.5. Thanks to the answer I use plainformat to do so. But this is a temporary solution.</p>
  <p>I like to know if there's an easy way to upgrade my postgres version for my docker container in localhost and production.</p>
  <p>In localhost, i have many databases and schema for development and exploration purposes.</p>
  <p>In production, there are far fewer but of course the data is far more important.</p>
  <p>I like to upgrade to a new version of postgres without jeopardizing the data and schema.</p>
  <p>In localhost, my host OS is macOS 10.15 catalina. In production, the host OS is ubuntu headless server edition 18.04</p>
  <p>Both production and localhost use teh same Dockerfile config</p>
  <pre><code>FROM postgres:10.3
  
  COPY ./maintenance /usr/local/bin/maintenance
  RUN chmod +x /usr/local/bin/maintenance/*
  RUN mv /usr/local/bin/maintenance/* /usr/local/bin \
      &amp;&amp; rmdir /usr/local/bin/maintenance
  
  </code></pre>
  <p>I did find this <a href="https://github.com/docker-library/postgres/issues/37#issuecomment-431317584" rel="nofollow noreferrer">https://github.com/docker-library/postgres/issues/37#issuecomment-431317584</a> but I don't have the conceptual understanding of what this comment is suggesting.</p>
  <p>Also I found this library <a href="https://github.com/bwbroersma/docker-postgres-upgrade" rel="nofollow noreferrer">https://github.com/bwbroersma/docker-postgres-upgrade</a></p>
  <p>Not sure how these two approaches are the same or different</p>
  <p>Hope to get someone experienced with both Docker and Postgres for advice here.</p>
  <h2>What I have tried</h2>
  <p>This is my original <code>local.yml</code> for docker. Local because for local development environment.</p>
  <pre><code>version: &quot;3.7&quot;
  
  volumes:
    postgres_data_local: {}
    postgres_backup_local: {}
  
  services:
    django:
      build:
        context: .
        dockerfile: ./compose/local/django/Dockerfile
      image: eno-a3-django_local_django
      depends_on:
        - postgres
        - mailhog
        - redis
      volumes:
        - .:/app
      env_file:
        - ./.envs/.local/.django
        - ./.envs/.local/.postgres
      ports:
        - &quot;8000:8000&quot;
      command: /start
  
    postgres:
      build: ./compose/production/postgres/
      image: eno-a3-django_production_postgres
      volumes:
        - postgres_data_local:/var/lib/postgresql/data
        - postgres_backup_local:/backups
      env_file:
        - ./.envs/.local/.postgres
      ports:
        - &quot;5432:5432&quot;
  
    mailhog:
      image: mailhog/mailhog:v1.0.0
      ports:
        - &quot;8025:8025&quot;
  
    redis:
      build: ./compose/production/redis/
      container_name: redis
      restart: always
  </code></pre>
  <p>And then I thought I would create a new docker container.</p>
  <p>So I changed to this</p>
  <pre><code>version: &quot;3.7&quot;
  
  volumes:
    postgres_data_local: {}
    postgres_backup_local: {}
  
  services:
    django:
      build:
        context: .
        dockerfile: ./compose/local/django/Dockerfile
      image: eno-a3-django_local_django
      depends_on:
        - postgres
        - mailhog
        - redis
        - postgres_new
      volumes:
        - .:/app
      env_file:
        - ./.envs/.local/.django
        - ./.envs/.local/.postgres
      ports:
        - &quot;8000:8000&quot;
      command: /start
  
    postgres:
      build: ./compose/production/postgres/
      image: eno-a3-django_production_postgres
      volumes:
        - postgres_data_local:/var/lib/postgresql/data
        - postgres_backup_local:/backups
      env_file:
        - ./.envs/.local/.postgres
      ports:
        - &quot;5432:5432&quot;
  
    postgres_new:
      build: ./compose/production/postgres_new/
      image: eno-a3-django_production_postgres_new
      volumes:
        - postgres_data_local:/var/lib/postgresql/data
        - postgres_backup_local:/backups
      env_file:
        - ./.envs/.local/.postgres_new
      ports:
        - &quot;15432:5432&quot;
  
    mailhog:
      image: mailhog/mailhog:v1.0.0
      ports:
        - &quot;8025:8025&quot;
  
    redis:
      build: ./compose/production/redis/
      container_name: redis
      restart: always
  </code></pre>
  <p>Notice how I use the same volumes for the <code>postgres_new</code> container.</p>
  <p>The Dockerfile for <code>postgres_new</code> is</p>
  <pre><code>FROM postgres:10.13
  
  COPY ./maintenance /usr/local/bin/maintenance
  RUN chmod +x /usr/local/bin/maintenance/*
  RUN mv /usr/local/bin/maintenance/* /usr/local/bin \
      &amp;&amp; rmdir /usr/local/bin/maintenance
  </code></pre>
  <p>When I run my docker build, and logged in using port 15432, I can see my old database schema etc.</p>
  <p>It appears that both containers can share the same data via the same volume.</p>
  <p>And then I restore into this Docker container using a 10.5 archive file and it succeeded.</p>
  <p>My commands I use to restore are like this in my host OS</p>
  <pre><code>docker cp ~/path/to/10.5.dump eno-a3-django_postgres_new_1:/backups
  docker exec eno-a3-django_postgres_new_1 pg_restore -U debug -d 1013-replicatelive /backups/10.5.dump
  </code></pre>
  <p>Because I thought both containers are talking to the same volume, but when I connect to the old postgres container via 5432, I noticed that the new database that I create via the new postgres container 10.13, it was not there.</p>
  <p>And it appears to work. Can I simply destroy the older container without accidentally destroying my existing data?</p>
  <h2>However..</h2>
  <p>When I changed some database value in a database (that I thought is common in the new postgres container) via port 5432 hence via the old postgres container, the change was not seen in the corresponding database in the new postgres container.</p>
  <h2>After I commented out the old postgres container in local.yml</h2>
  <p>I then only docker up the new postgres container and making it use the host port 5432. I can now see both the new schema (restored via host port 15432) and also the changes in the common database schema. So I guess this method works.</p>
  <p>But why does it work? Is it because the volume is reused?</p>