Flask Python SQLite AWS EC2 nginx gunicorn

Moving a Flask Python and SQLite application from a Windows desktop to Amazon EC2 instance running nginx and gunicorn

After adding a Webex Teams BOT function to a small team performance tracking application I created it was essentially a requirement to move it to a public server with a fixed IP address / DNS name. To keep it low cost and simple I chose to use a free-tier Amazon AWS EC2 instance running Ubuntu.  While Elastic Beanstalk is a free service AWS designed to simplify these deployments, the additional features deployed with it (S3, load balancing, etc) would only really complicate management.  After a  few failures using Elastic Beanstalk I also decided troubleshooting the deployment process wouldn't be a great use of time.  With that I deployed the server manually, configured the environment and moved the application manually.  Here are the bullet points, not unlike dozens of other tutorials out there:

I am not listing steps to deploy an AWS EC2 instance but here is a sample of what the production server looks like:

I used puTTY and WinSCP  to administer the server and transfer files. You can see see this blog post for a tip to allow file transfers to AWS servers using WinSCP.

The server configuration and application deployment steps were generally as follows:

• Create a requirements.txt file on the desktop to aid in Flask and python configuration on the target server:

                pip3 freeze > requirements.txt

• Decide on the install location o the target server. Here a directory in the default AWS EC2 user ubuntu home directory was used (/home/ubuntu/bttb)

                mkdir /bttb

• I SCP'd my files to the bttb folder using WinSCP here. See the picture above.
• update list of available packages:

                sudo apt update

• upgrade installed packages

                sudo apt upgrade

• install pip for python3

                sudo apt install python3-pip

• install venv to enable virtual environment use

                sudo apt install python3-venv

• create a virtual environment

                python3 -m venv bttb/venv

• activate the virtual environment

                cd bttb
                source venv/bin/activate

• install python requirements

                sudo pip3 install -r requirements.txt

• set environment variable to run Flask app and test

                export FLASK_APP=app_BTTB 

                flask run --host=0.0.0.0

• Flask should now be running the app at your AWS server IP address on port 5000 (default)
• Test web access and fix stuff if necessary.  Then stop Flask wit Ctrl-C.
• install nginx and gunicorn to replace Flask as the web framework
• nginx will be the webserver
• gunicorn will be running python code

                sudo apt install nginx

      sudo apt install gunicorn

• setup nginx and guicorn

                sudo rm /etc/nginx/sites-enabled/default
                sudo nano /etc/nginx/sites-enabled/bttb

• the nginx configuration file contents are like so:

server{

        listen 80;

        server_name 100.25.168.210;

        location /static {

                alias /home/ubuntu/bttb/app_BTTB/static;

        }

        location / {

                proxy_pass http://localhost:8000;

                include /etc/nginx/proxy_params;

                proxy_redirect off;

        }

}

• restart nginx

                sudo service nginx stop

                sudo service nginx start

                check the status with service nginx status

• you should now be able to view files in static folder, served by nginx
• via http://100.25.168.210/static/my_picture.png for example

• See https://serverfault.com/questions/947301/nginx-displaying-failed-13-permission-denied-when-trying-to-access-new-site and sudo -u www-data namei /home/ubuntu/bla/front/dist/vendor.js for issues nginx with permissions
• 
  
• the pages requiring python will fail now though
• test static file access and troubleshoot if required
• now, run gunicorn and specify 'workers' where worker is 2 x number of cores on server
• you should know this from the EC2 deployment or find it via 

                nproc --all

• here the create_app() represents a function in the __init__.py file in the app_BTTB directory

                gunicorn -w 3 'app_BTTB:create_app()'

• You should now have access to the python based pages as well as the static content
• test your application access and troubleshoot if required
• now install supervisor to start the application with the server and keep it running

                sudo apt install supervisor

• create a supervisor config file for your app

                sudo nano /etc/supervisor/conf.d/app_BTTB.conf

• the app_BTTB.conf contents are like so. Note you may need to reference the python virtual environment gunicorn path specifically:

[program:app_BTTB]

user=ubuntu

directory=/home/ubuntu/bttb

command=/home/ubuntu/bttb/venv/bin/gunicorn --chdir /home/ubuntu/bttb 'app_BTTB:create_app()' (note this line may appear word wrapped)

autostart=true

autorestart=true

stopasgroup=true

killasgroup=true

stderr_logfile=/var/log/app_BTTB/app_BTTB.err.log

stdout_logfile=/var/log/app_BTTB/app_BTTB.out.log

• create the supervisor log files referenced above

                sudo mkdir -p /var/log/app_BTTB
                sudo touch /var/log/app_BTTB/app_BTTB.err.log
                sudo touch /var/log/app_BTTB/app_BTTB.out.log

• reload supervisor

                sudo supervisorctl reload

                sudo supervisorctl status  

Assuming all goes as planned, your application is now running, accessible on the internet and running courtesy of nginx and gunicorn.

For reference, the tree of this application looks basically like below, where some folders were removed 

├── app_BTTB

│   ├── __init__.py

│   ├── auth.py

│   ├── db.py

│   ├── schema.sql

│   ├── static

│   │   ├── Boogie.png

│   │   └── Boogie_WebexTeams.png

│   ├── templates

│   │   ├── auth

│   │   ├── base.html

│   │   ├── tracker

│   │   └── webexteams

│   ├── tracker.py

│   └── webexteams.py

├── instance

│   ├── BTTB.sqlite

├── requirements.txt

└── venv

    ├── bin

    │   ├── activate

    │   ├── activate.csh

    │   ├── activate.fish

    │   ├── easy_install

    │   ├── easy_install-3.6

    │   ├── pip

    │   ├── pip3

    │   ├── pip3.6

    │   ├── python -> python3

    │   └── python3 -> /usr/bin/python3

    ├── include

    ├── lib

    │   └── python3.6

    ├── lib64 -> lib

    ├── pyvenv.cfg

    └── share

        └── python-wheels