Let's Play a Game

Of OpenStack and SSL

SSL in vanila OpenStack

The nature of OpenStack projects is largely like projects in Gentoo. Even though they are all under the OpenStack umbrella that doesn't mean they all have to work the same, or even work together.

For instance, nova has the ability to do ssl itself, you can define a CA and public/private keypair. Glance (last time I checked) doesn't do ssl yourself so you must offload it. Other service might do ssl themselves but not in the same way nova does it.

This means that the most 'standard' setup would be to not run ssl, but this isn't exactly desirable. So run a ssl reverse proxy.

Basic Setup

OpenStack services are set up on one host (just in this example).
OpenStack services are configed to listen on localhost only.
Public, Internal and Admin URLs need to be defined with https.
Some tuning needs to be done so services work properly, primarily to glance and nova-novnc.
Nginx is used as the reverse proxy.

Configs and Tuning

General Config for All Services/Sites

This is the basic setup for each of the openstack services, the only difference between them will be what goes in the location subsection.

server {
    listen LOCAL_PUBLIC_IPV4:PORT;
    listen [LOCAL_PUBLIC_IPV6]:PORT;
    server_name = name.subdomain.example.com;
    access_log /var/log/nginx/keystone/access.log;
    error_log /var/log/nginx/keystone/error.log;

    ssl on;
    ssl_certificate /etc/nginx/ssl/COMBINED_PUB_PRIV_KEY.pem;
    ssl_certificate_key /etc/nginx/ssl/COMBINED_PUB_PRIV_KEY.pem;
    add_header Public-Key-Pins 'pin-sha256="PUB_KEY_PIN_SHA"; max-age=2592000; includeSubDomains';
    ssl_dhparam /etc/nginx/params.4096;
    resolver TRUSTED_DNS_SERVER;
    resolver_timeout 5s;
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/nginx/ssl/COMBINED_PUB_PRIV_KEY.pem;
    add_header X-XSS-Protection "1; mode=block";
    add_header Content-Security-Policy "default-src 'self' https: wss:;";
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains; preload";
    add_header X-Frame-Options DENY;
    add_header X-Content-Type-Options nosniff;

    location / {
        # this changes depending on the service
        proxy_pass http://127.0.0.1:PORT;
    }
}

Keystone and Uwsgi

It turns out keystone has switched to uwsgi for it's service backend. This is good because it means we can have the web server connect to that, no more trying to do it all by itself. I'll leave the setting up of uwsgi itself as an exercise to the reader :P

Please keep in mind keystone has two services, admin and main (ports 35357 and 5000 by default).

This config has a few extra things, but it is currently what I know to be 'secure' (similiar config on this blog gets an A+ on all those ssl test things). It's the last location piece that changes the most between services.

location / {
    uwsgi_pass unix:///run/uwsgi/keystone_admin.socket;
    include /etc/nginx/uwsgi_params;
    uwsgi_param SCRIPT_NAME '';
}

Glance

Glance just needs one thing on top of the general proxying that it needs. It needs client_max_body_size 0; in the main server stanza so that you can upload images without being cut off at some low size.

client_max_body_size 0;
location / {
    proxy_pass http://127.0.0.1:9191;
}

Nova

The serviecs for nova just need the basic proxy_pass line. The only exception is novnc, it needs some proxy headers passed.

location / {
    proxy_pass http://127.0.0.1:6080;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header Upgrade $http_upgrade;
}

Rabbitmq

Rabbit is fairly simple, you just need to enable ssl and disable the plaintext port (setting up your config of course).

[
    {ssl, [{versions, ['tlsv1.2', 'tlsv1.1']}]},
    {rabbit, [
        {tcp_listeners, []},
        {ssl_listeners, [5671]},
        {ssl_options, [{cacertfile,"/etc/rabbitmq/ssl/CA_CERT.pem"},
                       {certfile,  "/etc/rabbitmq/ssl/PUB_KEY.pem"},
                       {keyfile,   "PRIV_KEYKEY.key"},
                       {versions, ['tlsv1.2', 'tlsv1.1']}
                      ]
        }]
    }
].

Openstack Configs

The openstack configs can differ slightly but they are all mostly the same now they are using the same libraries (oslo stuff).

General Config

[keystone_authtoken]
auth_uri = https://name.subdomain.example.com:5000
auth_url = https://name.subdomain.example.com:35357

[oslo_messaging_rabbit]
rabbit_host = name.subdomain.example.com
rabbit_port = 5671
rabbit_use_ssl = True

Nova

osapi_compute_listen = 127.0.0.1
metadata_listen = 127.0.0.1
novncproxy_host = 127.0.0.1
enabled_apis = osapi_compute, metadata
[vnc]
novncproxy_base_url = https://name.subdomain.example.com:6080/vnc_auto.html
# the following only on the 'master' host
vncserver_proxyclient_address = 1.2.3.4
vncserver_listen = 1.2.3.4

[glance]
host = name.subdomain.example.com
protocol = https
api_servers = https://name.subdomain.example.com:9292

[neutron]
url = https://name.subdomain.example.com:9696
auth_url = https://name.subdomain.example.com:35357

Cinder

# api-servers get this
osapi_volume_listen = 127.0.0.1

# volume-servers and api-servers get this
glance_api_servers=https://name.subdomain.example.com:9292

Glance

glance_api_servers=https://name.subdomain.example.com:9292

Glance

# api
bind_host = 127.0.0.1
registry_host = name.subdomain.example.com
registry_port = 9191
registry_client_protocol = https

# cache
registry_host = name.subdomain.example.com
registry_port = 9191

# registry
bind_host = 127.0.0.1
rabbit_host = name.subdomain.example.com
rabbit_port = 5671
rabbit_use_ssl = True

# scrubber
registry_host = name.subdomain.example.com
registry_port = 9191

Neutron

# neutron.conf
bind_host = 127.0.0.1
nova_url = https://name.subdomain.example.com:8774/v2

[nova]
auth_url = https://name.subdomain.example.com:35357

# metadata_agent.ini
nova_metadata_ip = name.subdomain.example.com
nova_metadata_protocol = https

social