Merge branch 'master' into openstack-docs

10 files changed, 271 insertions, 6 deletions

diff --git a/playbooks/provisioning/openstack/advanced-configuration.md b/playbooks/provisioning/openstack/advanced-configuration.md
index c03ca2737..17f0e2629 100644
--- a/playbooks/provisioning/openstack/advanced-configuration.md
+++ b/playbooks/provisioning/openstack/advanced-configuration.md
@@ -243,6 +243,26 @@ via the public IP of the server. You can not send updates via the private
 IP yet. This forces the in-stack private server to have a floating IP.
 See also the [security notes](#security-notes)
 
+## Flannel networking
+
+In order to configure the
+[flannel networking](https://docs.openshift.com/container-platform/3.6/install_config/configuring_sdn.html#using-flannel),
+uncomment and adjust the appropriate `inventory/group_vars/OSEv3.yml` group vars.
+Note that the `osm_cluster_network_cidr` must not overlap with the default
+Docker bridge subnet of 172.17.0.0/16. Or you should change the docker0 default
+CIDR range otherwise. For example, by adding `--bip=192.168.2.1/24` to
+`DOCKER_NETWORK_OPTIONS` located in `/etc/sysconfig/docker-network`.
+
+Also note that the flannel network will be provisioned on a separate isolated Neutron
+subnet defined from `osm_cluster_network_cidr` and having ports security disabled.
+Use the `openstack_private_data_network_name` variable to define the network
+name for the heat stack resource.
+
+After the cluster deployment done, you should run an additional post installation
+step for flannel and docker iptables configuration:
+
+    ansible-playbook openshift-ansible-contrib/playbooks/provisioning/openstack/post-install.yml
+
 ## Other configuration variables
 
 `openstack_ssh_public_key` is a Nova keypair - you can see your
@@ -608,23 +628,24 @@ The first infra node then becomes a bastion node as well and proxies access
 for future ansible commands. The post-provision step also configures Satellite,
 if requested, and DNS server, and ensures other OpenShift requirements to be met.
 
+
 ## Running Custom Post-Provision Actions
 
 A custom playbook can be run like this:
 
 ```
-ansible-playbook -i inventory/ openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions/custom-playbook.yml
+ansible-playbook --private-key ~/.ssh/openshift -i inventory/ openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions/custom-playbook.yml
 ```
 
 If you'd like to limit the run to one particular host, you can do so as follows:
 
 ```
-ansible-playbook -i inventory/ openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions/custom-playbook.yml -l app-node-0.openshift.example.com
+ansible-playbook --private-key ~/.ssh/openshift -i inventory/ openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions/custom-playbook.yml -l app-node-0.openshift.example.com
 ```
 
 You can also create your own custom playbook. Here are a few examples:
 
-#### Adding additional YUM repositories
+### Adding additional YUM repositories
 
 ```
 ---
@@ -648,9 +669,7 @@ This example runs against app nodes. The list of options include:
   - masters
   - infra_hosts
 
-
-
-#### Attaching additional RHN pools
+### Attaching additional RHN pools
 
 ```
 ---
@@ -669,12 +688,42 @@ This playbook runs against all cluster nodes. In order to help prevent slow conn
 problems, the task is retried 10 times in case of initial failure.
 Note that in order for this example to work in your deployment, your servers must use the RHEL image.
 
+### Adding extra Docker registry URLs
+
+This playbook is located in the [custom-actions](https://github.com/openshift/openshift-ansible-contrib/tree/master/playbooks/provisioning/openstack/custom-actions) directory.
+
+It adds URLs passed as arguments to the docker configuration program.
+Going into more detail, the configuration program (which is in the YAML format) is loaded into an ansible variable
+([lines 27-30](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml#L27-L30))
+and in its structure, `registries` and `insecure_registries` sections are expanded with the newly added items
+([lines 56-76](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml#L56-L76)).
+The new content is then saved into the original file
+([lines 78-82](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml#L78-L82))
+and docker is restarted.
+
+Example usage:
+```
+ansible-playbook -i <inventory> openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml  --extra-vars '{"registries": "reg1", "insecure_registries": ["ins_reg1","ins_reg2"]}'
+```
+
+### Adding extra CAs to the trust chain
+
+This playbook is also located in the [custom-actions](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions) directory.
+It copies passed CAs to the trust chain location and updates the trust chain on each selected host.
+
+Example usage:
+```
+ansible-playbook -i <inventory> openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions/add-cas.yml --extra-vars '{"ca_files": [<absolute path to ca1 file>, <absolute path to ca2 file>]}'
+```
+
 Please consider contributing your custom playbook back to openshift-ansible-contrib!
 
 A library of custom post-provision actions exists in `openshift-ansible-contrib/playbooks/provisioning/openstack/custom-actions`. Playbooks include:
 
 * [add-yum-repos.yml](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-yum-repos.yml): adds a list of custom yum repositories to every node in the cluster
 * [add-rhn-pools.yml](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-rhn-pools.yml): attaches a list of additional RHN pools to every node in the cluster
+* [add-docker-registry.yml](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml): adds a list of docker registries to the docker configuration on every node in the cluster
+* [add-cas.yml](https://github.com/openshift/openshift-ansible-contrib/blob/master/playbooks/provisioning/openstack/custom-actions/add-rhn-pools.yml): adds a list of CAs to the trust chain on every node in the cluster
 
 
 ## Install OpenShift
diff --git a/playbooks/provisioning/openstack/custom-actions/add-cas.yml b/playbooks/provisioning/openstack/custom-actions/add-cas.yml
new file mode 100644
index 000000000..b2c195f91
--- /dev/null
+++ b/playbooks/provisioning/openstack/custom-actions/add-cas.yml
@@ -0,0 +1,13 @@
+---
+- hosts: cluster_hosts
+  become: true
+  vars:
+    ca_files: []
+  tasks:
+  - name: Copy CAs to the trusted CAs location
+    with_items: "{{ ca_files }}"
+    copy:
+      src: "{{ item }}"
+      dest: /etc/pki/ca-trust/source/anchors/
+  - name: Update trusted CAs
+    shell: 'update-ca-trust enable && update-ca-trust extract'
diff --git a/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml b/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml
new file mode 100644
index 000000000..e118a71dc
--- /dev/null
+++ b/playbooks/provisioning/openstack/custom-actions/add-docker-registry.yml
@@ -0,0 +1,90 @@
+---
+- hosts: OSEv3
+  become: true
+  vars:
+    registries: []
+    insecure_registries: []
+
+  tasks:
+  - name: Check if docker is even installed
+    command: docker
+
+  - name: Install atomic-registries package
+    yum:
+      name: atomic-registries
+      state: latest
+
+  - name: Get registry configuration file
+    register: file_result
+    stat:
+      path: /etc/containers/registries.conf
+
+  - name: Check if it exists
+    assert:
+      that: 'file_result.stat.exists'
+      msg: "Configuration file does not exist."
+
+  - name: Load configuration file
+    shell: cat /etc/containers/registries.conf
+    register: file_content
+
+  - name: Store file content into a variable
+    set_fact:
+      docker_conf: "{{ file_content.stdout | from_yaml }}"
+
+  - name: Make sure that docker file content is a dictionary
+    when: '(docker_conf is string) and (not docker_conf)'
+    set_fact:
+      docker_conf: {}
+
+  - name: Make sure that registries is a list
+    when: 'registries is string'
+    set_fact:
+      registries_list: [ "{{ registries }}" ]
+
+  - name: Make sure that insecure_registries is a list
+    when: 'insecure_registries is string'
+    set_fact:
+      insecure_registries_list: [ "{{ insecure_registries }}" ]
+
+  - name: Set default values if there are no registries defined
+    set_fact:
+      docker_conf_registries: "{{ [] if docker_conf['registries'] is not defined else docker_conf['registries'] }}"
+      docker_conf_insecure_registries: "{{ [] if docker_conf['insecure_registries'] is not defined else docker_conf['insecure_registries'] }}"
+
+  - name: Add other registries
+    when: 'registries_list is not defined'
+    register: registries_merge_result
+    set_fact:
+      docker_conf: "{{ docker_conf | combine({'registries': (docker_conf_registries + registries) | unique}, recursive=True) }}"
+
+  - name: Add other registries (if registries had to be converted)
+    when: 'registries_merge_result|skipped'
+    set_fact:
+      docker_conf: "{{ docker_conf | combine({'registries': (docker_conf_registries + registries_list) | unique}, recursive=True) }}"
+
+  - name: Add insecure registries
+    when: 'insecure_registries_list is not defined'
+    register: insecure_registries_merge_result
+    set_fact:
+      docker_conf: "{{ docker_conf | combine({'insecure_registries': (docker_conf_insecure_registries + insecure_registries) | unique }, recursive=True) }}"
+
+  - name: Add insecure registries (if insecure_registries had to be converted)
+    when: 'insecure_registries_merge_result|skipped'
+    set_fact:
+      docker_conf: "{{ docker_conf | combine({'insecure_registries': (docker_conf_insecure_registries + insecure_registries_list) | unique }, recursive=True) }}"
+
+  - name: Load variable back to file
+    copy:
+      content: "{{ docker_conf | to_yaml }}"
+      dest: /etc/containers/registries.conf
+
+  - name: Restart registries service
+    service:
+      name: registries
+      state: restarted
+
+  - name: Restart docker
+    service:
+      name: docker
+      state: restarted
diff --git a/playbooks/provisioning/openstack/galaxy-requirements.yaml b/playbooks/provisioning/openstack/galaxy-requirements.yaml
index 93dd14ec2..1d745dcc3 100644
--- a/playbooks/provisioning/openstack/galaxy-requirements.yaml
+++ b/playbooks/provisioning/openstack/galaxy-requirements.yaml
@@ -4,3 +4,7 @@
 # From 'infra-ansible'
 - src: https://github.com/redhat-cop/infra-ansible
   version: master
+
+# From 'openshift-ansible'
+- src: https://github.com/openshift/openshift-ansible
+  version: master
diff --git a/playbooks/provisioning/openstack/net_vars_check.yaml b/playbooks/provisioning/openstack/net_vars_check.yaml
new file mode 100644
index 000000000..68afde415
--- /dev/null
+++ b/playbooks/provisioning/openstack/net_vars_check.yaml
@@ -0,0 +1,14 @@
+---
+- name: Check the provider network configuration
+  fail:
+    msg: "Flannel SDN requires a dedicated containers data network and can not work over a provider network"
+  when:
+    - openstack_provider_network_name is defined
+    - openstack_private_data_network_name is defined
+
+- name: Check the flannel network configuration
+  fail:
+    msg: "A dedicated containers data network is only supported with Flannel SDN"
+  when:
+    - openstack_private_data_network_name is defined
+    - not openshift_use_flannel|default(False)|bool
diff --git a/playbooks/provisioning/openstack/post-install.yml b/playbooks/provisioning/openstack/post-install.yml
new file mode 100644
index 000000000..417813e2a
--- /dev/null
+++ b/playbooks/provisioning/openstack/post-install.yml
@@ -0,0 +1,57 @@
+---
+- hosts: OSEv3
+  gather_facts: False
+  become: True
+  tasks:
+    - name: Save iptables rules to a backup file
+      when: openshift_use_flannel|default(False)|bool
+      shell: iptables-save > /etc/sysconfig/iptables.orig-$(date +%Y%m%d%H%M%S)
+
+# Enable iptables service on app nodes to persist custom rules (flannel SDN)
+# FIXME(bogdando) w/a https://bugzilla.redhat.com/show_bug.cgi?id=1490820
+- hosts: app
+  gather_facts: False
+  become: True
+  vars:
+    os_firewall_allow:
+      - service: dnsmasq tcp
+        port: 53/tcp
+      - service: dnsmasq udp
+        port: 53/udp
+  tasks:
+    - when: openshift_use_flannel|default(False)|bool
+      block:
+        - include_role:
+            name: openshift-ansible/roles/os_firewall
+        - include_role:
+            name: openshift-ansible/roles/lib_os_firewall
+        - name: set allow rules for dnsmasq
+          os_firewall_manage_iptables:
+            name: "{{ item.service }}"
+            action: add
+            protocol: "{{ item.port.split('/')[1] }}"
+            port: "{{ item.port.split('/')[0] }}"
+          with_items: "{{ os_firewall_allow }}"
+
+- hosts: OSEv3
+  gather_facts: False
+  become: True
+  tasks:
+    - name: Apply post-install iptables hacks for Flannel SDN (the best effort)
+      when: openshift_use_flannel|default(False)|bool
+      block:
+        - name: set allow/masquerade rules for for flannel/docker
+          shell: >-
+            (iptables-save | grep -q custom-flannel-docker-1) ||
+            iptables -A DOCKER -w
+            -p all -j ACCEPT
+            -m comment --comment "custom-flannel-docker-1";
+            (iptables-save | grep -q custom-flannel-docker-2) ||
+            iptables -t nat -A POSTROUTING -w
+            -o {{flannel_interface|default('eth1')}}
+            -m comment --comment "custom-flannel-docker-2"
+            -j MASQUERADE
+
+        # NOTE(bogdando) the rules will not be restored, when iptables service unit is disabled & masked
+        - name: Persist in-memory iptables rules (w/o dynamic KUBE rules)
+          shell: iptables-save | grep -v KUBE > /etc/sysconfig/iptables
diff --git a/playbooks/provisioning/openstack/post-provision-openstack.yml b/playbooks/provisioning/openstack/post-provision-openstack.yml
index a80e8d829..e460fbf12 100644
--- a/playbooks/provisioning/openstack/post-provision-openstack.yml
+++ b/playbooks/provisioning/openstack/post-provision-openstack.yml
@@ -76,6 +76,16 @@
   hosts: OSEv3
   gather_facts: true
   become: true
+  vars:
+    interface: "{{ flannel_interface|default('eth1') }}"
+    interface_file: /etc/sysconfig/network-scripts/ifcfg-{{ interface }}
+    interface_config:
+      DEVICE: "{{ interface }}"
+      TYPE: Ethernet
+      BOOTPROTO: dhcp
+      ONBOOT: 'yes'
+      DEFTROUTE: 'no'
+      PEERDNS: 'no'
   pre_tasks:
     - name: "Include DNS configuration to ensure proper name resolution"
       lineinfile:
@@ -83,6 +93,21 @@
         dest: /etc/sysconfig/network
         regexp: "IP4_NAMESERVERS={{ hostvars['localhost'].private_dns_server }}"
         line: "IP4_NAMESERVERS={{ hostvars['localhost'].private_dns_server }}"
+    - name: "Configure the flannel interface options"
+      when: openshift_use_flannel|default(False)|bool
+      block:
+        - file:
+            dest: "{{ interface_file }}"
+            state: touch
+            mode: 0644
+            owner: root
+            group: root
+        - lineinfile:
+            state: present
+            dest: "{{ interface_file }}"
+            regexp: "{{ item.key }}="
+            line: "{{ item.key }}={{ item.value }}"
+          with_dict: "{{ interface_config }}"
   roles:
     - node-network-manager
 
diff --git a/playbooks/provisioning/openstack/prerequisites.yml b/playbooks/provisioning/openstack/prerequisites.yml
index f2f720f8b..11a31411e 100644
--- a/playbooks/provisioning/openstack/prerequisites.yml
+++ b/playbooks/provisioning/openstack/prerequisites.yml
@@ -2,6 +2,9 @@
 - hosts: localhost
   tasks:
 
+  # Sanity check of inventory variables
+  - include: net_vars_check.yaml
+
   # Check ansible
   - name: Check Ansible version
     assert:
diff --git a/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml b/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml
index 970a07815..4c1ca8c96 100644
--- a/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml
+++ b/playbooks/provisioning/openstack/sample-inventory/group_vars/OSEv3.yml
@@ -51,3 +51,9 @@ openshift_override_hostname_check: true
 # NOTE(shadower): Always switch to root on the OSEv3 nodes.
 # openshift-ansible requires an explicit `become`.
 ansible_become: true
+
+# # Flannel networking
+#osm_cluster_network_cidr: 10.128.0.0/14
+#openshift_use_openshift_sdn: false
+#openshift_use_flannel: true
+#flannel_interface: eth1
diff --git a/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml b/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml
index fa1fb6c64..83289307d 100644
--- a/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml
+++ b/playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml
@@ -15,6 +15,10 @@ public_dns_nameservers: []
 openstack_ssh_public_key: "openshift"
 openstack_external_network_name: "public"
 #openstack_private_network_name:  "openshift-ansible-{{ stack_name }}-net"
+# # A dedicated Neutron network name for containers data network
+# # Configures the data network to be separated from openstack_private_network_name
+# # NOTE: this is only supported with Flannel SDN yet
+#openstack_private_data_network_name: "openshift-ansible-{{ stack_name }}-data-net"
 
 ## If you want to use a provider network, set its name here.
 ## NOTE: the `openstack_external_network_name` and