From e61f584dbc6bf931f5e2433cec07464f17217ea5 Mon Sep 17 00:00:00 2001 From: Jeremy Stanley Date: Sun, 30 Jan 2022 18:54:12 +0000 Subject: [PATCH] Override DOCTYPE in wheel cache autoindex As of pip 22.0, its HTML parser no longer accepts any page which doesn't start with the string "" and, unfortunately, Apache's mod_autoindex declares a very specific HTML 3.2 doctype instead, causing pip to break any time our wheel cache is added to its indices. The main index we generate has been updated with https://review.opendev.org/826969 but we need this change to address Apache's dynamically generated file lists for that site. Configure Apache to supply a custom header file for file indices within the /wheel/ subtree of our mirror vhosts, and alias it from outside the docroot in order to reduce clutter of the top-level directory index. Also instruct mod_autoindex to omit its own document preamble which would otherwise include the original doctype declaration. Note that this omits the header title and H1 level headings from the resulting pages, but as these are only meant for machine parsing anyway and not humans, it's a compromise to keep the solution as simple and straightforward as possible. Change-Id: Id71174954b13b80483256d37f773b781f4956c21 --- playbooks/roles/mirror/files/wheel_header.html | 2 ++ playbooks/roles/mirror/tasks/main.yaml | 8 ++++++++ playbooks/roles/mirror/templates/mirror.vhost.j2 | 8 ++++++++ 3 files changed, 18 insertions(+) create mode 100644 playbooks/roles/mirror/files/wheel_header.html diff --git a/playbooks/roles/mirror/files/wheel_header.html b/playbooks/roles/mirror/files/wheel_header.html new file mode 100644 index 0000000000..ec6dadc919 --- /dev/null +++ b/playbooks/roles/mirror/files/wheel_header.html @@ -0,0 +1,2 @@ + + diff --git a/playbooks/roles/mirror/tasks/main.yaml b/playbooks/roles/mirror/tasks/main.yaml index 47ec6b84cc..46e83ae75e 100644 --- a/playbooks/roles/mirror/tasks/main.yaml +++ b/playbooks/roles/mirror/tasks/main.yaml @@ -105,6 +105,14 @@ group: root mode: 0444 +- name: Install wheel_header.html + copy: + src: wheel_header.html + dest: '{{ www_base }}' + owner: root + group: root + mode: 0444 + - name: Apache proxy cache file: path: /var/cache/apache2/proxy diff --git a/playbooks/roles/mirror/templates/mirror.vhost.j2 b/playbooks/roles/mirror/templates/mirror.vhost.j2 index f13f9c2f9e..44fa04e7fe 100644 --- a/playbooks/roles/mirror/templates/mirror.vhost.j2 +++ b/playbooks/roles/mirror/templates/mirror.vhost.j2 @@ -36,6 +36,14 @@ ErrorLogFormat "[%{cu}t] [%-m:%l] [pid %P:tid %T] %7F: %E: [client\ %a] %M% , \ Require all granted + # Pip really doesn't like the DOCTYPE declaration for autoindexes + # https://github.com/pypa/pip/issues/10825 + Alias /wheel/.header.html /var/www/wheel_header.html + + IndexOptions +SuppressHTMLPreamble + HeaderName /wheel/.header.html + + # Caching reverse proxy for things that don't make sense in AFS # # General cache rules