diff --git a/doc/source/index.rst b/doc/source/index.rst index c489d89..c9360fe 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -63,6 +63,7 @@ section of this index. :glob: :maxdepth: 1 + specs/central-auth specs/irc specs/storyboard_integration_tests specs/storyboard_story_tags diff --git a/specs/central-auth.rst b/specs/central-auth.rst new file mode 100644 index 0000000..0e7a136 --- /dev/null +++ b/specs/central-auth.rst @@ -0,0 +1,447 @@ +:: + + Copyright 2020 OpenStack Foundation + + This work is licensed under a Creative Commons Attribution 3.0 + Unported License. + http://creativecommons.org/licenses/by/3.0/legalcode + +============================== +Central Authentication Service +============================== + +https://storyboard.openstack.org/#!/story/2007604 + +Now that OpenDev is entirely distinct from the OpenStack project, +it's a great time to revisit the single sign-on and central +authentication topic in a new light. We've rehashed and debated this +so very many times in the past 5+ years, but never really officially +documented what we want, our operating constraints, and what options +we seem to have. + +Problem Description +=================== + +Our Web-based services (currently at least Gerrit and StoryBoard, +but probably also MediaWiki, Zanata, maybe Askbot, RefStack, +LimeSurvey, and perhaps soon the Zuul dashboard, Gitea, +Mailman3/Hyperkitty or others) need logins, and we need to be able +to associate accounts across different services with the same +individual. We have traditionally used Launchpad/Ubuntu OpenID for +this. With the addition of non-OpenStack projects, requiring people +to have a UbuntuOne and OpenStackID accounts to use OpenDev services +is less than ideal. However, it's also important for OpenStack that +some individuals can be connected to a corresponding OSF profile for +affiliation and CCLA tracking. + +* We want a central single sign-on system for OpenDev services +* We may want distinct realms for different Zuul tenants +* We do not want it to directly handle authentication credentials +* We want the SSO infrastructure operated within OpenDev +* We want OpenStackID to be one of the available federated IDPs +* We need to be able to migrate current Gerrit and StoryBoard IDs + +Once we're at the finish line in the future, the system would look +like: + +* User wants to log into an OpenDev service +* Service login redirects them to something like opendevid.org +* The opendevid.org interface presents them a choice of identity + providers +* Selecting one of these options bounces them through the + corresponding IDP to authenticate +* Once authenticated, opendevid.org redirects the user back to the + original service +* A user can always go to a URL such as opendevid.org/account and + associate additional identities with their account, so that + they're not limited to just a single external IDP +* Optionally, OIDC tokens could be used for role/group claims to + avoid ACL management within some services (this could come in + handy for managing things like StoryBoard teams) + +It could be argued that an authentication service supporting a local +account database offers additional flexibility so that users still +have an avenue to log in if their external IDP(s) of choice are +suddenly defunct, but we've gotten by for nearly a decade relying on +external IDPs for our services (a mix of Launchpad/UbuntuOne SSO and +OpenStackID), and would prefer not to incur the operational and +legal overhead of securing a database full of user credentials. We +expect to recommend to our users that they affiliate more than one +external IDP with their OpenDev identity, to ensure continuity. +Failing that, we can work with them individually on a best-effort +basis to reestablish access for their accounts when necessary. We +also get pushback from users already to the effect, "Why do I need +yet another account for your services? I should just be able to +authenticate with my existing provider of choice." + +Proposed Change +=============== + +Infra will run a central authentication service, which we'll call +OpenDevID. It will serve as the basis of SSO for all OpenDev +services. OpenDevID will provide both OpenId and OAuth acting as a +single source of authentication for other OpenDev services. The term +OpenDevID and associated opendevid.org domains are used here as +placeholders. If possible, we should attempt to come up with a more +catchy name, one which is ideally also less likely to cause +confusion with OpenStackID. + +OpenDevID will be a pure federated system and not a primary source +of identity management itself. That means that a user will log into +OpenDevID using another credential their choice: raw OpenID, +Github, Google, Twitter, Ubuntu, OpenStackID, whatever. One will +also be able to associate as many other systems as one desires. This +will allow a user to log in to the OpenDevID service using an +account, such as Launchpad/UbuntuOne, then associate accounts from +other systems, such as OpenStackID, as desired. + +Over the years we've evaluated a number of options, some promising +at the time but later left fallow or discovered to be harboring +previously unforeseen impediments for our use case. The greatest +challenge across these has been support for Launchpad/UbuntuOne, due +to its reliance on OpenID v1 which has support in basically none of +the options still available to us. This specification covers +deploying Keycloak for our OpenDevID, with SimpleSAMLphp as a shim +for handling legacy Launchpad/UbuntuOne integration (at least +temporarily for purposes of transition) unless Keycloak gains +upstream support for OpenID v1. + +Alternatives +------------ + +Some of the options we've explored over the years are enumerated here. + +Keystone +~~~~~~~~ + +Keystone is working on becoming a suitable standalone identity +broker, but this is still in progress and likely won't support +OpenID v1. + +Extend an Existing Solution +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Add/write a Launchpad/UbuntuOne specific OpenID driver for the +broker of our choice, perhaps Dex. Preseed its database with the +OpenID mappings we already have, and generate the appropriate +reverse mapping to put into Gerrit. Then configure the broker to +allow people to add other IDP identities to their account, including +OpenStackID, and tell people they have six months to associate +another IDP with their OpenDevID account before we shut off +Launchpad/UbuntuOne as a viable IDP so that we're not stuck +maintaining it forever. + +This has the benefit of being all within our control, but also +involves us writing a chunk of code which may or may not be in a +language we find interesting. It also requires updating Gerrit et al +to authenticate with the broker over something other than OpenID v1, +but this is something we should be doing regardless once we're off +of Launchpad/UbuntuOne. OpenID v1 is pretty dead these days. For +example, if we go with Dex, there is a Dex-specific plugin for +Gerrit already; the OAuth2 plugin for Gerrit also has extensive +support for many identity providers. + +A challenge with the coreos/golang broker was that adding new +providers was not too bad, but there weren't generic providers other +than the openid-connect broker. If we're adding OpenID v1 support +along with a provider the amount of effort is higher. That said, the +fact that there are also provider specific drivers means we could +just take the easy route and implement a Launchpad/UbuntuOne driver +and hardcode things. OpenID itself is pretty simple, so maybe it +wouldn't be that terrible. So far the most promising suggestion for +the general OpenIDv1 problem has been to use simplesamlphp to make +an OpenIDv1 shim (we have contributors who have already done +basically that in the past). + +OpenStackID as a Proxy +~~~~~~~~~~~~~~~~~~~~~~ + +Work with the OpenStackID maintainers to add the ability to +associate Launchpad/UbuntuOne IDs with OpenStackIDs. Have +OpenStackID require a Launchpad/UbuntuOne ID be associated with an +OpenStackID account if the referrer is opendevid.org. Then write a +tiny service that takes an OpenstackID+Launchpad/UbuntuOne ID pair +as an input, that will write that info to the Gerrit and StoryBoard +databases. Add code to OpenStackID which hits that +service when someone logs in from opendevid.org so that any time +someone identifies we collect the mapping and update the user account. +After some time, allow other IDPs than OpenStackID. We won't need to +switch off openid v1 in gerrit if we go this route but we may still +want to. + +This is super hacky, and puts OpenStackID in the critical path for a +period of time, but has the benefit of the development work being +shared with OpenStackID manitainers (other than the tiny little DB +update service). The mapping work for OpenStackID to +Launchpad/UbuntuOne may already be something the OpenStackID +maintainers were looking at, as one of the things they want out of +this is to be able to make that association and initially thought +OpenStackID would do it. If this solution seems appealing, we should +ask them for more info just to be sure this isn't already accounted +for. + +Map Identities via ETL +~~~~~~~~~~~~~~~~~~~~~~ + +Do a behind the scenes OpenID mapping exchange with OpenStackID +based on user E-mail. Pre-generate a set of OpenStackID identities +for each account based on E-mail address. Put those accounts into the +backend DB of opendevid.org. Go ahead and add other IDPs. If someone +logs in from one of the other IDPs and it comes back with a known +E-mail address we have associated with an OpenStackID, make them log +into OpenStackID too proving they are that person, which will +then just add the association. If they log in with not-OpenStackID, +it'll just create a new account. + +In this case we would map Launchpad/UbuntuOne to OpenStackID at a +point in time, then rely on the OpenStackID E-mail matching any +other accounts from that point forward. This would work with +existing brokers because we don't need to OpenID v1, and allows us +to just do a hard cutover using, for example, the generic +openid-connect support in Dex. The biggest question here is, do we +believe that E-mail address matching for the intial mapping will be +good enough that followup support issues will be reasonable to +handle? + +This has the benefit of not needing to write any Launchpad/UbuntuOne +OpenID support code, but has a drawback of the initial database +mapping being potentially incomplete/inexact, so there might be +rectifications that need to be done. + +Help Improve Launchpad/UbuntuOne +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Development seems to have stalled years ago, so we would likely be +left carrying a fork best case. Also, while technically open source, +running our own rebranded version of this would be next to +impossible. On top of that, it's a source of truth for identity, and +likely unsuitable as an IDP broker, so this still sticks us with +maintaining a databse of user credentials and doesn't allow users to +bring their own external identities either. + +Reuse or Rebrand OpenStackID +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We're really looking for a service which delegates to other identity +providers, but OpenStackID is itself a source of truth. Not relying +solely on OpenStackID actually puts us in a good position to get +OpenStackID adoption faster than we would otherwise, since there +will be a legitimate path from Launchpad/UbuntuOne to OpenStackID. +There is no intent to prevent people from pushing changes for review +without first associating an OpenStackID. The purpose of this to +better support non-OpenStack projects, and requiring an "OpenStack +ID" sounds pretty antithetical to that. + +Independent IDPs Across Services +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We can't just have a place for people to write down all their +accounts that may be used across services and then let services use +arbitrary OpenID providers. Not all of our services support the same +protocols, nor do all IDPs, so the intersection of these may be +empty. We'd rather not require users to have more than one identity +and have to remember which one to use for what service. + +Note: this is essentially status quo, it's the situation we're +already in today with some services using Launchpad/UbuntuOne and +others using OpenStackID. + +Implementation +============== + +Bootstrapping +------------- + +We currently have a mapping of Launchpad/UbuntuOne OpenID accounts +with E-mails used for Gerrit. We can use this to pre-populate the +OpenDevID database so that all of our existing users will pre-exist +in the system. We can then ask people to log in and then associate +their OpenStackID account. + +Options +------- + +Five options for implementation have been evaluated: + +* Write our own +* Ipsilon +* Dex +* Hydra +* Keycloak (current consensus choice) + +Write our own +~~~~~~~~~~~~~ + +We probably shouldn't write our own from scratch, unless no other +options are remotely viable. We have enough bespoke software we're +already maintaining, and this is also how OpenStackID came into the +picture. + +Ipsilon +~~~~~~~ + +Ipsilon is awesome, but seems to be more focused on having a user +database (be that FreeIPA, OpenLDAP, et cetera) so can't serve as a +mere aggregation broker for external IDPs. Otherwise it fits the +majority of our criteria. + +https://ipsilon-project.org/doc/intro.html + +We tried our own proof of concept deployment around five years ago, +but it has come a long way since then + +Dex +~~~ + +Dex, from CoreOS, https://github.com/coreos/dex seems like a decent +fit, in that it's designed to provide OAuth2 and can be set up to +delegate auth to another configured provider: + +https://github.com/coreos/dex/blob/master/Documentation/oidc-connector.md + +https://github.com/coreos/dex/blob/master/Documentation/github-connector.md + +Downside to Dex is it's a bit more complex and is focused on being +run in a Kubernetes, so we might have to tease apart some things to +run it not in Kubernetes. There is also already Gerrit integration +written. + +It doesn't seem to support OpenID v1 (at least not after a quick +skim of the docs) so we would have to add that (and hope upstream +takes it) in order to continue supporting Launchpad/UbuntuOne. + +Hydra +~~~~~ + +Hyrdra seems simpler and not Kubernetes focused, + +https://www.ory.am/run-oauth2-server-open-source-api-security + +https://github.com/ory/hydra/tree/master/docs + +but might involve us needing to write some Java code for Gerrit +integration. + +This too doesn't seem to support OpenID v1 so we'd have to implement +it and hope upstream is receptive to those changes as part of +supporting Launchpad/UbuntuOne. + +Gluu ( https://www.gluu.org/ ) is another option we looked into, +though it would need OpenID support added. + +Keycloak +~~~~~~~~ + +Keycloak looks promising, and directly supports identity brokering: + +https://www.keycloak.org/docs/latest/server_admin/index.html#_identity_broker + +It does not support OpenID for authentication or brokering, so we +would potentially need to add this. It's also a large Java +application, but then again we have a fair amount of experience +running these lately anyway (Gerrit, Zanata, Zookeeper...). A +solution discussed at the virtual PTG in April 2020 was to use +https://simplesamlphp.org/ as a go-between identity broker to +provide a SAML authentication proxy for Launchpad/UbuntuOne. + +Based on the above evaluation, Keycloak is the consensus +front-runner. + +Gerrit Integration +------------------ + +We currently use OpenID 1.0 in SSO mode for Gerrit. There is an +OAuth2 provider: + +https://github.com/davido/gerrit-oauth-provider/tree/master/src/main/java/com/googlesource/gerrit/plugins/oauth + +One of the options it already supports is CoreOS Dex, so if we go +that route we should be able to integrate with Gerrit. If we go +Keycloak, Gluu, Ipsilon or Hydra, we might need to work with +gerrit-oauth-provider maintainers to make a more generic OAuth2 +driver or something, though this probably works already or would at +worst be straightforward to add. Gerrit's OAuth plugin supports +keycloak starting in the 2.14 version: + +https://gerrit.googlesource.com/plugins/oauth/+/refs/heads/stable-2.14/README.md + +It's worth noting, StoryBoard already contains OIDC support (and has +been tested with the OIDC implementation in OpenStackID), so this is +less of a concern for it. + +Assignee(s) +----------- + +Primary assignee: + None + +Gerrit Topic +------------ + +Use Gerrit topic "central-auth" for all patches related to this spec. + +.. code-block:: bash + + git-review -t central-auth + +Work Items +---------- + +* Stand up a Keycloak proof of concept deployment +* Work out an UbuntuOne auth proxy for Keycloak using SimpleSAMLphp +* Test the Gerrit OAuth driver's Keycloak support +* Test StoryBoard's OIDC support with the Keycloak PoC +* Write up a migration workflow plan for evaluation +* TODO: additional steps once the above has been knocked out and we + know more + +Repositories +------------ + +Probably one for our SimpleSAMLphp proxy, at a minimum (and perhaps +others). + +Servers +------- + +At least the ID broker service will need to run somewhere and, due +to its sensitivity, almost certainly on its own server isolated from +everything else. + +Basically any of our Web-based services which have public +authentication enabled will also be affected as we switch them to +use this for their single source of identity. + +DNS Entries +----------- + +Based on prior discussions, we should give the identity broker +service a distinct domain rather than putting it in a subdomain of +opendev.org, to minimize risk from cross-site request forgery and +similar sorts of attacks which are easier to pull off between sites +in the same parent domain. + +Documentation +------------- + +At a minimum, the new account creation documentation in the OpenDev +Manual will need updating for our new workflow. + +Security +-------- + +This is essentially the core of our security for user accounts on +public Web services we operate, so pretty much any security +consideration you can think of applies here. + +Testing +------- + +We should integrate the new service into our deployment testing and +exercise it with services whose deployments we already test. + +Dependencies +============ + +TODO: Too early to say, will definitely require deployment +automation and the like.