I spent a long while recently looking into identity management solutions. Well, a long while again, recently. I spent a long while about a year ago looking at them before writing one. There’s been a ton of movement inside the IDaaS space, but a lot of this is SAML and stuff like welding together sources, not actual identity providers. Besides, the idea of outsourcing credentials is… unsettling. That’s the kind of trust I’ll selfishly let others prove out and pioneer, and it’s gonna take a long while. That means software I run myself, and just LDAP means software I have to write myself. Multiple pieces of it. Again.
Now this has never been the most active of spaces. Enterprise IDM is hard. Hell, just getting MMR working for the directory component of an IDM is hard. So on prem software for this isn’t something tackled by a lot of small teams – support and feature reqs (per customer even) would be frankly murderous and the margins small. So it’s pretty much just the big enterprise vendors, RedHat, Oracle, IBM and of course Microsoft. They all work fairly similarly, a mixture of DNS, kerberos, LDAP and special sauce. Almost a micro services pattern actually. Anyway, Microsoft pretty much rules the roost here with Active Directory(and you can buy AD as a service, albeit only in Azure). In fact, it’s pretty much Active Directory and then everyone else.
So of course I decided to go with something else.
Fortunately once the decision not to buy something was made the choice is pretty easy. Of the everyone else, one has an open source upstream version, which is RedHat’s Directory Server. That upstream software of course, is FreeIPA. RHDS also has a focus on heterogeneous environments which include Windows so it can interop extensively (including two-way trusts) with Active Directory and has a community which does so. So it’s (relatively) easy to move off of.
Kinda flimsy reasoning, but it worked out. Mostly. I still have some lingering performance and replication problems, but overall a fairly robust setup with multi-DC redundancy was actually pretty trivially (relatively) doable with it. Clustering is hard, so that’s a pretty big achievement.
Anyway, there are a bunch of things I wish I’d known going in:
- Best practice is a max of 4 links per server, with tight coupling by locality, and a max of 20 servers total.
- Plan the topography out first as much as you can. Redoing / relinking a topography (for example because you changed underlying vm locations) seems to be when problems surface most often.
- There are some bad defaults. Like replicating over LDAP and allowing anon binds.
- The weakest component and the one that causes the most replication problems, by far, is dogtag, the CA. In retrospect anything TLS / SSL / PKI should be expected to be a pain point, but this one was pretty bad. I started out hoping to get some vault features out of it as a kind of bonus, and that dream died pretty quick.
- Pretty much everything you care about is stored in the 389 Directory. This includes things like disabling anonymous binds, forcing SSL / 636 connections for replication links ( nsDS5ReplicaPort to 636 and nsDS5ReplicaTransportInfo to SSL in the agreement under cn=replica,cn=${your-directory},cn=mapping tree,cn=config ), and some statuses (e.g. nsds5replicaLastUpdate* in the replica agreement ).
- Because of the above the directory server has to start first. The startup order is thus dirsrv@YOUR-REALM-COM, krb5kdc, kadmin, named-pkcs11, ipa_memcached, httpd, pki-tomcatd, ipa-otpd, ipa-dnskeysyncd.
- This matters because pki-tomcatd, the CA service, will run into problems sometimes and stop the whole stack during an ipactl controlled startup or restart. Starting everything up up to just before it and running an ipa-certupdate is kind of a kill –extreme-prejudice but will generally fix it (as well as pki-tomcatd startup issues in general).
- The 389 service is called dirsrv. The logs are in /var/log/dirsrv. The daemon though is ns-slapd.
- ipactl controls the whole stack
- CA replication and rest of the directory replication are different. They use different methods, have different wrapper commands (e.g. ipa-replica-manage vs. ipa-csreplica-manage), and have their own replication agreements.
- CA replication defaults to 389 / STARTTLS and I couldn’t figure out how to get 636 / SSL working with it. I’m just living with that.