rixx 🏴 is a user on chaos.social. You can follow them or interact with them if you have an account anywhere in the fediverse.
rixx 🏴 @rixx

Sooo, admins out there: How do you document your (own, private) infrastructure?

I've got lots of ansible roles that work for some part of it. I've tried going with a personal wiki (tiddlywiki), but it felt unintuitive and I tend to neglect it.

· Web · 6 · 9

@rixx dokuwiki with a lot of addons

@rixx I have a git repo with a lot of text files in it. One file with notes per service/component/jail... What ever seems like a good level of abstraction. In addition to the stuff like address or host lists live in their own files. Not the nicest cleanest way of doing it, but it sure beats my old blue notebook that I never had with me when I needed it.

@rixx after a lot of trying (and usually just skipping it after a few days), I ended with Boostnote, because Markdown Syntax, well searchable, multiple locations (for work/private), etc pp.
It's not perfect but the best so far for me, although this mainly covers notes/cheat sheets, not actual documentation.

My private servers are so old that they are almost completely undocumented, but I'm re-doing them at the moment, trying to stick with ansible for almost all of it.

@rixx Mostly I use Emacs with Org-Mode. These are then in multiple files (one per servise) synct with nextcloud. When other people have to use the files too I mostly use Markdow because more people know the syntax

@rixx I have to admit that most of my private systems aren't documented in any way. For non private systems I try to write good documentation but as often I neglect it for my own infra. But at the moment I started to rebuild the older infra, so it's a good opportunity to fix that. I think I will try some markdown in a documentation repository.

@rixx but I think the most important thing is, to have your documentation in the right place and not distributed over to many places. It should be easy to find an information and also easy and hassle-free to use, because if the 'burden' to use it is to high, you will end up without a documentation anyway.

@leah @rixx so true. I started using multiple tools, and you can judge their usability in "days of use until I forgot or actively abandoned them"...

@xpac @rixx jap, usability is one of the main keys. An idea I have since a few years is a small tool with all my infrastructure registered in there and a log like interface where I have to select the system, maybe the service and then write a short summary for every change. Ok, the idea in total is a lot more complex but I think it solves another problem, proper change management.

@leah @xpac @rixx Yeah, and after having all those great ideas, I look at the customers I work at, and then have bad dreams of "file sharing via Outlook", "stuff buried in SharePoint" and other crimes =D

@leah @rixx Speaking of documentation… what should go into it for you to consider it good documentation? IMHO server infrastructure is significantly different from software/programming projects. How could we improve the server documentation side?

@MacLemon @leah From the things that were helpful for me:

- Documentation of concepts, such as the postgres backup documentation
- Good installation documentation such as icinga's, explaining about files, concepts, and common required overrides (e.g. "you may want to override systemd services like this")
- Documented recipes. "A common use cas is x, this is how this tool can be used to solve it". *Very* valuable.
- Explicit maintenance notes (even if it's just 'no maintenance needed')

@leah @rixx if you want to use markdown, mdwiki.info might be a nice thing to use. just a file, parses markdown into a browsable, clickable wiki.

That stuff can then be put right into the ansible-repo as well if one desires, having ansible and docs right next to each other.

@rixx Depends on how hard it was to set up. When I do document then in my personal mediawiki but that's overkill for sure. Sometimes text files in markdown in my personal Gogs are the easiest and best way to do proper documentation that fits my needs.

@rixx after a long time trying various way of documenting my personal infra, like text files with many different formats or some of the configuration management systems hyped during the last decade, and this with at least three different (D)VCS, I am starting to think the problem does not come from the documentation system but from the /personal/ part.

@rixx I have documented things with all kind of text formats or even created a full infra with dozen of servers all deployed entirely with ansible (and transfered its adminstration to a team with minimal effort), all of this because I was either in a team, or alone but working for somebody else than me ...

@rixx So my new goal is to find people with which I could share the burden of administrating services and then put myself in a situation where I effectively document things (and where I can also stop worrying about what will happen when I will be unable to administrate the service).

@rixx plaintext or markdown in git is the approach with the lowest barrier to entry for me, so I'm using that more these days. A wiki is just one more thing to admin...

@rixx a bunch of scratchpads in a private Dokuwiki + config and playbooks in git(ea).

@rixx "The documentation is in the manifests". I've got a GitLab running with a project for my control repository, and I'm opening up implementation issues with drive-by documentation in the comments that I am planning to consolidate into a wiki at some point.

Editing Git(Lab|Hub) wikis is pretty nice since you can just check out project/repo.wiki and throw vim at the problem.

@towo GitHub wikis have Version control? Huh, new to me. But yeah, other than that your method sounds very similar to mine, including the good intentions for future development ;)

Ref. several comments related to the (excellent) Markdown format; here is a #tip ..: Show more

@rixx Usually, I'll document it on a private instance of MediaWiki. If it's something that I think other people might find interesting, I'll remove all the personal/informal bits, and dump it on GitHub as a Gist.