Jordan's Space

Technology, music, fishkeeping, me.

News

The Robots Are Coming! Or were they already here?

I’ve worked in IT long enough to see numerous semantics changes over the years. Before I even had my first job, as a teenager, I had dreams of being a “linux systems administrator.” I think with my 2nd job, I actually had that title. Since then, I’ve probably had half a dozen other jobs and with even more titles on top of that, all doing essentially the same thing. Linux systems administration.

Linux itself, it’s changed here and there, but the core unix principals are hard to escape. Simple, interconnected tools. Everything-as-a-file, etc. I certainly saw people fight change along the way, systemd is a good example. I was never in that camp. I lived the pain of hotplug.d, I wanted better, I wanted more. A bunch of shell scripts as an init system was never going to suffice for a modern system, where anything from a USB thumb drive to an entire PCIe bus (thunderbolt) could be a hot-pluggable entity. That’s all here nor there, because end of the day, the core principals are still the same, there’s just more layers. With more sockets (files!), busses, triggers, but still amounts to small, interconnected pieces.

That’s often how my job changed, too. Layers. It’s not that I’m not doing Linux administration anymore, it’s that there’s more layers to it now. You can build an entire career around just one of those layers, without necessarily understanding the rest. Container orchestration platforms come to mind. I think this first stood out to me while interviewing candidates to fill a linux role.

Many candidates, I would have been inclined to call “kubernetes engineers”, before I would have called them “linux engineers”, and coming to this realization was baffling, as it’s almost the polar opposite to my natural curiosity about things. To me, docker, kubernetes, etc, were just a natural progression, on top of linux. I remember the early days of LxC, when VMware was still the norm, I’d wished we had a platform that could do that same orchestration with containers, without the overhead. This was easy to see coming, in hindsight. Why emulate a whole machine, when you can containerize just the pieces needed to run the software? Another layer.

I never called myself a database administrator, yet, thinking back, if I wasn’t running applications or web servers, chances are, I was running databases. I’ve run so many different types of databases, both SQL and NoSQL, data lakes, time series, I’d probably struggle to name them all. For a few years, I was automating some very large ElasticSearch clusters. Search was all the rage. You may see where I’m going with this. With all the prior examples I gave, there was always an obvious “what’s next?” pattern. With search, what came next was vector search. And this, too, wasn’t entirely unexpected. Layer a few more APIs on top of that, and “you can call me AI.”

All while my job title was semantically changing, so was another. First, there was a statisticians, then we had data analysts, next, there was data scientists, sometime after this we had machine learning engineers, and now, AI engineers, which can be anything at this point. However, one thing doesn’t change. Layers. And the best engineers (and leaders!) understand the layers around them, not just the one they are on.

When machine learning was all the rage, I was building temperature controllers in my spare time. That may sound entirely unrelated, and it certainly was to my job, however, the opposite is true when it comes to the types of logic involved. I started with cheap PID controllers, later on, I started playing with some PID + Fuzzy logic devices, which could additionally compensate beyond my initial curve set points. At the time, it seemed like a bit of magic, but then I realized there’s a science to this: Calculus.

Ironically, I never took a calculus class, my first exposure to the topic was likely my father’s big-headed-ness over the subject, apparently he was quite good at it in high school. I won’t pretend to be good at math, or calculus, however, thinking back, I realized, I had another very interesting exposure to calculus over the years, at a past job.

The calculus predictive disk usage check. I remember seeing this code, and wanting to go shake the hand of the guy who wrote it. Of course, in typical fashion, he’d already left the company. He wrote a calculus function to predict disk growth. It stored a trailing log of the disk size over time, and would trend the data using formulas. It was amazing. His job title? It certainly wasn’t AI engineer, machine learning engineer, or anything like that. He was, of course, just another linux systems administrator. All in a days work.

Now, don’t get me started about predicting network administration…..I kid, I kid. IPv6 for you all!