“The Morning Paper”s of the year — 2018 Edition
Continuing in my time-honored tradition since its establishment all the way back in 2017, here is my opinionated list of the best / most interesting / most relevant / most eye-opening papers featured in The Morning Paper by the incredible Adrian Colyer during the course of last year.
AI and such
Concrete problems and AI safety
In between the AI techno-utopians and the AI doomsday prophets lie the AI nerds, just chugging away thinking about the problems. The authors represent some of the marquee institutions working in this area and their reputations ensure that we’ll pay attention. Taxonomies help distill the knowledge of a field into digestible chunks, and this paper delivers with succinct and layman-friendly formulations like “reward hacking”. (They could only take it so far, finally succumbing to “fragility in the face of distributional shift”, when “confusion in unexpected environments” would have done just fine.) Jokes aside, the paper is exceedingly accessible and the authors did a fantastic job with the examples. Speaking of robots that break urns so they can optimize cleaning rewards, the funniest thing to me is that we know robots/agents would have these problems with simple objective functions – after all, don’t humans have those problems all the time? (Cue story about developers that allow bugs into their code so they can get the bug fix bonus.)
EU regulations on algorithmic decision making
When the tech press, the mainstream press, and Hacker News all speak in hushed, respectful tones about an EU regulation — an EU regulation! — you can be sure something’s afoot. Well, what’s afoot is a possible (and possibly wholly unintentional) attack at the heart of an unspoken rule of AI model evaluation: that the proof is in the revenue, all else be damned. Until the regulation is actually tested with a legal challenge, or maybe several legal challenges, its full scope remains a matter of interpretation and speculation. Nevertheless, granting victims (or “subjects”) of algorithms the right to know why a decision was made makes explainability a [m|b]illion-dollar proposition.
Same stats, different graphs
I picked this just for the T. Rex, to be honest.
Neural architecture search with reinforcement learning
This paper was one that highlighted some of the active research happening at the intersection of the tribes of Pedro Domingos The Master Algorithm. Kenneth Stanley popped up quite frequently in this vein with articles, talks, and podcasts on neuroevolution.
Dynamic routing between capsules
A mandatory inclusion. We don’t know yet how impactful this will be, but some of Hinton’s prior work has proved, shall we say with gross understatement, “somewhat useful”.
Automatic database management system tuning
In the boring old world of boring old enterprise systems, administrators are routinely overwhelmed by configuration knobs. Glad to know people are thinking about those administrators as well. Hooray! ML isn’t just about telling hot dogs and golden retrievers apart. Larry Ellison turned the marketing up to 11 with his announcement about Oracle Autonomous Database Cloud at Oracle OpenWorld 2017, but there has been plenty of ongoing research on this topic.
This does bring with it a significant challenge. The more opaque the black box gets — and most enterprise software is a pretty opaque box — the more users are going to miss having some control over the black box’s behavior. This is going to be compounded by the fact that, most likely, even the builders of the black box will often be unable to satisfactorily explain why the black box did what it did.
So what to do? Cue alternatives like Bayesian methods.
BOAT: Building auto-tuners with structured Bayesian optimization
And a second entrant to the topic of system optimizations. One interesting aspect of this paper to me is that it seems to be part of a resurgence in interest in Bayesian methods, partly as a response to the strong desire for explainability in such systems. It may not be all that interesting to “understand” how a model figured out what Van Gogh’s painting style in Starry Nights is, but it sure would be interesting if networking gear decided that terminating a particular connection was “better overall” and is then unable to explain to an irate user how it determined that to be the case.
Software Engineering
On understanding software agility
It doesn’t quite say it this way, but the paper argues that we too often fall into the trap of Survivorship bias — restrospectively ascribing predictive value, where none existed, to various actions that were taken or events that occured. Rather, it would be appropriate to accept the complexity (as opposed to mere complicatedness) of the ssytem — people, business, technology, code, … — and take an approach of taking small steps, observing the effects of that step, studying any changes in the environment, adjusting as necessary, and then taking another small step. ML folks call it “gradient descent”. Software engineering folks call it “Agile”. Leading me back to the point on which I agreed the most, right near the beginning of the paper, which Adrian summarizes thus:
the original spirit of Agile has often been replaced by cargo-culting of rules and checklists
A dissection of the test-driven development process
Here’s a second paper that studies a phenomenon that seems to work pretty well in practice and tries to undergird it with theory. I suspect the paper will be well-received by people on both sides of the religious divide around TDD because the one thing that they agree on is that small batches rule (Toyota, Lean, etc.).
How complex systems fail
Delightfully applicable to software systems even though the author is an MD and writing from a healthcare perspective! Organized as a series of observations, its downright amazing (perhaps this speaks to how limited my knowledge has been) how many of them apply directly to software systems. The best observation of the lot in my opinion is “Complex systems therefore run in degraded mode as their normal mode of operation”, but there are several nuggets of note, e.g., on the dangers that RCA is fraught with, and on the producer/operator balance and what happens when they are the same person. With perhaps just a little dose of confirmation bias, its not hard to see all the ways in which this supports Agile, Lean, and DevOps.
Cloud
Serverless computing: economic and architectural impact
The first of a pair of papers on the paradigm shift that so-called serverless brings. To me, the name is telling: of course there’s a server somewhere executing code, but to the developer, there’s no server. Serverless makes cloud computing all about the code — the developer has to worry about nothing else. Of course, this paper is very nuts-and-bolts, dollars and cents, pros and cons. Think of it as an academic version of a presentation to a CFO and CIO, or the ThoughtWorks Technology Radar. Its not designed to convince anyone of anything, but it places serverless in the realm of the, shall we say, reasonably well understood.
Occupy the cloud
Vinod Khosla, once Silicon Valley royalty, once said at a conference, “Change is the only thing worth optimizing for.” Ladies and gentlemen, I give you: serverless. This second of a pair of papers I liked on the topic is more of a broader vision statement that tries to build support for the assertion that the benefits of a highly decoupled architecture by focusing on the diminishing value of optimizations such as compute/data colocation, noting that “on AWS EC2 writing to remote storage is faster than storing data on a single local SSD”. The paper is unlikely to convince skeptics, but it will definitely weaken their resistance. I do take a bit of issue with the formulation “distributed computing for the 99%”. I would prefer to say “cloud computing for the 99%” — giving those 99% a way to use the cloud without worrying (too much) about distributed computing.
Spanner
This was kind of a mandatory inclusion, what with all the hype around it. Global! Distributed! No ops! ACID! CAP! Atomic clocks! This announcement had coolness value well beyond database geekdom.
BRR: congestion-based congestion control
For the last paper on my list, here’s a nod to what pays my bills — networking. As unsexy as plumbing, as important, and as difficult to clear when things clog up. As a quick aside, buffer bloat is a surprisingly ill-understood problem in networking (“common” sense and all that), and something I learned about from expert colleagues at Citrix when working on WAN Optimizers and such. Adrian delivered possibly the best visualizations of his 2017 blogs explaining the TCP congestion-control problem and how BRR addresses it.
So there you have it. The flood of information continues and thanks to the curation efforts of Adrian Colyer, we can meaningfully digest a chunk of it.
Looking forward to Adrian’s selections for 2018!
Bonus Papers
Use “cloud-native virtualization”. Hypervisor-aware virtualization is so passé. Also known as “cloud is the new operating system”.
While their web scale use is common, counting filters seem to be very useful but grossly underutilized constructs in networking. I wonder why.
