Learning is Living

Since I've been asked this a few times now, I figured I might as well turn it into a blog post to link people towards it.

To understand perspective: I've now been reading about various learning algorithms for 3+ years in my PhD at EPFL, and actively working on RL research for about a year now. Before that I was working as an ML freelancer and during that time I've given a few workshops trying to vulgarize ML.

My recommended progression would be

  1. Read A (long) peek into Reinforcement Learning to get started, but don't get hung up on the details
  2. Then, concurrently, play with SpinningUp and follow the references to get into Deep RL and read Reinforcement Learning which focuses more on the pre-Deep Learning RL methods and theory. You might also want to check out Stable Baselines, Berkeleys rlpyt and the other RL libraries as references.
  3. Then, once you are actually getting into research reading the notes from CS598, the draft of RL Theory and maybe the excellent Algorithms for Reinforcement Learning are something to keep close for references and deeper understanding. Lillian Wengs excellent blog is also something to keep revisiting.

Some other links (might update this in the future) are:

edit: I liked the book enough that I made a printable worksheet licensed as CC-BY-SA which you can use to implement the 4-DX mentioned below.

I just finished reading “Deep Work” and found it an enjoyably easy read (I think it took me 2-3 hours total) but out of the self-improvement books I've read it's one of the better ones:

  1. it doesn't waste too much time selling you the idea and
  2. quickly moves on to it's actionable items while
  3. not being totalitarian about it and discussing various variants.

The book like many of its contemporary is somewhat US centric (and so assumes ad-decimated attention spans and work styles which would make e.g. a german or french worker scoff both at the intrusion into personal life and at the inefficiencies) and targeted at “knowledge workers” in that setting, so as an european PhD student who doesn't really have a concept of “work” per se (I am lucky enough that I always was interested in doing things for which only need slight tweaking to be monetized, e.g. software engineering, machine learning and research) I'm not quite the core demographic. However, I was still able to get a lot of valuable things out of the book, some of which I'll briefly summarize here (the others go into my journal, since that should hopefully increase my chances at following through). This will be thought snippets, since other people have already summarized sections properly, and you should really read the book for details :–)

Core thesis

The book makes its main thesis very explicit in the introduction, mainly that the world is becoming more and more optimized towards “shallow work” (i.e. work amenable to be done intermittently in small bursts) in contrast to “deep work” (i.e. work which requires focus, concentration and often a “bootup” time), while actually requiring more and more of the latter. It then goes on to present four rules (with a few sub-tasks) which are meant to help cultivate “deep work” as a skill. Summarizing them here in my words:

  1. Work deeply:Pick a deep work philosophy which fits you; ritualize it; make grand gestures where required;don't randomly look for serendipity;execute well; be lazy and shut down
  2. Embrace boredom:take breaks from focus, not from distractions;work intensely, then get bored;meditate productively;control your attention
  3. Quit social media:do a goal-benefit analysis and then think on how to use social media;don't use the internet for mindless entertainment;choose your leisure
  4. Drain the shallows:work smart and hard,not a lot;respect cognitive capacity constraints;schedule every minute but be flexible about rescheduling;quantify depth;set or request a time budget for shallow activities;clear your head by planning how to pick up work at the end of the workday, then stop;manage communication purposefully

Work deeply

This is the core thesis, and I think one of the most important bits the book does is to give clear actionable steps in order to achieve deep work.

Pick a philosophy that fits you

As a sample highlight,the book acknowledges people work differently and hence lays out the following spectrum of deep work styles

  • monastic deep work: live as a sort of hermit, focus your complete (professional) life onto deep tasks. Examples: Richard Feynman, Donald Knuth, Neal Stephenson
  • bimodal deep work: Alternate between chunks of monastic life and shallow life. Examples: Carl Jung, Adam Grant
  • rhythmic deep work: Carve out chunks of your day in which you do deep work, then stick with them. Examples: Brian Chappell, Jerry Seinfeld, Stephen King
  • journalistic deep work:Take whatever time you can get to work deeply, but heavily make use of habits to get into “deep” mode.Walter Isaacson, Cal Newport (ish)

For me, I already do rhythmic deep work, but I might move more towards a bimodal model, as I think the rhythmic method suited my freelance life better than the my current life as a researcher.

Execute well and don't look for random serendipity

These are respectively a recap of the 4DX business strategies in the deep work framework, i.e.

  1. Focus on the wildly important: Set ambitious specific, non VAPID goals (for a quick and nice intro into VAPID, see CGPG's video on misery) to focus on, with a clear payoff and motivation. In Cals example being the publication of 5 papers in high impact journals in the pre-tenure year.
  2. Optimize lead metrics, not lag metrics: lead metrics being things that show you are doing the preliminary work for you goal done, in our case Cal recommends hours of deep work per day
  3. Keep a compelling scoreboard: not only focus on upping those deep work hours, keep track of them, and highlight milestones like parts of a difficult proof. Cal tracks these on a post it on his PC by checking of the hours and highlighting them in case of milestones, allowing judgement on how much deep work/result is required
  4. Create a cadence of accountability: review your scoreboard regularly (Cal recommends weekly) and make adjustments

Step four is the hard bit here by the way (as anyone who journals and occasionally reviews knows).

The point on random serendipity ties into the general theme of deliberate breaking of deep work. Instead of blindly embracing open offices to facilitate overhearing things, it might be a better idea to have long hallways/natural meeting points where people who aren't focusing can run into each other.

Ritualize it, make grand gestures where required, be lazy and shut down

These come down to “standard” cognitive research based self improvement tips like separating “work mode” from “fun mode” by dedicating surroundings and fixed schedules to work, having “bootup” and “shutdown” rituals like planning and reviewing your day. It also highlighted the importance of downtime, complete with actionable tips on how to improve this. The only thing I would have liked would have been some discussion of the importance of unstructured time and on the difficulty of separating work from leisure if your friends work with you as well. The key messages to highlight here are the power of habits and the importance of removing obstacles so you don't have to use willpower (see also the next two sections)

Embrace boredom & Quit social media & Drain the shallows

All of these sections are very much about being deliberate about your time and attention. The key point here is that focus should become the norm, with break from focus being a scheduled thing. This schedule doesn't need to be set in stone, but you do need a schedule to not get drained of into shallow activities and business. Taking breaks and boredom isn't the same thing as mindless distractions etc.

This ties the book very close to the mindfulness practices which became very popular in Silicon Valley and related communities a while ago and seem to have died off again (and obviously have been practiced by Zen people for ages). By steadily practicing focus itself, it becomes easier to apply to work (apparently because of myelination? The book refers to Daniel Coyles slideshow but sadly it is in flash only...I've reached out to Prof. Coyle to see if he could update it).

I appreciated the highlight of downtime as well and liked the bit where Cal essentially describes forest bathing and gives some references onto why it works. What I would have liked would have been some additional discussion of the importance of unstructured downtime, which can still be scheduled.

Remaining comments

  • I really liked the small quip about psychologists “mistrusting everything Gladwellian”
  • the highlight that executives and certain types of work are inherently shallow seemed a bit off to me, because the only justification for why they are not...bad was the high pay of those doing the labour. I think a better framing would be that the deep work these executives do is greasing the grooves, connecting other deep workers and making sure they are productive. While I agree with the author that shallow work isn't lesser work, I think if you can find a way to get into the zone you will generally enjoy it more (and I think it's possible if you have sufficient degree of sovereignty in your labor)
  • the anecdotes about Teddy Roosevelt, Andy Grove, Bill Gates and Mervin Kelly on intensity, execution, taking monastic periods and designing spaces for structured serendipity were very inspiring to me
  • The book points out how deep work is something very rewarding, without needing to be pleasant...which makes the next point I'd like to highlight more poignant
  • A big chunk of the obstacles towards deep work described in the book only exist because of advertising, power imbalances and workplace dynamics. This means knowledge workers with some leverage, or in countries with sane labor laws will be much more likely to be able to do deep work....while those in weaker position might be stuck in hostile environments. Since deep work is rewarding and less stressful to most people, I kind of expect it to correlate with mental health, which makes this an example of an unjust and inefficient system in my eyes. So we have work cut out for us to make sure everyone can choose to engage in shallow or deep work, not be forced to do either

I thought I had it all figured out, just don't do anything immoral, or anything you wouldn't be comfortable with airing out if forced, and you are immune to blackmail. But what happens if you are forced to do something you think is justified, but your workplace and other important factors of your life might not react well to, because you are not as lucky as me?

I'm a white, middle class, mostly heterosexual cis-male, in a very liberal country, in a very liberal industry without any dependents. I do not have to deal with being judged for abortions, with being subtly discriminated for my sexuality or gender, or my race, or any other factor. I can easily reject employers which won't treat me with respect, but what if I couldn't? How would I deal with that? What if for example someone judged one of my hobbies as “bad for the orgs image” and “had to” distance the org from me by firing me? Or if I had an abortion and would have to face harassment because of this? Owning it wouldn't be possible, and my integrity would not matter.

This insight might seem trivial to you, but I had to be pushed face first into this by some of my friends. Which I guess says something about my naivete. Thank god I have friends who'll call me on my bullshit.

Aside from driving home the point for strong privacy laws and technology, and protective laws making sure employers don't have any business snooping around in he private life, this thing has me chewing on this problem now:

How should one deal in such a situation? One in which there one had to do something not illegal, not necessarily immoral, but something other people with power of you could possibly treat you worse if they knew.

How could one make sure one becomes immune against blackmail again? What systemic protection could we come up with to protect people in such a setting? If you have good solutions, please mail me at blogcontact@krawczuk.eu

As another year passes, staying true to my word, I'm still haunted by a small owls cry. With it memories of a lovely mind, and all our pieces we made,changed,shared broke and left behind. Struggling to make and let go a history of love and hurt and so much more. Still remains wishing to see it grow. To see her soar

Lee Sharkey, a friend of mine just published a wondeful guide on neuroscience for AI/ML researchers, focusing on

The functions that particular brain areas are believed to perform and, where possible, how they are believed to perform them. The high-level algorithms the brain is believed to be implementing and what we know about how they are performed Relations between the discussed brain parts and systems Links from the neuroscience to modern AI algorithms and architectures.

Enter your email to subscribe to updates.