CWT 2024 Retrospective - predictions with Claude

I recently embarked on a coding project despite being a fairly novice python programmer. I was successful thanks to my cheerful helper named Claude. The goal: make some end of year predictions ahead of the Conversations with Tyler "2024 Retrospective" show on December 25.

CWT is Tyler Cowen's podcast. It's an interview style show with a wide variety of guests across disciplines. Each year, Tyler does a end of year review where he and his producer discuss some of the more popular, controversial, and underrated episodes. I am curious to see if me and Claude can make some decent predictions.

Like many CWT listeners, I am also a Marginal Revolution reader. Tyler writes an MR blog post to introduce each new podcast episode. These posts are titled "My Conversation with..." sometimes qualified with "My Excellent Conversation with..." or "My Conversation with the excellent...". Tyler gives a bit of commentary, followed by a couple excerpts from the transcript, closing with a final bit of commentary, occasionally containing the word "Recommended" in some form.

I've always wondered if the title and guest qualifiers (e.g. "excellent") are clues into his preferences. Also, are the "recommended" episodes the best ones, or does Tyler recommend basically every one? My hypothesis is that these blog posts contain enough information to make some guesses about Tyler's preferences. The resulting blog discussion (comment quantity, not quality!) may help us infer episode popularity or controversy.

The first step was to find all of the MR blog posts introducing a new episode since January 1, 2024. Claude was up to the task, writing a python script that (respectfully) scrapped MR for the date, title, and URLs for each post that had a title format like "My Conversations with..." or "My [qualifier] Conversation with...". It also made a list of qualifiers used to describe the episode, total comment count on the post, as well as determined whether Tyler "recommended" it in the closing paragraph. This script produced a tidy CSV file. I did some manual pruning; some of the blog posts were for other topics (including a funny one from Alex describing how his sons were using his text messages to make a "Dad LLM"). Claude missed the episode with Marc Andreesen which was recorded for an a16z event and released as a "bonus" CWT. I don't consider this a true CWT episode, so I kept it out of the analysis.

Credit to Claude for the data visualization.

We see that Tyler's preferred qualifier is "excellent", and in fact it's basically a default now. The poor souls who received no qualifier were:

• Rebecca F. Kuang
• Fareed Zakaria
• Velina Tchakarova
• Musa al-Gharbi
• Russ Roberts (a different book review-style episode)

This is good company, and at this point I've updated my hypothesis: "excellent" vs "no qualifier" is unlikely to be predictive of Cowen's feeling about the episode. Based on my memory, Tyler had excellent conversations with all of these guests, though I haven't yet listened to the al-Gharbi one.

The Conversation with Joseph Stiglitz was "very good" and the Jonathan Haidt one was "contentious". I have no memory of the Stiglitz conversation. The Haidt one was contentious in a good way and might be one of the most-listened-to episodes and maybe one of Tyler's favourites.

The Haidt episode received the most comments by far (273) with the al-Gharbi episode coming in a distant second (128). MR commenters are easily baited by "culture war" content. I would guess that both of these episodes are in the running for most downloaded.

Interestingly, 13 episodes were "recommended" by Tyler in his closing commentary. I actually thought this would be higher. This is a good list. Before any further analysis, I expect the year's top episode to be one of these:

• Paula Byrne (Dec 13)
• Russ Roberts (Nov 26)
• Tom Tugendhat (Oct 10)
• Kyla Scanlon (Oct 3)
• Tobi Lütke (Sep 19)
• Philip Ball (Sep 5)
• Nate Silver (Aug 22)
• Brian Winter (Jul 11)
• Joseph Stiglitz (Jun 27)
• Peter Thiel (Apr 18)
• Jonathan Haidt (Apr 4)
• Marc Rowan (Mar 7)
• Patrick McKenzie (Jan 12)

Claude then created a script to collect the text from each blog post into a CSV file, splitting it into Tyler's commentary and the episode transcript. I used this data for the next step of the analysis.

At first, I had Claude perform sentiment analysis on the blog post content using the TextBlob library in python. After looking at the scores, I realized this was pointless. There wasn't enough text in Tyler's commentary to get a meaningful score. Also, I had Claude, probably the best tool available to read and analyze text.

I fed Claude the CSV file containing the blog content and began asking it questions. My first question:

Which episodes do you think Tyler liked the best or was most engaged in?

Claude's thoughts (lightly edited): After reading through Tyler's commentary and the transcripts, here are the episodes that seem to have particularly engaged him:

1. Peter Thiel conversation - This stands out because:

• Tyler notes it's "Interesting throughout, definitely recommended"
• The conversation gets deeply philosophical about political theology
• The transcript shows intense back-and-forth intellectual engagement, with Tyler challenging Thiel's views on "muddling through"
• It's notable that this was a return guest from the very first CWT episode

2. Jonathan Haidt conversation - While labeled "contentious," this appears to be a positive:

• Shows unusually vigorous intellectual debate, with Tyler directly challenging Haidt's theories
• The transcript reveals detailed point-counterpoint exchanges
• Tyler explicitly notes it has "a very different tone than most CWTs"

3. Masaaki Suzuki conversation - Tyler's enthusiasm is clear:

• He calls it "Self-recommending, and then some"
• The conversation delves deeply into Bach, clearly a subject Tyler cares about
• The "very excellent" qualifier in the title is unusual (most are just "excellent")

4. Fareed Zakaria conversation - Tyler was notably impressed:

• He specifically praises Zakaria's interview skills: "You can tell he knows what an interview is!"
• Notes Zakaria adapted well to the unique CWT format
• Calls it "Excellent throughout"

I agree with all of these, except maybe the Zakaria one. I felt it was a good episode, but Cowen's comment "You can tell he knows what an interview is!" seemed a bit sarcastic to me, rather than a glowing compliment. It's harder to get substantiative answers from people who are professional interviewers, and this episode was less satisfying for me because of that. I'm curious if Cowen felt the same way.

In this list, which do you think were the most listened to episodes?

Looking at the list, I would predict these episodes had the highest listenership:

1. Peter Thiel

• Major public intellectual with broad name recognition
• Previous episode was the first CWT ever, suggesting proven audience interest
• Covers timely topics like AI and political theology
• Has large following across tech, politics, and academic audiences

2. Nate Silver

• Very well-known name, especially around elections/predictions
• Return guest, suggesting previous episode was popular
• Broad appeal across sports, politics, and data science audiences
• Timely given his recent departure from FiveThirtyEight

3. Jonathan Haidt

• Leading public intellectual on major current issues (social media, mental health)
• Topic (youth mental health) is of broad current interest
• Contentious discussion likely generated word-of-mouth
• Book release timing added relevance

4. Tobi Lütke

• CEO of Shopify, major tech figure
• Appeals to both business/tech audiences
• Discussion of entrepreneurship and management likely draws professional listeners
• Canadian/European perspective adds interest

These predictions are based on:

• Name recognition of guests
• Breadth of potential audience (tech, academic, business, etc.)
• Timeliness/relevance of topics
• Guest's existing platform/following

I think Claude nails it here. These are the biggest names of the year. Last year, Paul Graham was the most downloaded episode because of name recognition and audience (predominantly male, tech-enthusiasts). I'm guessing Thiel takes the prize this year.

Tyler usually discusses the most "underrated" episode; i.e the one that might have been less popular with the listeners but Tyler enjoyed or found it to be of high quality. Which episodes do you think could be his pick for most underrated?

Based on reading the content, here are likely candidates for "underrated" episodes - ones where Tyler's enthusiasm seems high but the comment counts and likely listener numbers might be lower:

1. Marilynne Robinson (52 comments)

• Deep intellectual discussion of theology, literature, and biblical interpretation
• Tyler's commentary suggests strong engagement with the material
• As a literary figure, might draw smaller audience than tech/politics guests
• The transcript reveals sophisticated discussion of religious texts

2. Masaaki Suzuki (19 comments, lowest in dataset)

• Tyler notes it's "Self-recommending, and then some"
• Very specialized discussion of Bach and classical music
• Likely appeals to a niche audience despite high quality
• The transcript shows unique insights into Japanese culture and classical music

3. Benjamin Moser (32 comments)

• Tyler calls it "one of my favorite books on art criticism ever"
• Deep discussion of Dutch masters and art history
• Relatively specialized topic might limit broad appeal
• Quality of discussion seems very high in transcript

4. Paula Byrne (36 comments)

• Tyler notes "one of my favorite episodes in some while"
• Covers literature and British culture in depth
• Described as "one of the most fun — and funny — CWTs of all time"
• Lower comment count suggests smaller audience

Of these, I'd guess Tyler might pick either the Suzuki or Moser conversations as most underrated. Both combine:

• Very specialized knowledge
• Low comment counts suggesting smaller audience
• Tyler's clear enthusiasm for the content
• Unique perspectives that aren't commonly found elsewhere

The Suzuki episode in particular, with only 19 comments but Tyler's strong endorsement and unique subject matter, seems like a prime candidate for "most underrated."

Again, a great pick by Claude. This episode was far in the weeds of classical music, but I could tell Cowen really enjoyed it. I suspect Suzuki had fairly low name recognition when the episode came through the feed, and with few people going back and searching for this episode through word of mouth.

As a side note: one of the things I like best about CWT is being introduced to new subjects and interesting people. I generally enjoy episodes like these despite not being familiar with the guest or subject matter. Tyler does a really good job of bringing out interesting ideas and engaging the guest, and thus the audience. Sometimes my favourite episodes are with guests I've never heard of before.

Which episode do you think was Cowen's favourite to record, irrespective of audience popularity?

After reviewing the blog content thoroughly, I think the Stephen Kotkin episode may have been Cowen's favorite to record. Here are the key signals:

1. Unique enthusiasm in the commentary:

• "It was so much fun we ran over and did about ninety minutes instead of the usual hour"
• This explicit mention of going overtime is rare in Tyler's commentary
• "Excellent throughout" - while he often uses "excellent," the placement at the end after a substantive excerpt suggests genuine enthusiasm

2. The conversation content shown in the excerpt is incredibly rich:

• Deep intellectual discussion about Foucault and power
• Historical analysis of totalitarianism
• Personal anecdotes that illuminate larger themes
• Complex discussion of how power works in everyday life

3. The conversational dynamic:

• The transcript shows Kotkin giving detailed, thoughtful responses
• There's a natural flow between abstract theory and concrete historical examples
• The level of detail and engagement suggests both participants were deeply invested in the discussion

4. Subject matter alignment:

• The conversation combines several of Tyler's key interests: political philosophy, history, power structures, and the nature of totalitarianism
• The discussion moves fluidly between high-level theory and specific historical examples
• There's both intellectual depth and personal narrative

The combination of Tyler explicitly noting they went overtime (which seems rare), the intellectual depth of the conversation, and the natural flow between theory and practice makes this stand out as potentially his favorite to record, even though it might not have been the most popular with listeners (58 comments, middle of the range).

Other contenders might be the Peter Thiel episode (given their long history of intellectual engagement) or the Haidt episode (given the vigorous debate), but the Kotkin conversation seems to have hit a special sweet spot of intellectual depth, personal connection, and natural flow.

Wow, Claude. This was exactly my pick for Tyler too, and probably my favourite from this year. I thought I might have recency bias since this episode came out a few weeks ago, but Claude thinks this was a special one, and I didn't instruct it to weight its analysis based on date of release.

Claude likely doesn't know that Tyler's wife was born in the Soviet Union, something I've gleaned from reading and listening to Tyler over the years. I believe that this personal fact drives some of Tyler's interest and engagement in this topic and it's another reason why I thought that this might be Tyler's favourite.

Closing thoughts

The coding aspect of this project took me about 30 minutes with Claude. I intentionally slowed down the code execution so as not to overwhelm the MR servers, so I'm not including runtime in this calculation. For a fairly novice programmer like myself, this code would have taken me hours to write. It would have been a weekend project. Maybe a couple of weekends, once you take into the account the time it takes to return to your coding context and flow. With Claude's help, I did the bulk of this analysis on a weekday evening after the kids were in bed. It took me longer to write up this post than to write the python scripts. It continues to astonish me what I can do with LLMs. Intentional use of these tools is such a powerful way to raise one's "floor" of abilities.

I didn't preregister my guesses, but Claude was in line with my expectations for the most popular show (Thiel) as well as Tyler's favourite (Kotkin). The most underrated episode was a wild card for me; I didn't have a good prior guess before Claude's analysis. However, I think Claude's choice of the Suzuki episode is excellent.

Finally, thanks to Tyler for creating such a great podcast. I genuinely enjoy every episode, thanks to his thoughtful questions (the conversation that HE wants to have). I'm eagerly awaiting the annual to review to see how these predictions fare.