I’m usually teaching myself several new things, and this page is a suggestion from a friend about how to share that learning. If you’re working on similar things, and want to talk about them more or learn together, I’m always excited to chat (especially if your application is socially beneficial)!

Here are some themes of current personal and work projects I’m particularly excited about. Footnotes expand on resources I’m using to learn more:

Personal Projects

1. Scalable Bayesian Inference: What flavors and augmentations of variational inference allow us to most flexibly and reliably scale Bayesian inference to large datasets? Are there other viable tools for scalable Bayes that are less explored? ¹.
2. Modern Survey Experiment Designs: As high quality survey completes grow ever more expensive, can tools like discrete choice models or conjoint experiments more efficiently approximate the same learnings? How can we tell when these methods will and won’t produce the same findings as experiments? ².
3. How Media Shapes Politics: How do social media and TV as platforms shape US Politics? How real are posited polarization effects and other suggested harms, if at all?³.

(Previous) Work Projects

1. Hierarchical Forecasting for Media Planning: Being able to accurately forecast the likely reach of a media plan at a variety of granularities enables better planning; what are the most reliable and accurate forecasting models for our applications? ⁴.
2. Productizing HTE Estimation: Exploring which heterogeneous treatment effect models perform best in our industry in a variety of contexts, with a fully productized solution coming online as of early 2023⁵.
3. Better Polling Methods: Improving our political survey methodology choices to improve our resulting model quality in the 2022 midterm elections and beyond⁶.

I also want to start building out a broader list of topics that I’ve spent time with or want to do a deeper dive in. For example, in 2021 I spent a ton of time exploring Potential Outcome vs DAG approaches to causal inference. In grad school I was particularly focused on the effects of educational polarization in the US, etc. This is both a personal thing (to track what I’ve been interested in over the years), and a social one (I love sharing resources and working with others to learn these topics).

Footnotes

1. There are lots of applications in which I’d prefer a Bayesian model, but where scaling them is simply impractical or unreliable. I started learning more here this out by working through Depth First Learning’s Variational Inference with Normalizing Flows curriculum. To explore more tools in this area, I’m writing a blog post series where I try to get reasonable variational approximations posterior distributions for a basic MRP model, exploring the literature on various flavors and augmentations of VI and diagnostic techniques along the way. I’m particularly excited about normalizing flows-based approaches based on past successes I’ve had in other work. I am also absolutely fascinated by work like Hoffman and Ma (2020) that show Black Box VI actually follows a similar gradient flow to some forms of MCMC, with the resulting algorithmic suggestion that many short chains averaged may be another reasonable formula for scalable Bayes; in practice, I haven’t seen this outperform VI, but would love to see any counter examples!

2. I’m inspired here by Data for Progress/Priorities USA’s testing around replacing in-survey RCTs with a MaxDiff/Best-Worst Scaling deisgn on the discrete choice side. Commonsense Solidarity is a similar inspiration in terms of conjoint experiments that can provide huge amounts of information efficiently. To learn more here, several of the chapters in Advances in Experimental Political Science have been a fantastic starting point for me. Working in advertising where discrete choice modelers commonly rely on Sawtooth, I’ve been thinking about strange contradictions of how it implements MaxDiff ; Jim Savage’s notes and Stan Forum comments here have been super helpful, and I’m working on building Stan implementations of both what Sawtooth chooses to do and the perhaps more reasonable ranked choice random coefficients implementation for the bwsTools R package to get more hands on here.

3. The rough intuition I have here as someone working in TV is that social media gets a lot of hype about how it negatively shapes politics, but if anything, TV is much more likely to be the more negatively impactful platform. Robert Putnam famously theorizes that television is and was responsible for 20% of the recent decay in American social capital; how plausible is that? To see if that intuition is supported by research, I’m working through reading the papers and books mentioned in Chris Bail’s Social Media and Political Dysfunction Collaborative Review, and working on finding similar books and papers that ask the same questions of television.

4. By forecasting reach and impressions of a proposed media plan at any level from coarse network-daypart combinations all the way down to individual household likelihoods of viewing a placed ad, our clients can adjust their plans to meet their goals. Of course, different granularities of forecast are orders of magnitude apart in difficulty and resulting accuracy expectations; how do we combine them into a stronger, coherent picture? Without getting too close to IP, our solution relies on Bayesian time series models that are then reconciled using hierarchical forecasting technqiues to maximize accuracy at all levels.

5. There are a ton of ideas to explore here, but some of the most promising are double/debiased ML estimators, Meta Learners, and other ideas explored in recent ACIC competitions.

6. None of these links will get at any IP, but broadly I’m synthesizing a lot of what I learned from AAPOR 2022 (this thread is a good starting point) and examining how some decent ideas would’ve changed our recent predictions.