Everything you wanted to know about running a lab but were afraid to ask

Russ Poldrack

Stanford University

Roadmap

• What is different about being a PI?
• Managing yourself
• Hiring, managing and mentoring others
• Managing projects
• Managing money

OK, not quite “everything”

• Topics I will not discuss
  • Getting a PI position
  • Getting tenure
  • Publishing
  • Writing grants
  • Data management
  • Teaching

https://www.hhmi.org/science-education/programs/resources/making-right-moves

Where my perspective comes from

Disclaimers

• I have all of the biases inherent in being a lucky survivor
• I have only worked at large, well funded US R1 institutions with generous startup packages

There are almost certainly people who violate each of the rules I will outline and still succeed in academia.

What’s different about being PI vs. being a trainee?

• You are expected to be a leader and a manager
  • and less of a “doer”
• The total weight of responsibilty as well as the range of responsibilities will likely increase

What’s not different:

• You are still you
• You will still need mentoring
• You constantly have to learn new things and adapt to changing realities

The hardest problems you will encounter in science are psycho/social, not technical

The hardest problems you will encounter in science are psycho/social, not technical

‘If you find yourself concentrating on the technology rather than the sociology, you’re like the vaudeville character who loses his keys on a dark street and looks for them on the adjacent street because, as he explains, “The light is better there.”’(DeMarco & Lister, Peopleware, 2nd ed.)

The training mismatch for academic PIs

There are many aspects of the PI’s job that we are never trained for!

• Self-management
  • Self-care
  • Productivity
• Project management and planning
• Leadership

• Personnel management
  • Hiring
  • Mentoring
• Financial management
  • Budgeting and planning

Failure should always be an option

• No one is born with the skills to be a PI
• Anyone can learn them
• This requires getting outside of your comfort zone
  • Embrace failure and learn from it
  • Constantly seek feedback and use it to change

Managing yourself

• Self-management is essential to thriving as a PI
  • What are your personal values and priorities?
  • What are your psychological strengths and weaknesses?
  • What are your boundaries or absolute no-go areas?
  • How open are you to critical feedback and change?

Work/life balance

• You can be successful without working an unreasonable amount
  • It requires being highly organized, efficient, and disciplined
• You have to identify your limits up front

Self-care

• Your mental and physical health is key to your long-term success
  • Develop routines that will keep you healthy
• Don’t be hesitant to reach out for help

https://chrisneumann.com/blog/the-elephant-in-the-room

The “Goldilocks Zone” of anxiety

Arnsten, 2012

How not to manage anxiety

• Avoid the anxiety-provoking situation
• Assume it will just go away on its own

Establishing a productivity workflow

• Your work responsibilities will be much greater as a PI than as a trainee.
• The only way to thrive as a PI is to have a good system in place to manage them.
• This will require a level of organizational discipline that is beyond what most of you ever needed as a trainee.

Establishing a productivity workflow

• Ruthlessly identify and eliminate unnecessary points of friction in your workflow
• Anything you can automate should be automated
• Any time you find yourself doing something repeatedly, look for a technological solution to make it easier

Risk management

Identify possible risks to your productivity and do whatever you can to mitigate them

• Keep all important files backed up to the cloud
• If feasible, keep a hot spare laptop that remains synced to your cloud drive
• Make PDFs of any talks and put them in a place where you can easily access them from someone else’s computer if yours dies/gets lost or stolen

Task management

• Google doc, Evernote
• Ultimately a full-featured project management system is probably a good idea
  • e.g. Trello, Notion

• GTD (“Getting Things Done”)
  • You need a trusted system to organize all of your tasks

How many of you have more than 100 emails in your inbox right now?

Inbox Zero

• One of the worst things you can do is to use your inbox as a todo list
• You should aim for Inbox Zero

Inbox Zero

• Few things feel as satisfying as an empty inbox.

Time management

• You need a single trusted calendar system
• Use a meeting scheduler
• Set reminders for important meetings/events

The messy desk syndrome

Keep your desk clean. Your psyche will thank you.

Motivational surfing

• Your motivation to do particular things will wax and wane
• Learn how your personal motivational rhythms work, and surf them
  • Depending on the frequency of your cycles, could require long lead times
• Sometimes you will just have to power through and do something that you don’t feel like doing

<opinion>Things to say “no” to as a junior PI

• Journal editorial positions
• Book chapters
• Book editor positions
• High-intensity committee memberships (e.g. IRB membership)
• Unwanted collaborations with senior colleagues

https://twitter.com/russpoldrack/status/1626630616040226817

<opinion>Things you should say “yes” to

• Invitations to speak
  • If you can’t travel then ask for a virtual talk instead
• Anything that someone at a granting agency asks you to do
  • Especially study section or review panel membership

(Some) rules are made to be broken

(Some) rules are made to be broken

Managing a lab effectively requires:

• Good organization
• Good personnel management
• Good financial management

Lab documentation : Outward-facing

Show the world who your lab is.

• Mission/values statements
• Mentorship style/philosophy
• SOPs

http://poldracklab.github.io

Lab documentation: Internal

• The Goal:
  • DRY (Don’t Repeat Yourself)
• Onboarding documents
• Lab resources

Building a lab culture

• “How do you do things around here, and why?”
• You are responsible for establishing the culture of the lab
• “The culture of any organization is shaped by the worst behavior the leader is willing to tolerate.” (Grunert & Whitaker)

Annual climate survey

IRB compliance

• A culture of strict compliance is essential
  • Never cover up a deviation; report it to the IRB immediately.
• Organized record-keeping for all human subjects is essential

Hiring people: Follow the science

• There is an extensive literature showing that prediction of human behavior based on in-person interviews (known as “clinical prediction”) performs systematically worse than prediction based on data (known as “actuarial prediction”)

“There is no controversy in social science which shows such a large body of qualitatively diverse studies coming out so uniformly in the same direction as this one.” (Meehl, 1986)

“Perhaps the greatest technological achievement in industrial and organizational (I–O) psychology over the past 100 years is the development of decision aids (e.g., paper-and-pencil tests, structured interviews, mechanical combination of predictors) that substantially reduce error in the prediction of employee performance. Arguably, the greatest failure of I–O psychology has been the inability to convince employers to use them.” (Highhouse, 2008)

Debiasing interviews

• Structured interviews outperform unstructured interviews
  • “Interviews with higher degrees of structure show higher levels of validity, rater reliability, rater agreement, and less adverse impact.” (US Office of Personnel Management)
• Joint vs separate evaluation
  • “Evaluators are more likely to base their decisions on individual performance in joint than in separate evaluation and on group stereotypes in separate than in joint evaluation”(Bohnet et al., 2016)

Effective personnel management

The best thing you can do is to hire good people, inspire them to do good work, keep them happy, and get out of their way!

Behaviors of a good manager

• Lead by example

Behaviors of a good manager

• Lead by example
• Follow through on your promises

Behaviors of a good manager

• Lead by example
• Follow through on your promises
• Listen.

Behaviors of a good manager

• Lead by example
• Follow through on your promises
• Listen.
• Be sensitive to signs of problems

Behaviors of a good manager

• Lead by example
• Follow through on your promises
• Listen.
• Be sensitive to signs of problems
• Recognize the contributions of team members

Behaviors of a good manager

• Lead by example
• Follow through on your promises
• Listen.
• Be sensitive to signs of problems
• Recognize the contributions of team members

• Delegate with trust

Behaviors of a good manager

• Lead by example
• Follow through on your promises
• Listen.
• Be sensitive to signs of problems
• Recognize the contributions of team members

• Delegate with trust
• Being decisive and transparent

Behaviors of a good manager

• Lead by example
• Follow through on your promises
• Listen.
• Be sensitive to signs of problems
• Recognize the contributions of team members

• Delegate with trust
• Being decisive and transparent
• Be open to new ideas

Behaviors of a good manager

• Lead by example
• Follow through on your promises
• Listen.
• Be sensitive to signs of problems
• Recognize the contributions of team members

• Delegate with trust
• Being decisive and transparent
• Be open to new ideas
• Show empathy for your team members

Behaviors of a good manager

• Lead by example
• Follow through on your promises
• Listen.
• Be sensitive to signs of problems
• Recognize the contributions of team members

• Delegate with trust
• Being decisive and transparent
• Be open to new ideas
• Show empathy for your team members
• Be available

Mentoring

• It is essential to develop a culture of mentorship within your lab
• Mentorship should be a two-way relationship
  • But the power imbalance often makes it difficult for mentees to say what they really think
• Mentoring is a skill that needs to be honed

https://nap.nationalacademies.org/resource/25568/interactive/

My former trainees

Preparing trainees for the future

• Our job is to train people for interesting and fulfilling careers
  • We don’t “lose” people to industry, we “gift” them
• We should help trainees to develop broad skills
  • Computing
  • Data science
  • Personnel management

When people leave

- Let them take any ideas that they developed in the lab with them, and don't compete with them
- Ask for coauthorship on any work they did in the lab, but set them free after that

• Academia is a naturally itinerant business
• Be as generous as possible to your trainees when they leave
• Your trainees’ success will shine back on you

Managing projects: The reality

https://medium.com/swlh/the-big-lie-of-good-fast-cheap-fb8905818250

Managing projects: The reality

https://medium.com/swlh/the-big-lie-of-good-fast-cheap-fb8905818250

• The speed-accuracy tradeoff is an iron law of human behavior
  • Holding resources constant, going faster will necessarily reduce accuracy

The planning fallacy

• People tend “to underestimate the time required to complete a project, even when they have considerable experience of past failures to live up to planned schedules.” (Kahneman & Tversky, 1979)
  • We form an initial optimistic estimate and fail to incorporate knowledge about past problems
  • Memory for the duration of past events is systematically underestimated (Roy et al., 2005)

Avoid premature optimization

Donald Knuth

“The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.””

Agile project management

• Reaction to traditional project management approaches (“waterfall” planning)

https://agilemanifesto.org/

Agile project management

• Reaction to traditional project management approaches (“waterfall” planning)

• Principles of agile management for a scientific lab (West, 2018)
  • Individuals and interactions over processes and tools
  • Seeking improvements over sustaining practices
  • Collaboration over competition
  • Responding to change over following a plan

https://agilemanifesto.org/

https://www.agilealliance.org/resources/experience-reports/reinventing-research-agile-in-the-academic-laboratory/

Managing finances

• Managing money is one of the most stressful aspects of being a PI
  • Your career and your employee’s paychecks lie in the balance
• Overspending is an obvious problem
  • But underspending on grants can be almost as bad
• PI’s rarely have the financial management resources that are available in business
  • It can be very difficult to get actionable insight into one’s financial situation

Poldracklab budget, 2022

• 3 research scientists
• 4 software engineers
• 5 postdocs
• 5 graduate students
• 2-3 research coordinators
• 2 part-time contractors

Develop a close relationship with your financial manager.

Develop a close relationship with your financial manager.

Keep in mind that they are probably overworked and are doing their best.

Grants

• Your program officer is your advocate and wants you to succeed
  • It shines a good light on them and their program

Grants

• Your program officer is your advocate and wants you to succeed
  • It shines a good light on them and their program
• “It’s a grant, not a contract” - Sue Corkin
  • The work funded by the grant must address the specific aims of the grant.
  • It is common to make changes to the specific experiments proposed in the grant.
    
  • Any major changes should be discussed with your Program Officer.

Suzanne Corkin

Grant budgeting

• Never propose to do more than the budget can actually pay for
• Prepare to receive less than you budgeted
  • It has been common in recent years for NIH Institutes to fund 10% less than the proposed budget

Conclusions

• Your success as a PI will largely be determined by a combination of non-intellectual factors and luck
• The primary non-intellectual factors under your control are managing yourself and others
• With effort and discipline, any of you can succeed and thrive as a PI

Thanks to Poldracklab members, Ari Beller, and Tyler Bonnen for helpful comments!

Questions?