Everything you wanted to know about running a lab but were afraid to ask
Stanford University
Roadmap
- What is different about being a PI?
- Managing yourself
- Hiring, managing and mentoring others
- Managing projects
- Managing money
OK, not quite “everything”
- Topics I will not discuss
- Getting a PI position
- Getting tenure
- Publishing
- Writing grants
- Data management
- Teaching
https://www.hhmi.org/science-education/programs/resources/making-right-moves
Where my perspective comes from
Disclaimers
- I have all of the biases inherent in being a lucky survivor
- I have only worked at large, well funded US R1 institutions with generous startup packages
There are almost certainly people who violate each of the rules I will outline and still succeed in academia.
What’s different about being PI vs. being a trainee?
- You are expected to be a leader and a manager
- and less of a “doer”
- The total weight of responsibilty as well as the range of responsibilities will likely increase
What’s not different:
- You are still you
- You will still need mentoring
- You constantly have to learn new things and adapt to changing realities
The hardest problems you will encounter in science are psycho/social, not technical
The hardest problems you will encounter in science are psycho/social, not technical
‘If you find yourself concentrating on the technology rather than the sociology, you’re like the vaudeville character who loses his keys on a dark street and looks for them on the adjacent street because, as he explains, “The light is better there.”’(DeMarco & Lister, Peopleware, 2nd ed.)
The training mismatch for academic PIs
There are many aspects of the PI’s job that we are never trained for!
- Self-management
- Self-care
- Productivity
- Project management and planning
- Leadership
- Personnel management
- Hiring
- Mentoring
- Financial management
- Budgeting and planning
Failure should always be an option
- No one is born with the skills to be a PI
- Anyone can learn them
- This requires getting outside of your comfort zone
- Embrace failure and learn from it
- Constantly seek feedback and use it to change
Managing yourself
- Self-management is essential to thriving as a PI
- What are your personal values and priorities?
- What are your psychological strengths and weaknesses?
- What are your boundaries or absolute no-go areas?
- How open are you to critical feedback and change?
Work/life balance
- You can be successful without working an unreasonable amount
- It requires being highly organized, efficient, and disciplined
- You have to identify your limits up front
Self-care
- Your mental and physical health is key to your long-term success
- Develop routines that will keep you healthy
- Don’t be hesitant to reach out for help
https://chrisneumann.com/blog/the-elephant-in-the-room
The “Goldilocks Zone” of anxiety
Arnsten, 2012
How not to manage anxiety
- Avoid the anxiety-provoking situation
- Assume it will just go away on its own
Establishing a productivity workflow
- Your work responsibilities will be much greater as a PI than as a trainee.
- The only way to thrive as a PI is to have a good system in place to manage them.
- This will require a level of organizational discipline that is beyond what most of you ever needed as a trainee.
Establishing a productivity workflow
- Ruthlessly identify and eliminate unnecessary points of friction in your workflow
- Anything you can automate should be automated
- Any time you find yourself doing something repeatedly, look for a technological solution to make it easier
Risk management
Task management
- GTD (“Getting Things Done”)
- You need a trusted system to organize all of your tasks
How many of you have more than 100 emails in your inbox right now?
Inbox Zero
- One of the worst things you can do is to use your inbox as a todo list
- You should aim for Inbox Zero
Inbox Zero
- Few things feel as satisfying as an empty inbox.
Time management
- You need a single trusted calendar system
- Use a meeting scheduler
- Set reminders for important meetings/events
The messy desk syndrome
Keep your desk clean. Your psyche will thank you.
Motivational surfing
<opinion>Things to say “no” to as a junior PI
- Journal editorial positions
- Book chapters
- Book editor positions
- High-intensity committee memberships (e.g. IRB membership)
- Unwanted collaborations with senior colleagues
https://twitter.com/russpoldrack/status/1626630616040226817
<opinion>Things you should say “yes” to
- Invitations to speak
- If you can’t travel then ask for a virtual talk instead
- Anything that someone at a granting agency asks you to do
- Especially study section or review panel membership
(Some) rules are made to be broken
(Some) rules are made to be broken
Managing a lab effectively requires:
- Good organization
- Good personnel management
- Good financial management
Lab documentation : Outward-facing
Show the world who your lab is.
- Mission/values statements
- Mentorship style/philosophy
- SOPs
http://poldracklab.github.io
Lab documentation: Internal
- The Goal:
- DRY (Don’t Repeat Yourself)
- Onboarding documents
- Lab resources
Building a lab culture
- “How do you do things around here, and why?”
- You are responsible for establishing the culture of the lab
- “The culture of any organization is shaped by the worst behavior the leader is willing to tolerate.” (Grunert & Whitaker)
Annual climate survey
IRB compliance
- A culture of strict compliance is essential
- Never cover up a deviation; report it to the IRB immediately.
- Organized record-keeping for all human subjects is essential
Hiring people: Follow the science
- There is an extensive literature showing that prediction of human behavior based on in-person interviews (known as “clinical prediction”) performs systematically worse than prediction based on data (known as “actuarial prediction”)
“There is no controversy in social science which shows such a large body of qualitatively diverse studies coming out so uniformly in the same direction as this one.” (Meehl, 1986)
“Perhaps the greatest technological achievement in industrial and organizational (I–O) psychology over the past 100 years is the development of decision aids (e.g., paper-and-pencil tests, structured interviews, mechanical combination of predictors) that substantially reduce error in the prediction of employee performance. Arguably, the greatest failure of I–O psychology has been the inability to convince employers to use them.” (Highhouse, 2008)
Debiasing interviews
- Structured interviews outperform unstructured interviews
- “Interviews with higher degrees of structure show higher levels of validity, rater reliability, rater agreement, and less adverse impact.” (US Office of Personnel Management)
- Joint vs separate evaluation
- “Evaluators are more likely to base their decisions on individual performance in joint than in separate evaluation and on group stereotypes in separate than in joint evaluation”(Bohnet et al., 2016)
Effective personnel management
The best thing you can do is to hire good people, inspire them to do good work, keep them happy, and get out of their way!
Behaviors of a good manager
- Lead by example
Behaviors of a good manager
- Lead by example
- Follow through on your promises
Behaviors of a good manager
- Lead by example
- Follow through on your promises
- Listen.
Behaviors of a good manager
- Lead by example
- Follow through on your promises
- Listen.
- Be sensitive to signs of problems
Behaviors of a good manager
- Lead by example
- Follow through on your promises
- Listen.
- Be sensitive to signs of problems
- Recognize the contributions of team members
Behaviors of a good manager
- Lead by example
- Follow through on your promises
- Listen.
- Be sensitive to signs of problems
- Recognize the contributions of team members
- Delegate with trust
Behaviors of a good manager
- Lead by example
- Follow through on your promises
- Listen.
- Be sensitive to signs of problems
- Recognize the contributions of team members
- Delegate with trust
- Being decisive and transparent
Behaviors of a good manager
- Lead by example
- Follow through on your promises
- Listen.
- Be sensitive to signs of problems
- Recognize the contributions of team members
- Delegate with trust
- Being decisive and transparent
- Be open to new ideas
Behaviors of a good manager
- Lead by example
- Follow through on your promises
- Listen.
- Be sensitive to signs of problems
- Recognize the contributions of team members
- Delegate with trust
- Being decisive and transparent
- Be open to new ideas
- Show empathy for your team members
Behaviors of a good manager
- Lead by example
- Follow through on your promises
- Listen.
- Be sensitive to signs of problems
- Recognize the contributions of team members
- Delegate with trust
- Being decisive and transparent
- Be open to new ideas
- Show empathy for your team members
- Be available
Mentoring
- It is essential to develop a culture of mentorship within your lab
- Mentorship should be a two-way relationship
- But the power imbalance often makes it difficult for mentees to say what they really think
- Mentoring is a skill that needs to be honed
https://nap.nationalacademies.org/resource/25568/interactive/
My former trainees
Preparing trainees for the future
- Our job is to train people for interesting and fulfilling careers
- We don’t “lose” people to industry, we “gift” them
- We should help trainees to develop broad skills
- Computing
- Data science
- Personnel management
When people leave
- Academia is a naturally itinerant business
- Be as generous as possible to your trainees when they leave
- Your trainees’ success will shine back on you
Managing projects: The reality
https://medium.com/swlh/the-big-lie-of-good-fast-cheap-fb8905818250
Managing projects: The reality
https://medium.com/swlh/the-big-lie-of-good-fast-cheap-fb8905818250
- The speed-accuracy tradeoff is an iron law of human behavior
- Holding resources constant, going faster will necessarily reduce accuracy
The planning fallacy
- People tend “to underestimate the time required to complete a project, even when they have considerable experience of past failures to live up to planned schedules.” (Kahneman & Tversky, 1979)
- We form an initial optimistic estimate and fail to incorporate knowledge about past problems
- Memory for the duration of past events is systematically underestimated (Roy et al., 2005)
Avoid premature optimization
Donald Knuth
“The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.””
Agile project management
- Reaction to traditional project management approaches (“waterfall” planning)
https://agilemanifesto.org/
Agile project management
- Reaction to traditional project management approaches (“waterfall” planning)
- Principles of agile management for a scientific lab (West, 2018)
- Individuals and interactions over processes and tools
- Seeking improvements over sustaining practices
- Collaboration over competition
- Responding to change over following a plan
https://agilemanifesto.org/
https://www.agilealliance.org/resources/experience-reports/reinventing-research-agile-in-the-academic-laboratory/
Managing finances
- Managing money is one of the most stressful aspects of being a PI
- Your career and your employee’s paychecks lie in the balance
- Overspending is an obvious problem
- But underspending on grants can be almost as bad
- PI’s rarely have the financial management resources that are available in business
- It can be very difficult to get actionable insight into one’s financial situation
Poldracklab budget, 2022
- 3 research scientists
- 4 software engineers
- 5 postdocs
- 5 graduate students
- 2-3 research coordinators
- 2 part-time contractors
Develop a close relationship with your financial manager.
Develop a close relationship with your financial manager.
Keep in mind that they are probably overworked and are doing their best.
Grants
- Your program officer is your advocate and wants you to succeed
- It shines a good light on them and their program
Grants
- Your program officer is your advocate and wants you to succeed
- It shines a good light on them and their program
- “It’s a grant, not a contract” - Sue Corkin
- The work funded by the grant must address the specific aims of the grant.
- It is common to make changes to the specific experiments proposed in the grant.
- Any major changes should be discussed with your Program Officer.
Suzanne Corkin
Grant budgeting
- Never propose to do more than the budget can actually pay for
- Prepare to receive less than you budgeted
- It has been common in recent years for NIH Institutes to fund 10% less than the proposed budget
Conclusions
- Your success as a PI will largely be determined by a combination of non-intellectual factors and luck
- The primary non-intellectual factors under your control are managing yourself and others
- With effort and discipline, any of you can succeed and thrive as a PI
Thanks to Poldracklab members, Ari Beller, and Tyler Bonnen for helpful comments!
Questions?
https://poldrack.github.io/talks-RunningALab/