I really enjoyed the read. I'm an undergrad looking at PhD programs for next year, which means I'm thinking hard about what I want to research, and I'm angling towards the intersection of NLP and education right now. I might get a chance to help build a tutor chatbot. Any advice on what to think about as I decide what to specialize in?
So full disclosure I wrote this article when I'm usually sleeping. Some thoughts on the space:
- I think that the core problems for AI tutors right now are not technical research questions. I think impactful work in the field will either involve improving global LLM performance (through better pretraining or instruction tuning) or pedagogical questions about how to best create or use a tutor with infinite time. My guess is the largest problem is engagement: how do we design chatbots that make students interested in learning. This problem will need to be solved by a mixture of UX research, A/B testing of pedagogical interventions, and possibly even some model tuning to reward engagement.
- Cutting edge research in this space will require models at least as good as GPT-4.
- From this point forward I think all NLP + Education research worth its salt will require studies involving humans. I don't believe automated eval will be able to cut it anymore!
- Given the above two points, NLP + Education research is probably going to be resource intensive!
I think that the list of applications of NLP models that will have a positive impact on the human condition is more than just tutoring. I am blessed with sufficient resources to work on fundamental LLM research, and also enjoy working on technical problems, so I work on that.
Some general thoughts on what you want to specialize in:
- broaden your horizon for what all you could do. Academia is the right choice if it gives you the resources / skills you need to work on important problems
- when doing risky work (which good research is) you want massive upside if everything goes according to plan, and reasonable belief that things could go according to plan. (dream as big as you can while being realistic)
- work on very specific problems, whether technical or nontechnical. One way research mentors are incredibly helpful is to help you identify the specific problems in a field worth working on
The intersection of the above though blurbs:
- either work on a specific problem with existing language models or humans interfacing with ai tutors
- look for places that have the resources to enable you to do the research {gpus for the former, infrastructure for UX tests for the latter}
thanks so much for the advice! I really appreciate it. I think I could enjoy research in improving global LLM performance as well. I was looking at Prof. Kim's lab at MIT; are they accepting PhD students for fall 2024?
For what it's worth, I'm sure the experience you and your brother shared will remain in his mind long after ai tutors become ubiquitous. That is something that will not be lost, and something an ai tutor can not provide.
Also, it's interesting to learn about these parts of your life that I previously had a much more external view of. Thank you for sharing this, Ani. I enjoyed reading this piece very much.
I really enjoyed the read. I'm an undergrad looking at PhD programs for next year, which means I'm thinking hard about what I want to research, and I'm angling towards the intersection of NLP and education right now. I might get a chance to help build a tutor chatbot. Any advice on what to think about as I decide what to specialize in?
So full disclosure I wrote this article when I'm usually sleeping. Some thoughts on the space:
- I think that the core problems for AI tutors right now are not technical research questions. I think impactful work in the field will either involve improving global LLM performance (through better pretraining or instruction tuning) or pedagogical questions about how to best create or use a tutor with infinite time. My guess is the largest problem is engagement: how do we design chatbots that make students interested in learning. This problem will need to be solved by a mixture of UX research, A/B testing of pedagogical interventions, and possibly even some model tuning to reward engagement.
- Cutting edge research in this space will require models at least as good as GPT-4.
- From this point forward I think all NLP + Education research worth its salt will require studies involving humans. I don't believe automated eval will be able to cut it anymore!
- Given the above two points, NLP + Education research is probably going to be resource intensive!
I think that the list of applications of NLP models that will have a positive impact on the human condition is more than just tutoring. I am blessed with sufficient resources to work on fundamental LLM research, and also enjoy working on technical problems, so I work on that.
Some general thoughts on what you want to specialize in:
- broaden your horizon for what all you could do. Academia is the right choice if it gives you the resources / skills you need to work on important problems
- when doing risky work (which good research is) you want massive upside if everything goes according to plan, and reasonable belief that things could go according to plan. (dream as big as you can while being realistic)
- work on very specific problems, whether technical or nontechnical. One way research mentors are incredibly helpful is to help you identify the specific problems in a field worth working on
The intersection of the above though blurbs:
- either work on a specific problem with existing language models or humans interfacing with ai tutors
- look for places that have the resources to enable you to do the research {gpus for the former, infrastructure for UX tests for the latter}
let me know if you have further questions!
thanks so much for the advice! I really appreciate it. I think I could enjoy research in improving global LLM performance as well. I was looking at Prof. Kim's lab at MIT; are they accepting PhD students for fall 2024?
For what it's worth, I'm sure the experience you and your brother shared will remain in his mind long after ai tutors become ubiquitous. That is something that will not be lost, and something an ai tutor can not provide.
Also, it's interesting to learn about these parts of your life that I previously had a much more external view of. Thank you for sharing this, Ani. I enjoyed reading this piece very much.