I'm pushing myself to publish more often, trying to push through my perfectionism. I'm publicly committing to making three posts in February, ideally more. This post has less editing than I usually do, so feel free to give feedback however you like.I also changed the name of my substack.
The paper in question
YOLOv3: An Incremental Improvement is one of pieces of research writing I think about most often. Its not even a paper, but a tech report, created so that the author (Joseph Redmon) would have something to cite for changes made to a prior version of the model. The paper's eschews normal section titles for phrases like "The Deal" and "Things that Didn't work". When I first read it in 2019 I was working in efficient networks, and was shocked how good the report was: it was fun to read, informative, well written, and short. It didn't have the fluff needed to pass peer review, and it was better for it. Like many of my those in my field, I have had a love/hate relationship with the ML research as long as I've been a part of it. The second reason I like this paper is that Redmon speaks honestly about the perils of ML research. 1
In the beginning, Redmon writes that he has had an unproductive year, mostly doing iterative improvements to YOLO and helping out with other projects. One common and underdiscussed aspect of academia is how much productivity oscillates: the raw amount of work a researcher does can change week to week, month to month, and year to year. Success for input of work is equally temperamental, as is recognition for that success. Put that all together, and someone who publishes ten papers one year might not publish at all the next.
Due to these inconsistent returns, internal motivation is necessary for success. But internal motivation often fades for many reasons. In the last section, Redmon hints at one common one in machine learning: moral qualms about the impact of your work.
But maybe a better question is: “What are we going to do with these detectors now that we have them?” A lot of the people doing this research are at Google and Facebook. I guess at least we know the technology is in good hands and definitely won’t be used to harvest your personal information and sell it to.... wait, you’re saying that’s exactly what it will be used for?? Oh. Well the other people heavily funding vision research are the military and they’ve never done anything horrible like killing lots of people with new technology oh wait... [1]
[1] The author is funded by the Office of Naval Research and Google
I have felt this. During the pandemic, I worked on recommender systems for around a year at Berkeley. While the problem itself was interesting, I realized in the later stages of the project which companies was interested in my work. I realized that, on the margin, I was encouraging just a little more consumption out of a would be shopper, and making someone spend just a bit more time browsing a site they probably didn't consciously want to be on. Was that the impact I wanted to have? Afterwards, I subconsciously avoided the project, focusing on other projects until I graduated. Just thinking about the project gave me a sinking feeling. Once I came to MIT, I mostly forgot about it. Furthermore, I became more thoughtful and assertive about what I wanted to work on.
I suspect that Redmon had a similar moment around the time he was writing this report, but to a larger degree than I did. After this, he didn't publish much. He has only published one paper since Yolov3, a paper outside of CV that he was third author on. I did a small amount of cyber-sleuthing2 to find out that he probably went to work on a startup that sprung out of his school, then submitted his thesis in 2021, then joined the circus3. I don't know this person. But I have known enough grad students to make guesses about what happened. What interests me is how what I assume to be a self reinforcing mixture of disillusionment, low productivity, and poor mood led to this paper in two ways.
The first was clarity about the alignment between his research and his personal values. When in the publishing pressure chamber, its so easy to lose sight of the reasons for and against doing your research, both personal and ethical. The second was humility to write not the paper that was best for his future or the future of his research, but to say his honest thoughts. What didn't work is often far more useful than what did, but is rarely published due to the review process. He could have tried to squeak a paper out of this, and I think it would be worse for it.
Clarity and humility should be important for researchers. The problem is that the cocktail of emotions that led to his and my clarity makes doing any kind of research really hard. Research often requires irrational belief in yourself and what you do to stay motivated in the face of the crazy odds every step of the way. Thinking about all the potential problems with your research once you do succeed can stop you from ever getting there. Furthermore, being honest with the faults of your research can make it ignored in favor of an unending stream of fool's gold.
How do you do navigate the minefield? Emotionally balance your confidence in moving forward yourself with deep uncertainty over what impact you may have? Balance the interests of your sponsors and your career with the interests of the world et. large? These are questions I think I will always wrestle with.
But at the moment, I don't think I will walk away and join the circus. To be clear, there is nothing at all wrong with that. I'm just not the type of person who can watch from the sidelines as this technology breaks the world and reshapes into something else, not when I believe that the impact of technology is not inevitable but will be shaped by those who create it. Some instinctual, illogical part of me feels like I can have an impact. As long as that is true, I am a moth before a flame.
But also, to be real4 with you, dear reader, I've seen what happens when you leave. I've seen it in the aftermath of the very paper I just discussed.
Preventing the raising of zombies
Two years after Yolov3 was published, a group from taiwan published Yolov4. They did nothing illegal: there is no copyright on method names, and their paper highlighted real improvements to the architecture as far as I can tell. It was a smart move: the paper got eight thousand citations. While no one can know for certain how many of their citations are due to the quality of their research and how many came from the YOLO brand, similar papers from more well known labs that went to more prestigious conferences got far fewer citations. Unfortunately for the authors, others realized they could profit from the abandoned brand of Yolo. A startup called ultralytics quickly made Yolov5, and charged for commercial usage. A third team from Chinese company Meituan published Yolov6, followed by the Yolov4 team publishing Yolov7, followed by the Yolov5 team publishing Yolov8. The brand has been picked apart like carrion, but the vultures still fight over bones. This is an unsurprising outcome. The brand is valuable. Of course it is picked apart.
It did make me sad, though. The brand and recognition came from his work. Because he left, all that work is being used by company trying to get a enterprise licensing fee. To me, it is a reminder that the people most likely to leave the field due to ethical qualms will not make the field better at the margin by doing so. To others, this is probably obvious. To me it was not.5
You have power over your research, but it is fleeting. Leverage it while you can. Be strict about creating research that advances the field as well as your career. Don't stop yourself from moving forward due to worst case scenarios. Find like minded people and try to push the field in the direction you want it to go into. Talk openly and encourage conversation around how to improve the field and its impact on the world.
Perhaps this is naïve. But as corny as it sounds it is the way I believe I can change the world while doing what I love.
I had enough thoughts on peer review that this footnote had footnotes. This is not allowed on substack, so I will probably make that set of ideas into its own article and publish it. Content!
a.k.a. I went through his google scholar and his twitter.
I'm not kidding.
its time to BeReal d-d-d-duh, duhduhduhduh DUEL
Marginal thinking is why I only consume dairy and eggs socially. Also, I think that not working at certain companies because you don't think they are good for society should be more common. Given I have these odd beliefs, can you see why I might entertain the idea that the best thing for the world is to avoid ML research if you know it will probably be used for unethical purposes?