How and From to Read Machine Learning Research Papers
Getting started with reading Deep Learning research papers: the Why and the How
With hundreds of papers beingness published every calendar month, anybody who is serious about learning in this field cannot rely just on tutorial-style manufactures or courses where someone else breaks down the latest research for him/her.
How practise you proceed the learning after you have consumed that book or completed that amazing online course on Deep Learning? How do you lot become "self-sufficient" and then that you don't have to rely on someone else to break down the latest quantum in the field?
— Y'all read inquiry papers.
A short note earlier you lot offset — I am no proficient at Deep Learning. I have only recently started reading inquiry papers. In this article, I am going to write virtually everything that I found helpful when I started.
The WHY
In the answer to a question on Quora, request how to test if i is qualified to pursue a career in Automobile Learning, Andrew Ng (founder Google Brain, old head of Baidu AI group) said that anyone is qualified for a career in Machine Learning. He said that after you have completed some ML related courses, "to go even further, read research papers. Even better, try to replicate the results in the enquiry papers."
Dario Amodei (researcher at OpenAI) says that, "To test your fit for working in AI safety or ML, just trying implementing lots of models very quickly. Find an ML model from a contempo paper, implement information technology, try to get it to piece of work quickly."
This suggests that reading enquiry papers is crucial to further one's understanding of the field.
With hundreds of papers being published every calendar month, everyone who is serious nigh learning in this field cannot rely merely on tutorial-style manufactures or courses where someone else breaks down the latest research for him/her. New, footing-breaking inquiry is being done equally you read this article. The footstep of research in the field has never been higher. The only way yous can hope to keep upwardly with the footstep is past making a addiction to read research papers every bit they are released.
In this article, I will try to give yous some actionable communication on how you tin start reading a newspaper yourself. Then, in the end, I volition try to break down an bodily paper so you may get started.
The HOW
First things offset, reading a scientific research paper is difficult. In fact—
"Zilch makes you experience stupid quite like reading a scientific journal article."
I just wanted to put that first so you lot don't become discouraged if you lot feel similar you can't really empathise the contents of a paper. It is unlikely that yous sympathize it in the first few passes. So, simply be gritty and accept another shot at information technology!
Now, allow u.s.a. talk about a few valuable resources that will help you in your reading journeying..
arXiv.org
Think of it every bit this place on the cyberspace where researchers publish their papers before they are actually published in the those reputable scientific journals or conferences (if ever).
Why would they do that?
Well, it turns out that doing the research and really writing the paper is not the end of information technology (!). Getting a paper from being submitted to being published in some scientific journal is quite a long process. Subsequently a paper is submitted to one of these journals, there'due south a peer review process which can exist quite slow (sometimes even spanning multiple years!) Now, this is really undesirable for a fast moving field like Machine Learning.
That's why, arXiv.
Researchers publish their papers on a pre-print repositories similar arXiv to apace disseminate their research and get quick feedbacks on information technology.
Arxiv Sanity Preserver
Okay, and so assuasive researchers to easily pre-print their inquiry papers is skillful. But what about the people reading those papers? If you get to the arXiv website, it is easy to experience scared and pocket-size and lost. Definitely not a place for newcomers ( just my opinion, you are welcome to effort it though ☺ ).
Enter, Arxiv Sanity Preserver.
Arxiv Sanity does to arXiv, what Twitter's newsfeed does to Twitter (except that it is totally open-sourced and complimentary of advertizement, apparently). Just as the newsfeed lets you come across the most interesting tweets, personalised to your ain sense of taste, from among the large large ocean that is Twitter, similarly Arxiv Sanity brings to y'all the papers on ML, published on arXiv, that might be the nigh interesting for you. It lets you sort the papers based on what's trending, based on your past likes and the likes of the people that yous follow. (Just those personalised recommendations features that we have got so used to over the social media, you know.)
Machine Learning- WAYR thread on Reddit
WAYR is short for What Are Yous Reading. Its a thread on the subreddit Motorcar Learning where people post the ML papers that they have read in this current week and discuss what they found interesting in it.
As I said, the number of research papers being published in the field of Machine Learning every week on arXiv is extremely large. This means that information technology is well-nigh impossible for a person to read all of them, every week and do regular things like attention higher or going to a job or well, interacting with other human beings. Also, its not like all the papers are even worth reading.
Hence, you need to devote your free energy to reading only the most promising papers and the thread that I mentioned above is 1 way of doing so.
Newsletters, Newsletters, Newsletters!
Newsletters are my personal best source of keeping up with the latest advances in the field of AI. Y'all can simply subscribe to them and have them delivered to your inbox every Monday for free! And merely like that, you can become to know most the virtually interesting news, articles and enquiry papers of the calendar week related to AI.
Here are the ones that I accept subscribed to:
- Import AI by Jack Clark
This is my favourite considering in improver to giving information about everything that I mentioned above, it also features a section called "Tech Tales". This section contains a new AI- related short sci-fi story based on by calendar week'south events!
(Psst.. a confession: Even on those weeks when I don't feel so enthusiastic about new things in AI, I will skim though this newsletter just because of the Tech Tales) - Auto Learnings by Sam DeBrule
He besides maintains a medium publication by the aforementioned name. It contains some really interesting articles. Be sure to check them out too. - Nathan.ai by Nathan Benaich
While the higher up 2 newsletters are weekly, this is a quarterly newsletter. And then, y'all get 1 long email every 3 months which summarises the nigh interesting developments in the field for the by 3 months. - The Wild Week in AI by Denny Britz
I actually liked this ane because how its make clean, curtailed presentation but it seems like this has get inactive since the past two months. Anyway, I am mentioning it here simply in case Denny starts sending those emails once again.
"AI people" on Twitter
Another expert way by which you could proceed up with the best and the latest in the field is by following the famous researchers and developers accounts on Twitter. Here's a list of people that I follow:
- Michael Nielsen
- Andrej Karpathy
- Francois Chollet
- Yann LeCun
- Chris Olah
- Jack Clark
- Ian Goodfellow
- Jeff Dean
- OpenAI (I know this is non "people" but yeah..)
"That'south all good, but how do I get-go??"
Yes, that is the more than pressing concern.
Okay so, first of all make sure that you sympathise the basics of Machine Learning like regression and other such algorithms, the basics of Deep Learning — plain vanilla neural networks, backpropagation, regularisation and a little more than the basics like how ConvNets, RNN and LSTM work. I really don't think that reading research papers is the best style to clear your basics on these topics. There are enough of other resources that you can refer to for doing then.
Once you take done that, you should showtime by reading a paper that originally introduced 1 of those above ideas. This way, you will exist able to focus on just getting used to how a research paper looks. You won't have to worry besides much nearly actually understanding your first enquiry papers since yous are already quite familiar with the thought.
I recommend that you start with the AlexNet newspaper .
Why this newspaper?
Look at this graph:
See how the Computer Vision and Patter Recognition curve merely shoots up in the year 2012? Well, that's largely because of this paper.
!!!
This is the paper that rekindled all the interest in Deep Learning.
Authored by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, and titled ImageNet Nomenclature with Deep Convolutional Networks, this paper is regarded every bit 1 of the most influential papers in the field. It describes how the authors used a CNN (named AlexNet) to win the ImageNet Large-Scale Visual Recognition Claiming (ILSVRC) 2012.
For those of y'all who don't know, enabling computers to see and place objects (aka Computer Vision) is one of the primeval goals of Information science. ILSVRC is like the Olympics for such "seeing computers" in which the participants (estimator algorithms) endeavour to correctly place images equally belonging to 1 of the k categories. And, in 2012 AlexNet was able to win this challenge by a huge HUGE margin:
It accomplished a tiptop 5 mistake charge per unit of 15.iii% compared to the 26.2% that the 2d best entry recieved!
Needless to say, the entire Computer Vision community was awestruck and research in the area accelerated like never before. People started realising the power of Deep Neural Networks and well, here you are trying to empathize how you tin can go a piece of the pie!
That being said, if you have a basic understanding of CNNs through some form or tutorial, it will exist quite like shooting fish in a barrel to grasp the contents of this paper. And so, more ability to you!
One time you are done with this newspaper, yous may check out other such seminal papers relating to CNN or perhaps movement to another architecture that interests y'all (RNNs, LSTMs, GANs).
There are also lots of repositories that have a good collection of important research papers in Deep Learning on Github (here's a absurd one). Exist sure to check them out when you are starting. They will help you lot in creating your own reading list.
It would be highly remiss of me to not mention this ane other source
- Distill.pub :
I take merely one affair to say virtually it —
If all research papers were published in the Distill periodical, then probably, I wouldn't be writing this article, yous wouldn't have to read an article to guide yous to reading a enquiry paper and the net would need much fewer (if at all) courses and tutorials that try to explicate those seminal research ideas in comprehensible terms.
Distill: An Interactive, Visual Journal for Machine Learning Enquiry
The periodical Distill launches today. In a nutshell, Distill is an interactive, visual journal for motorcar learning enquiry. Going forward, Distill will publish both original research articles and expository articles. If y'all're interested in submitting, delight consult the folio about publishing in Dis
So, be certain to check out the manufactures in there. It is really the side by side generation stuff!
Thanks for reading all the style through! I hope this article helps you in the job of proceed up with the latest ML inquiry. And remember, reading a scientific newspaper is difficult. So, at that place is no need to get discouraged. Just power through some other reading if you don't empathize it.
Let me know your thought in the comments section beneath. You lot can besides reach out to me on Twitter or LinkedIn.
I am a freelance writer. You tin can hire me to write similar indepth, passionate articles explaining an ML/DL engineering for your company'south blog. Shoot me an email at nityeshagarwal[at]gmail[dot]com to discuss our collaboration.
Also, you can follow me on Twitter; I won't spam your feed ;-)
Source: https://www.nityesh.com/getting-started-reading-research-papers-dl/
0 Response to "How and From to Read Machine Learning Research Papers"
Post a Comment