Differences

This shows you the differences between two versions of the page.

--- nlp:history_of_nlp [2022/07/19 22:47] – [Early Work (prior to 2000)] jmflanig
+++ nlp:history_of_nlp [2024/02/20 09:05] (current) – jmflanig
@@ Line 11: / Line 11: @@
 ===== Early Work (prior to 2000) =====
+=== Models of Language ===
+  * [[https://ieeexplore.ieee.org/document/1056813|Chomsky 1956 - Three Models for the Description of Language]] [[http://web.mit.edu/6.441/spring06/projects/1/aak@mit.edu.1.pdf|overview]] Introduced the [[https://en.wikipedia.org/wiki/Chomsky_hierarchy|Chomsky hierarchy]]
+  * [[https://www.sciencedirect.com/science/article/pii/S0019995858900822|Chomsky & Miller 1958 - Finite State Languages]]. From [[https://www.sciencedirect.com/journal/information-and-control/vol/1/issue/2|here]].
+  * [[https://pdf.sciencedirectassets.com/273276/1-s2.0-S0019995800X0157X/1-s2.0-S0019995859903626/main.pdf?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEID%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJHMEUCIQD09vkBwVhm5rnhMeGy0RVxzbNlTmJIyhGvtUsWQN8qiwIgOFZhbef1NmFzZz1TR9mlys22qFGannbGcL18GQB6gWwq2wQI%2Bf%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAFGgwwNTkwMDM1NDY4NjUiDAl5PazPWbisJcjwiSqvBGZHVVp54W5q8XVrXCQVIsLWCJVxkUT7NqxPE6%2BGSSZHiVq4UyKdznaia9qhCu5O%2B72jtH0cSbJdFepgRWCzIox9KDItVIRcwh4hsFsJL7rjZIUCIJuegBa2SDnJ%2BghJSbL8xzJ9OOL8j4%2FAynfILN9fxWD9%2BR52NVxxopwUMqM4bfYlqyD9fRoxVfCfIyX5ZUKOGALs%2BoHKpQAre8wACBuhYk%2F1gmHdgJhDbkMinJ0S1cZiL%2FWP6s%2BesOda3dcO9cTRu67yjRJpxprmM6HFKa2ZWkJ7DdQF83Mj0%2Femw5%2BJM4kiBHqSG3KaWg2sn5cUxwprJ3FtqQncF3ZrBf0rch3v1Q%2FGDR%2FsT532lFmzaNuZvTSZhy7im5I0X8JfVkZ4TWCWxQeV8ZkWoJNWpF%2F%2Fzu8%2FF60oUnM7Jn3nVXX1rI8V849Afai8z9cA7p02oO8OmoZH6wqcsB53X1wUR25k4rIfAmuamE%2Bt8Hv7g5IQtfLTLU%2BbWkTEu2eMCqCR1EOJHbRlVW%2BEZftq%2FKFNkSLAL%2F1E5iQ28BmgYXgRrjkAZEkzBd4sYVZ8nR2wtA6VqBixzPaQZmyWlg%2Bx1jLkqDi4gSJ7tJGhLnw9JVTxNgOu%2FghGJhzm9PqIVQF0zAAULYAD4tM8ikB6s5x%2BzNUDyJSCAyGFvb%2BpoKkBYAtVcbItu%2BUfdIilWfGJ6d8ZCcIWXh9DAmtXj8yNmw5mQano%2BRAZfZYm72Q%2FIJDi0nUcJkFFj%2Fcwm8%2BqmAY6qQExF1OQyq0lxVSNLCZaOkggiR%2Fmc30pTvPhNXTc4AifB8fAZPI4dhEoXUquP6NUCfkBON00QdS4CHOmgNDS8q023JC2V%2BZszhPoCdZA1NYG%2B%2BAlA2qtrLPzsqEwxEyRBL7QG1yxRosGF%2BYwc321hrMYEUz3TXiC7ZFc%2FZwpM02sHifk5eyYgfvjb3DBqyhi9lMRCwtS0gGteUQBcO9wVFNP4JQf4zu4lioj&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20220828T003003Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=ASIAQ3PHCVTYSEFI44MJ%2F20220828%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=3f7a8a84c78623558304902421089f522b047e87054fa5c78828dd9a5dbaa698&hash=e7d5f8909080b4caec879073ccc6d2c6840b4411c3b59b5a8d736bd16af9724b&host=68042c943591013ac2b2430a89b270f6af2c76d8dfd086a07176afe7c76c2c61&pii=S0019995859903626&tid=spdf-759690fa-e320-4e4d-9b21-928d9892cc6d&sid=072472816e3c7840759a50441ebf483750d5gxrqa&type=client&ua=4d525f5c05080e56590707&rr=7418efa59dd89e5f|Chomsky 1959 - On Certain Formal Properties of Grammars]]
+  * [[http://www-igm.univ-mlv.fr/~berstel/Mps/Travaux/A/1963-7ChomskyAlgebraic.pdf|Chomsky & Schützenberger
+- The Algebraic Theory of Context Free Languages]]
 === Machine Translation ===
   * Interlingua-based
     * [[https://aclanthology.org/1988.tmi-1.4.pdf|Nirenburg et al 1988 - Lexical Realization in Natural Language Generation]] Describes the generation system of DIOGENES.
 === Question Answering ===
   * [[https://aclanthology.org/T75-1005.pdf|Lehnert 1975 - What Makes Sam Run? Script Based Techniques for Question Answering]] Very cool work.
+  * [[https://aclanthology.org/W00-0603.pdf|Riloff & Thelen 2000 - A Rule-based Question Answering System for Reading Comprehension Tests]]
 === Dialog ===
   * ELIZA: [[https://cse.buffalo.edu/~rapaport/572/S02/weizenbaum.eliza.1966.pdf|Weizenbaum 1966 - ELIZA - A Computer Program For the Study of Natural Language Communication Between Man And Machine]] [[https://en.wikipedia.org/wiki/ELIZA|wikipedia]] [[https://sites.google.com/view/elizagen-org/the-original-eliza|original source code]] [[https://github.com/jeffshrager/elizagen.org/blob/master/1965_Weizenbaum_MAD-SLIP/ELIZA_transcription_annotated_20220216.txt|commented source]] [[https://github.com/wadetb/eliza|Python reimplementation]] [[https://sites.google.com/view/elizagen-org/commonly-known-eliza-clones?authuser=0|other versions]]
-  * PERRY: [[https://en.wikipedia.org/wiki/PARRY|wikipedia]] [[http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/areas/classics/parry/|source code]]
+  * PERRY: [[https://en.wikipedia.org/wiki/PARRY|wikipedia]] [[http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/areas/classics/parry/|source code]] PERRY was the first program to pass the Turing test.
   * [[https://nlp.stanford.edu/acvogel/gus.pdf|Bobrow 1977 - GUS, A Frame-Driven Dialog System]]
   * [[https://aclanthology.org/P94-1009.pdf|Green & Carberry 1994 - A Hybrid Reasoning Model For Indirect Answers]]
@@ Line 28: / Line 37: @@
 === Speech Recognition ===
 Included here since some of the algorithms are shared with statistical NLP methods
+  * [[https://asa.scitation.org/doi/pdf/10.1121/1.1907936|Denes 1960 - Spoken Digit Recognition Using Time‐Frequency Pattern Matching
+]] Word-based matching, cited by [[https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1171882|Bridle 1982]]
+  * [[https://link.springer.com/content/pdf/10.1007/BF01074755.pdf|Vintsyuk 1968 - Speech Discrimination by Dynamic Programming]] [[https://drive.google.com/uc?export=view&id=1RWVDPJIqSdW-9S3u4DAUYHkoiNK0IKuP|pdf (UCSC only)]] [[https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1171882|Bridle 1982]] says this was pioneering work which was not well known in the West.
+  * [[https://www.sciencedirect.com/science/article/pii/S0020737370800086|Velichko & Zagoruyko 1970 - Automatic Recognition of 200 Words]] [[https://drive.google.com/uc?export=view&id=1OgInzWSk1nmb7ESTDPYKKYugWjHtu_ee|pdf (UCSC only)]] Cited by [[https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1171882|Bridle 1982]]
   * [[https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1454428|Jelinek 1976 - Continuous Speech Recognition by Statistical Methods]] [[https://drive.google.com/uc?export=view&id=1szYEgJmf39fs1IrMBdMDNMG6Lf028JbO|pdf (UCSC only)]]
-  * [[https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1171882|Bridle et al 1982 - An Algorithm for Connected Word Recognition]] [[https://drive.google.com/uc?export=view&id=1_6CtQWkLFSAKMitHUALnw_jgEA_KocYx|pdf (UCSC only)]]
+  * [[https://apps.dtic.mil/sti/citations/ADA035146|Lowerre 1976 - The Harpy Speech Recognition System]] (Ph.D. Thesis) {{papers:1976_-_harpy.pdf|pdf}} [[https://stacks.stanford.edu/file/druid:rq916rn6924/rq916rn6924.pdf|Summary]], missing one page. Cited Bridle 1982 (and Ney 1992) for the term "beam search"
+  * [[https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1171882|Bridle et al 1982 - An Algorithm for Connected Word Recognition]] [[https://drive.google.com/uc?export=view&id=1_6CtQWkLFSAKMitHUALnw_jgEA_KocYx|pdf (UCSC only)]] Cited by Ney 1992 for beam search
+  * [[https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=124938|Ney et al. 1992 - Data Driven Search Organization for Continuous Speech Recognition in the SPICOS System]] [[https://drive.google.com/uc?export=view&id=1wsoLDFcAIuVNVCPBVGXMaa0err2zgXKm|pdf (UCSC only)]] See p. 4 bottom for a history of beam search, which it says is called "beam search, DP beam search, or pruned DP search."
 === Generation ===
@@ Line 92: / Line 108: @@
 ===== Related Pages =====
   * [[ml:History of ML]]
+  * [[Key Papers in NLP]]