Future Blog Assignments

As of Reading 27, recognition of chemical hand-drawn diagrams, we are finished with class-wide assigned readings.  We have covered much of the core sketch recognition material by this point, although there are still a lot of sketch recognition applications and algorithms which we haven't explicitly discussed.  Some will still be covered in lecture to an extent, but at this point, it is time to start focusing more on your projects.

For that reason, the remaining readings will all be selected by you.  You will find 8 research papers, primarily related to sketch recognition, that apply to your project in some way.  Each week, you should post your summary and key concepts from three of them to your blog.  This corresponds to roughly one per class, maintaining a similar rate of reading as before.  These papers are related to your project though.  So you should select them based on if they seem useful to your work.  Posts will not be evaluated based on how well the paper is a match for your project, since that's up to you, but we will expect to see some of these papers appear in the bibliography of your final paper, so make sure that you can get something out of them.  Every person in the class needs 8 papers, so if your team has 3 people, you'll have 24 papers to select for referencing just by doing this assignment.

The schedule of readings for the past few weeks and remainder of the class is as follows:

10/25 - Reading 19
11/01 - Readings 20, 21, and 22
11/08 - Readings 23 and 24
11/15 - Readings 25, 26, and 27
11/22 - Project readings 1, 2, and 3
11/29 - Project readings 4, 5, and 6
12/13 - Project readings 7 and 8

Technically, 7 would be due the 6th and 8 the 13th, but due to the complications added with Thanksgiving, nothing will be due the 6th.  Both 7 and 8 will be checked on the 13th of December, which is why I say readings will be roughly one each class; it's actually slightly less than that.  Hopefully, by the 13th, you'll have the project more or less complete, but you could still get a couple useful references to add to your paper by then.  And if you want to, you can actually read all 8 and do them now!  There's no problem doing them early, so if you want to get all the blogging out of the way before Thanksgiving, I recommend it!  Doing it early is not required though.

Reading 27: Visual Symbolic Recognizer

Paper
Tom Y. Ouyang and Randall Davis. 2009. A visual approach to sketched symbol recognition. In Proceedings of the 21st international jont conference on Artifical intelligence (IJCAI'09), Hiroaki Kitano (Ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1463-1468.
Direct Link: http://dspace.mit.edu/openaccess-disseminate/1721.1/71572

Notes
As the last class-wide assigned reading, we'll look at another interesting recognition algorithm which has been applied in multiple domains.  One relatively unique domain which we have not discussed that these authors applied their work to is chemical diagram recognition, so you could check out that paper if you're interested.  We've read about many underlying algorithms and many domain applications by this point, and while there's still a few more small topics relating to gesture/activity and a couple of recognition techniques which will be covered in class lectures, they will not be covered to the extent that more papers will be assigned.  Future blog posts will be team-specific, the goal being to start helping you build a collection of reference works for your final project.  More details will follow on that shortly.

Reading 26: Shape Context

Paper
Oltmans, Michael. Envisioning sketch recognition: a local feature based approach to recognizing informal sketches. Diss. Massachusetts Institute of Technology, 2007.
Direct Link: http://rationale.csail.mit.edu/publications/Oltmans2007Envisioning.pdf

Notes
This paper is an MIT thesis about recognizing shape context.  It is very long, but you do not need to review the whole paper.  Focus your efforts on Chapters 2 and 3 (2 being perhaps the most relevant).  There's some interesting features and approaches inspired by computer vision used in this paper, and hopefully, it helps gives you an impression of how vision and sketch recognition differ and agree.

Final Project Details

I meant to get this out earlier to everyone, but it should be a useful guide as you choose your projects and work on them.

Final Project Details

Reading 25: Who Dotted That i?

Paper
Eoff, Brian David, and Tracy Hammond. "Who dotted that'i'?: context free user differentiation through pressure and tilt pen data." Proceedings of Graphics Interface 2009. Canadian Information Processing Society, 2009.
Publication Link: http://dl.acm.org/citation.cfm?id=1555916

Notes
This paper shows some of the additional information carried in your sketching that could be applied to forensics applications.

Reading 24: SketchREAD

Paper
Alvarado, Christine, and Randall Davis. "SketchREAD: a multi-domain sketch recognition engine." Proceedings of the 17th annual ACM symposium on User interface software and technology. ACM, 2004.
Publication Link: http://dl.acm.org/citation.cfm?id=1029637

Reading 23: HMM Overview

Paper
Rabiner, Lawrence R. "A tutorial on hidden Markov models and selected applications in speech recognition." Proceedings of the IEEE 77.2 (1989): 257-286.
Direct Link: http://www.robots.ox.ac.uk:5000/~vgg/rg/papers/hmm.pdf

Notes
This paper is very long, but it also goes into a lot of detail.  For the class, if you can just look over the primary overview in the first few pages, you'll be fine.

Reading 22: HMM Recognizer

Paper
Sezgin, Tevfik Metin, and Randall Davis. "HMM-based efficient sketch recognition." Proceedings of the 10th international conference on Intelligent user interfaces. ACM, 2005.
Publication Link: http://dl.acm.org/citation.cfm?id=1040899

Reading 21: LADDER

Paper

Hammond, Tracy, and Randall Davis. "LADDER, a sketching language for user interface developers." Computers & Graphics 29.4 (2005): 518-532.

Publication Link: http://www.sciencedirect.com/science/article/pii/S0097849305000865

Reading 20: Multi-Stroke Primitives

Paper
Hammond, Tracy, and Brandon Paulson. "Recognizing sketched multistroke primitives." ACM Transactions on Interactive Intelligent Systems (TiiS) 1.1 (2011): 4.
Publication Link: http://dl.acm.org/citation.cfm?id=2030369

Reading 19: Sketch Sounds

Paper
Li, Wenzhe, and Tracy Anne Hammond. "Recognizing text through sound alone." Twenty-Fifth AAAI Conference on Artificial Intelligence. 2011.
Direct Link: http://www.aaai.org/ocs/index.php/AAAI/AAAI11/paper/download/3791/4119

Notes
This paper takes an interesting perspective on sketch recognition.  We examine how sound may be used to identify different stroke types, letter in particular in this paper.  Hopefully, this gives you an idea of how multiple features from multiple modalities may be used in recognition, as indeed they may be used by humans in certain contexts.  In the future, we'll see how even more features, like the angle of the pen, may be used as a feature, and this paper fits in well with that conversation.

Reading 18: Geometry Review

Paper
Geometry Review - Appendices.  This document is not available online.  You will need to meet with me to obtain a copy.

Notes
This is an overview of some of the background material used in this course.  It should be a helpful review for the midterm.

Reading 17: Paleosketch

Paper
Paulson, Brandon, and Tracy Hammond. "PaleoSketch: accurate primitive sketch recognition and beautification." Proceedings of the 13th international conference on Intelligent user interfaces. ACM, 2008.
Publication Link:http://dl.acm.org/citation.cfm?id=1378775

Notes
Paleosketch has been referenced as an underlying recognizer in several other papers we've discussed.

Reading 16: Combining Corner Segmenters

Paper
Wolin, Aaron, Martin Field, and Tracy Hammond. "Combining corners from multiple segmenters." Proceedings of the Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling. ACM, 2011.
Publication Link: http://dl.acm.org/citation.cfm?id=2021185

Notes
This paper addresses how corner detection can be improved significantly by considering the results of multiple algorithms.

Reading 15: IStraw

Paper
Xiong, Yiyan, and Joseph J. LaViola Jr. "Revisiting ShortStraw: improving corner finding in sketch-based interfaces." Proceedings of the 6th Eurographics Symposium on Sketch-Based Interfaces and Modeling. ACM, 2009.
Publication Link: http://dl.acm.org/citation.cfm?id=1572759

Notes
This work is closely related to ShortStraw, but includes modifications intended to improve the accuracy of the algorithm.

Reading 14: ShortStraw

Paper
Wolin, Aaron, Brian Eoff, and Tracy Hammond. "ShortStraw: A Simple and Effective Corner Finder for Polylines." SBM. 2008.
Direct Link: http://www.researchgate.net/profile/Tracy_Hammond/publication/220772398_ShortStraw_A_Simple_and_Effective_Corner_Finder_for_Polylines/links/0deec529f76e58d523000000.pdf

Notes
This is a fun and interesting corner detection algorithm that's also very simple.

Reading 13: Domain-Independent Recognition

Paper
Yu, Bo, and Shijie Cai. "A domain-independent system for sketch recognition." Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia and South East Asia. ACM, 2003.
Publication Link: http://dl.acm.org/citation.cfm?id=604499

Reading 12: Sketch-Based Interfaces

Paper
Sezgin, Tevfik Metin, Thomas Stahovich, and Randall Davis. "Sketch based interfaces: early processing for sketch understanding." ACM SIGGRAPH 2006 Courses. ACM, 2006.
Publication Link: http://dl.acm.org/citation.cfm?id=1185783

Notes
This paper is a good example of how several of these sketch features and algorithms can be combined into a user interface.

Reading 11 - No Rubine? Geometric-Based Features

Paper
Paulson, Brandon, et al. "What!?! no Rubine features?: using geometric-based features to produce normalized confidence values for sketch recognition." HCC Workshop: Sketch Tools for Diagramming. 2008.
Direct Link: https://www.cs.auckland.ac.nz/research/conferences/skekchws/proceedings/vlhcc_stws_p57.pdf

Notes
Continuing our discussion of different sketch features, this paper looks at some geometric-based features.

Reading 10 - Visual Similarity of Gestures

Paper
Long Jr, A. Chris, et al. "Visual similarity of pen gestures." Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 2000.
Publication Link: http://dl.acm.org/citation.cfm?id=332458

Notes
These authors have a lot of work in the area of gesture recognition using different form factors and inputs.  This paper discusses some of the background for their work, examining visual similarity of gestures and how this may be determined.