2024

  1. FrameKit: A Tool for Authoring Adaptive UIs Using Keyframes Wu, Jason, Todi, Kashyap, Chan, Joannes, Myers, Brad A, and Lafreniere, Ben In Proceedings of IUI 2024, ACM Conference on Intelligent User Interfaces 2024 [PDF]
  2. UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback Wu, Jason, Schoop, Eldon, Leung, Alan, Barik, Titus, Bigham, Jeffrey P, and Nichols, Jeffrey In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) 2024 [PDF]
  3. Computational Understanding of User Interfaces Wu, Jason 2024 [PDF]
  4. DreamStruct: Understanding Slides and User Interfaces via Synthetic Data Generation Peng, Yi-Hao, Faria Huq, Yue Jiang, Wu, Jason, Li, Xin Yue, Bigham, Jeffrey P, and Pavel, Amy In Proceedings of ECCV 2024, The 18th European Conference on Computer Vision 2024 [PDF]
  5. UIClip: A Data-driven Model for Assessing User Interface Design Wu, Jason, Peng, Yi-Hao, Li, Amanda, Swearngin, Amanda, Bigham, Jeffrey P, and Nichols, Jeffrey In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology 2024 [PDF]

2023

  1. Screen Correspondence: Mapping Interchangeable Elements between UIs Wu, Jason, Swearngin, Amanda, Zhang, Xiaoyi, Nichols, Jeffrey, and Bigham, Jeffrey P arXiv preprint arXiv:2301.08372 2023 [PDF]
  2. WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics Wu, Jason, Wang, Siyan, Shen, Siman, Peng, Yi-Hao, Nichols, Jeffrey, and Bigham, Jeffrey P In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems 2023 [PDF]
  3. Towards Never-ending Learning of User Interfaces Wu, Jason, Krosnick, Rebecca, Schoop, Eldon, Swearngin, Amanda, Bigham, Jeffrey P, and Nichols, Jeffrey In ICML Workshop on Artificial Intelligence & Human Computer Interaction 2023 [PDF]
  4. Never-ending Learning of User Interfaces Wu, Jason, Krosnick, Rebecca, Schoop, Eldon, Swearngin, Amanda, Bigham, Jeffrey P, and Nichols, Jeffrey In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology 2023 [PDF]
  5. Towards Automated Accessibility Report Generation for Mobile Apps Swearngin, Amanda, Wu, Jason, Zhang, Xiaoyi, Gomez, Esteban, Coughenour, Jen, Stukenborg, Rachel, Garg, Bhavya, Hughes, Greg, Hilliard, Adriana, Bigham, Jeffrey P, and others, arXiv preprint arXiv:2310.00091 2023 [PDF]
  6. STAR: Smartphone-analogous Typing in Augmented Reality Kim, Taejun, Karlson, Amy, Gupta, Aakar, Grossman, Tovi, Wu, Jason, Abtahi, Parastoo, Collins, Christopher, Glueck, Michael, and Surale, Hemant Bhaskar In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology 2023 [PDF]
  7. Using LLMs to Customize the UI of Webpages Li, Amanda, Wu, Jason, and Bigham, Jeffrey P In Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology 2023 [PDF]

2022

  1. Understanding Screen Relationships from Screenshots of Smartphone Applications Feiz, Shirin, Wu, Jason, Zhang, Xiaoyi, Swearngin, Amanda, Barik, Titus, and Nichols, Jeffrey In 27th International Conference on Intelligent User Interfaces 2022 [PDF]
  2. Towards Complete Icon Labeling in Mobile Applications Chen, Jieshan, Swearngin, Amanda, Wu, Jason, Barik, Titus, Nichols, Jeffrey, and Zhang, Xiaoyi In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems 2022 [PDF]
  3. Extracting replayable interactions from videos of mobile app usage Chen, Jieshan, Swearngin, Amanda, Wu, Jason, Barik, Titus, Nichols, Jeffrey, and Zhang, Xiaoyi arXiv preprint arXiv:2207.04165 2022 [PDF]
  4. Reflow: Automatically Improving Touch Interactions in Mobile Applications through Pixel-based Refinements Wu, Jason, Barik, Titus, Zhang, Xiaoyi, Lea, Colin, Nichols, Jeffrey, and Bigham, Jeffrey P arXiv preprint arXiv:2207.07712 2022 [PDF]
  5. Diffscriber: Describing Visual Design Changes to Support Mixed-ability Collaborative Presentation Authoring Peng, Yi-Hao, Wu, Jason, Bigham, Jeffrey, and Pavel, Amy In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology 2022 [PDF]

2021

  1. Screen recognition: Creating accessibility metadata for mobile applications from pixels Zhang, Xiaoyi, Greef, Lilian, Swearngin, Amanda, White, Samuel, Murray, Kyle, Yu, Lisa, Shan, Qi, Nichols, Jeffrey, Wu, Jason, Fleizach, Chris, and others, In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems 2021 [PDF]
  2. When can accessibility help? an exploration of accessibility feature recommendation on mobile devices Wu, Jason, Reyes, Gabriel, White, Sam C, Zhang, Xiaoyi, and Bigham, Jeffrey P In Proceedings of the 18th International Web for All Conference 2021 [PDF]
  3. MultiBench: Multiscale Benchmarks for Multimodal Representation Learning Liang, Paul Pu, Lyu, Yiwei, Fan, Xiang, Wu, Zetian, Cheng, Yun, Wu, Jason, Chen, Leslie Yufan, Wu, Peter, Lee, Michelle A, Zhu, Yuke, and others, arXiv preprint arXiv:2107.07502 2021 [PDF]
  4. Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots Wu, Jason, Zhang, Xiaoyi, Nichols, Jeff, and Bigham, Jeffrey P In The 34th Annual ACM Symposium on User Interface Software and Technology 2021 [PDF]
  5. Pixel-based optimization for a user interface Bigham, Jeffrey P, Lea, Colin, Wu, Jason, and Zhang, Xiaoyi 2021 [PDF]

2020

  1. Automatic Class Discovery and One-Shot Interactions for Acoustic Activity Recognition Wu, Jason, Harrison, Chris, Bigham, Jeffrey P, and Laput, Gierad In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems 2020 [PDF]
  2. Making Mobile Augmented Reality Applications Accessible Herskovitz, Jaylin, Wu, Jason, White, Samuel, Pavel, Amy, Reyes, Gabriel, Guo, Anhong, and Bigham, Jeffrey P. In Proceedings of the 22nd international ACM SIGACCESS conference on Computers and accessibility 2020 [PDF]
  3. Towards Recommending Accessibility Features on Mobile Devices Wu, Jason, Reyes, Gabriel, White, Sam C., Zhang, Xiaoyi, and Bigham, Jeffrey P. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility 2020 [PDF]
  4. Disability and the COVID-19 pandemic: Using Twitter to understand accessibility during rapid societal transition Gleason, Cole, Valencia, Stephanie, Kirabo, Lynn, Wu, Jason, Guo, Anhong, Jeanne Carter, Elizabeth, Bigham, Jeffrey, Bennett, Cynthia, and Pavel, Amy In Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility 2020 [PDF]

2019

  1. TongueBoard: An Oral Interface for Subtle Input Li, Richard, Wu, Jason, and Starner, Thad In Proceedings of the 10th Augmented Human International Conference 2019 2019 [PDF]
  2. ScratchThat: supporting command-agnostic speech repair in voice-driven assistants Wu, Jason, Ahuja, Karan, Li, Richard, Chen, Victor, and Bigham, Jeffrey Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2019 [PDF]
  3. SelfSync: exploring self-synchronous body-based hotword gestures for initiating interaction Lee, Juyoung, Aggarwal, Shaurye, Wu, Jason, Starner, Thad, and Woo, Woontack In Proceedings of the 2019 ACM International Symposium on Wearable Computers 2019 [PDF]

2018

  1. Synchrowatch: One-handed synchronous smartwatch gestures using correlation and magnetic sensing Reyes, Gabriel, Wu, Jason, Juneja, Nikita, Goldshtein, Maxim, Edwards, W Keith, Abowd, Gregory D, and Starner, Thad Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2018 [PDF]
  2. Synchronous Interfaces for Wearable Computers Wu, Jason 2018 [PDF]
  3. Seesaw: rapid one-handed synchronous gesture interface for smartwatches Wu, Jason, Colglazier, Cooper, Ravishankar, Adhithya, Duan, Yuyan, Wang, Yuanbo, Ploetz, Thomas, and Starner, Thad In Proceedings of the 2018 ACM International Symposium on Wearable Computers 2018 [PDF]
  4. Nadia: Neural network driven virtual human conversation agents Wu, Jason, Ghosh, Sayan, Chollet, Mathieu, Ly, Steven, Mozgai, Sharon, and Scherer, Stefan In Proceedings of the 18th International Conference on Intelligent Virtual Agents 2018 [PDF]
  5. NADiA-Towards Neural Network Driven Virtual Human Conversation Agents Scherer, Stefan, Wu, Jason, Ghosh, Sayan, Chollet, Mathieu, Ly, Steven, and Mozgai, Sharon In International Conference on Autonomous Agents and Multiagent Systems 2018 [PDF]

2016

  1. Whoosh: non-voice acoustics for low-cost, hands-free, and rapid input on smartwatches Reyes, Gabriel, Zhang, Dingtian, Ghosh, Sarthak, Shah, Pratik, Wu, Jason, Parnami, Aman, Bercik, Bailey, Starner, Thad, Abowd, Gregory D, and Edwards, W Keith In Proceedings of the 2016 ACM International Symposium on Wearable Computers 2016 [PDF]