I am broadly interested in the design of interactive intelligent systems to extend human musical creation and expression. This research lies in the intersection of Machine Learning, HCI, Robotics, and Computer Music. My representative works include interactive composition via style transfer, human-computer interactive performances, autonomous dancing robots, large-scale content-based music retrieval, haptic guidance for flute tutoring, and bio-music computing using slime mold.
Particularly, I build music agents that compose and arrange music via style transfer and analogy, perform and improvise music expressively in concert with human musicians by learning from rehearsal experience, and tutor music beginners using haptic guidance. These efforts unify expressive performance rendering, automatic accompaniment, and algorithmic composition in a machine-learning framework, making music a more accessible and friendly tool for everyone.
I am an Assistant Professor in Computer Science at NYU Shanghai. I received my Ph.D. in the Machine Learning Department at Carnegie Mellon University where I study Machine Learning and Computer Music under the advice of Prof. Roger Dannenberg. I was a Neukom Fellow at Dartmouth from 2016 to 2017. In 2010, I received my undergraduate degree in Information Science with a minor in Psychology at Peking University. I am also a professional DI and XIAO (Chinese flute and vertical flute) player. I was the prime soloist of the Chinese Music Institute (CMI) of Peking University, where I also served as the president and assistant conductor. I held my solo concert in 2010 and a Music AI concert in 2022. (See Music Events at the bottom.)
Deep Music Generation:
| Authors | Title | Year | Book/Journal/Proceedings |
|---|---|---|---|
| Z. Wang, D. Xu, Y. Shan, G. Xia. | Auido-to-symbolic Arrangement via Cross-modal representation Learning [PDF] |
2022 | Proc. 47th International Conference on Acoustics, Speech and Signal Processing |
| Z. Wang and G. Xia. | MuseBERT: Pre-training Music Representation for Music Understanding and Controllable Generation [PDF] |
2021 | Proc. 22nd the International Conference on Music Information Retrieval (ISMIR) |
| Z. Wang, D. Wang, Y. Zhang, G. Xia. | Learning Interpretable Representation for Controllable Polyphonic Music Generation [PDF] |
2020 | Proc. 21st the International Conference on Music Information Retrieval (ISMIR) |
| J. Jiang and G. Xia. | Transformer VAE: A Hierarchical Model for Structure-aware and Interpretable Music Representation Learning [PDF] |
2020 | Proc. 45th International Conference on Acoustics, Speech and Signal Processing |
| Z. Wang, Y. Zhang, Y. Zhang, J. Jiang, R. Yang, J. Zhao, and G. Xia | PIANOTREE VAE: Structured Representation Learning for Polyphonic Music [PDF] |
2020 | Proc. 21st the International Conference on Music Information Retrieval (ISMIR) |
| R. Yang, D. Wang, Z. Wang, T. Chen, J. Jiang and G. Xia. | Deep Music Analogy Via Latent Representation Disentanglement [PDF] | 2019 | Proc. 20th the International Conference on Music Information Retrieval (ISMIR) |
| S. Dai and G. Xia. | Music Style Transfer: A Position Paper[PDF] | 2018 | 6th International Workshop on Musical Metacreation |
Computer-aided Music Learning via Multimodal feedbacks
| Authors | Title | Year | Book/Journal/Proceedings |
|---|---|---|---|
| D. Chin, Y. and G. Xia | A Computer-aided Multimodal Music Learning System with Curriculum: A Pilot Study
|
2021 | Proc. 22ND The International Conference on New Interfaces for Musical Expression (NIME) |
| D. Chin, Y. Zhang, and G. Xia | Hyper-hybrid Flute: Simulating and Augmenting How Breath Affects Octave and Microtone
|
2021 | Proc. 21st The International Conference on New Interfaces for Musical Expression (NIME) |
| Y. Li, Z. Piao, and G. Xia | A Wearable Haptic Interface for Breath Guidance in Vocal Training |
2021 | Proc. 21st The International Conference on New Interfaces for Musical Expression (NIME) |
| D. Chin, Y. Zhang, T. Zhang, J. Zhao, and G. Xia | Rainbow Score: A Visual-centered Multimodal Flute Tutoring System
|
2020 | Proc. 20th The International Conference on New Interfaces for Musical Expression (NIME) |
| Y. Zhang, Y. Li, D. Chin, and G. Xia | Adaptive Multimodal Music Learning via Interactive-haptic Instrument
|
2019 | Proc. 19th The International Conference on New Interfaces for Musical Expression (NIME) |
| G. Xia, C. Jacobsen, Q. Chen, X-D. Yang, and R. Dannenberg | ShIFT: A Semi-haptic Interface for Flute Tutoring
|
2018 | Proc. 18th The International Conference on New Interfaces for Musical Expression (NIME) |
Expressive and Interactive Performance
| Authors | Title | Year | Book/Journal/Proceedings |
|---|---|---|---|
| M. Xu, Z. Wang, and G. Xia | Modeling Perceptual Loudness of Piano Tone:Theory and Pratice [PDF coming soon] |
2022 | Proc. 23th the International Conference on Music Information Retrieval (ISMIR) |
| M. Xu, Z. Wang, and G. Xia | Transferring Piano Performance Control Across Environments [PDF] [Poster] |
2019 | Proc. 20th the International Conference on Music Information Retrieval (ISMIR) |
| G. Xia, and R. Dannenberg | Duet Interaction: Learning Improvisation Techniques for Automatic Accompaniment [PDF] |
2017 | Proc. 17th The International Conference on New Interfaces for Musical Expression (NIME) |
| S. Dai, and G. Xia, | Computational Models for Common Pipa Techniques [PDF coming soon] | 2017 | Proc. 5th National Conference on Sound and Music Technology (CSMT) Best student paper award |
| G. Xia, K. Mao, M. Fu, S. Cosentino, G.Trovato, S. Sessa, A. Takanishi, and R. Dannenberg | Expressive Humanoid Robot for Automatic Accompaniment. [PDF] [Slides] |
2016 | Proc. 13th the International Conference on Sound and Music Computing (SMC) |
| G. Xia, Y. Wang, R. Dannenberg, and G. Gordon | Spectral Learning for Expressive Interactive Ensemble Music Performance. [PDF] [Slides] [Audio Demo] |
2015 | Proc. 16th the International Conference on Music Information Retrieval (ISMIR) |
| G. Xia & R. Dannenberg | Duet Interaction: Learning Musicianship for Automatic Accompaniment. [PDF] [Slides] [Audio Demo] [Data] |
2015 | Proc. 15th the International Conference on New Interfaces for Musical Expression (NIME) |
| M. Fu, G. Xia, R. Dannenberg, and L. Wasserman | A Statistical View on the Expressive Timing of Piano Rolled Chords. [PDF] [Poster] |
2015 | Proc. 16th the International Conference on Music Information Retrieval (ISMIR) |
| G. Xia, J. Tay, R. Dannenberg, and M. Veloso | Autonomous Robot Dancing Driven by Beats and Emotions of Music. [PDF] [Slides] [Demo1] [Demo2] |
2012 | Proc. 12th International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS) |
| D. Liang, G. Xia, and R. Dannenberg | A Framework for Coordination and Synchronization of Media. [PDF] [Slides] [Talk + Demo] |
2011 | Proc. 11th the International Conference on New Interfaces for Musical Expression (NIME) |
Music Information Retrieval
| Authors | Title | Year | Book/Journal/Proceedings |
|---|---|---|---|
| J. Jiang, D. Chin, Y. Zhang, and G. Xia | Learning Hierarchical Metrical Structure Beyond Measures [PDF] |
2022 | Proc. 23nd the International Conference on Music Information Retrieval (ISMIR) |
| L. Lin, Q. Kong, J. Jiang, and G. Xia | A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesis. [PDF] |
2021 | Proc. 22nd the International Conference on Music Information Retrieval (ISMIR) |
| J. Jiang, K. Chen, W. Li, and G. Xia | Large Vocabulary Chord Transcription via Chord Structure Decomposition. [PDF] |
2019 | Proc. 20th the International Conference on Music Information Retrieval (ISMIR) |
| G. Xia, T. Huang, Y. Ma, R. Dannenberg, and C. Faloutsos | MidiFind: Similarity Search and Popularity Mining in Large MIDI Databases. [PDF] [Website: online MIDI search] |
2014 | Sound, Music, and Motion, LNCS 2014, pp 259 - 276 |
| G. Xia, D. Liang, R. Dannenberg, and M. Harvilla | Segmentation, Clustering, and Display in a Personal Audio Database for Musicians. [PDF] [Poster] |
2011 | Proc. 12th the International Conference on Music Information Retrieval (ISMIR) |
| Title | Description | Year | Institution |
|---|---|---|---|
| Music AI concert | A Music AI concert[Video]. | Summer 2022 | MBZUAI and Music X Lab |
| Sally Garden | Human-Robot Interactive Performances in collaboration with the world-leading saxphone robot [Video] and piano robot [Video]. | Fall 2015 | Carnegie Mellon, Waseda University, and TeoTronica Inc. |
| Laptop Orchestra |
A collection of distributed performers who play laptop computers as instrument. [Webpage] [Poster] | Summer 2012 | Carnegie Mellon |
| DI & XIAO Solo Concert |
Some representitive pieces of DI and XIAO were performed, including: [Video1] [Video2] [Video3] [Video4] [Video5] | Summer 2010 | Peking University |
Web templates from Byron Boots: http://www.cc.gatech.edu/~bboots3/ |
|---|