Ongoing Projects

Project Overview: This project tackles the risks of AI-generated content (AIGC) by developing a multi-level governance framework. It will create a taxonomy to categorize AIGC risks like deepfakes and algorithmic biases, using empirical analysis and expert interviews to identify high-risk scenarios.

The project will explore AIGC’s impact on public trust and media credibility through case studies. It will co-design technical solutions, such as explainability tools and content detection systems and integrate them into media workflows. Governance efforts will include creating the Hong Kong AIGC Risk Database, evaluating standards through adversarial testing, and developing policy toolkits informed by global governance models. The project aims to deliver a scalable framework for responsible AIGC innovation.

Principal Investigator: Celine Yunya SONG (HKUST)

Collaborators: Janet Hsiao (HKUST), Masaru Yarime (HKUST), Yangqiu Song (HKUST)

Project Overview: This project pioneers a new paradigm for human–AI interaction by reframing Generative AI (GenAI) as a cognitive partner rather than a content generator. Instead of producing text on behalf of users, GenAI engages them in AI-mediated meta-communication—a reflective process that prompts clarification, elaboration, and refinement of thought. Through this interaction, individuals learn to express their ideas with greater clarity, specificity, and persuasiveness.

The study moves beyond traditional single-domain experiments by adopting a multi-domain, multi-level experimental framework. Each experiment situates GenAI in a distinct communicative context—such as personal feedback, career advising, economic forecasting, journalism, and gendered self-promotion—to examine how AI scaffolds human reasoning across diverse cognitive demands, levels of expertise, and communication asymmetries. This comparative design enables a deeper understanding of how AI can support human cognition in varied real-world situations.

Methodologically, the project introduces several innovations. A proof-of-concept pipeline integrates real-time AI prompts into open-ended responses, followed by behavioral and perceptual evaluations by external raters. Hybrid evaluation metrics combine linguistic algorithms (e.g., clarity, concreteness) with behavioral outcomes (e.g., forecast accuracy, writing quality) and user experience data (e.g., effort, satisfaction). Cross-domain experimentation operationalizes the “AI-as-partner” model across contexts, while an equity testing framework examines how AI-mediated reflection shapes self-promotion behaviors across gender, yielding both theoretical and policy-relevant insights.

By conceptualizing AI as a meta-communicator—an active co-thinker that enhances reasoning and reflection—this project redefines the boundaries of communication science and human–AI collaboration. Its open-source tools and evaluation rubrics will inform the design of AI-enhanced education, journalism, and policymaking interfaces, advancing a more reflective, equitable, and human-centered AI future.

Investigators: David Hagmann (PI, HKUST), Yang Lu (Co-PI, HKUST), Celine Yunya Song (Co-PI, HKUST), and George Loewenstein (Co-PI, Carnegie Mellon University)

Project Overview: Video generation technology has developed rapidly in recent years. Applying video generation technology to content production such as film, television, and animation has huge commercial value and can inspire more creative inspiration and forms.

However, the film and television industry has special requirements for video quality. On the content side, the generated video content needs to meet the plot requirements and the lens expression of the director, screenwriter, and creator; on the video quality side, it needs to meet a series of standards such as clarity, image quality, color, aesthetics, and physical laws. Currently, video generation models in the industry still have many problems in these two aspects.

This project aims to study quality evaluation methods for generated videos, study refined human-computer interaction methods, and guide the generation of highly controllable and high-quality videos to meet the industrial needs of film and television production in terms of both content and image quality.

Investigator: Anyi Rao (HKUST)

Project Overview: In today’s fast-paced digital landscape, the demand for timely, accurate, and engaging news content is higher than ever. Traditional news production processes, however, face challenges in scaling output while maintaining quality, particularly for global audiences.

This project proposes leveraging Artificial Intelligence Generated Content (AIGC) technologies to automate the creation of high-quality multilingual news texts, images, and short videos tailored to different regions. By integrating AIGC into media workflows, we aim to revolutionize news production efficiency, reduce operational costs, and enhance content accessibility across linguistic and cultural boundaries for global dissemination.

Principal Investigator: Anyi Rao (HKUST)

Collaborators: Celine Song (HKUST), Xiaomin Ouyang (HKUST)

Project Overview: This project addresses the limitations of current AI music creation (AMC) by developing a pioneering audio-oriented, end-to-end framework that unifies composition, performance control, and audio synthesis. Instead of relying on symbolic intermediaries that discard timbre, emotion, and multi-track effects, it treats finished audio as the sole workspace, enabling large-scale disentangled representation learning to capture expressive, harmonic, and timbral qualities in a single model. A comprehensive, open-source dataset of high-resolution audio recordings will be curated and released to fuel the development of algorithms. Guided by human-like perception, the project will jointly optimise all musical factors through these learned representations, allowing users to flexibly control style, emotion, instrumentation, and harmony in multi-track generation. The exact representations will support precise “re-creation” of existing pieces—enabling note-level editing, timbre swapping, or emotional re-targeting without re-recording. The deliverables—dataset, open models, and editing toolkit—will advance academic research on artificial creativity while equipping the media, entertainment, and metaverse industries with controllable, culturally aware music generation capabilities.

Principal Investigator: Wei XUE (HKUST)

Collaborators: Qiuqiang KONG (CUHK), Xu TAN (Formerly Microsoft)

Project Overview: This project confronts the fragility of speech processing in noisy, real-world settings by introducing a unified “acoustic-environment representation” that fuses spatial cues, temporal-spectral texture, and noise statistics into one compact, learnable vector. Instead of cascading separate estimators for direction-of-arrival, coherence, and interference power—each demanding oracle knowledge of the others—the framework jointly optimizes every acoustic factor end-to-end, enabling spatial filters to be generated directly from the representation with minimal prior assumptions. A continuously updated memory module will enable the system to self-adapt on-the-fly to new data captured, thereby closing the loop between environmental sensing and enhanced performance.

Principal Investigator: Wei XUE (HKUST)

Collaborators: Qiuqiang KONG (CUHK)

Project Overview: In the rapidly evolving landscape of a technology-driven world, the fusion of arts and technology has birthed Art Tech, fundamentally transforming the creation, reception, and interaction with arts and culture. This research project positions itself at the intersection of arts and science, leveraging cutting-edge science and technology to revolutionize human-AI interactions. By focusing on the sustainability of Hong Kong’s arts ecosystem, it explores AI-driven Art Tech opportunities to invigorate the city’s cultural scene, fostering innovative modalities of artistic production and consumption that promise substantial socio-economic ripple effects across business, healthcare, and education sectors. Drawing on interdisciplinary expertise from AI, cognitive science, and the arts, the project pursues three core tasks: developing algorithmic systems for artefact creation informed by cognitive, physiological, and behavioral data; pioneering immersive XR platforms for artistic delivery and audience engagement in education; and deploying applications to enhance human creativity. Anticipated outcomes include dedicated application projects for global technology testing, a groundbreaking Research Theatre for Art Tech innovations, and a comprehensive Digital Art and Policy Network connecting Hong Kong to international developments. Harnessing AI advancements, this initiative is primed to reshape the art world and creative industries, delivering profound social, educational, and economic advantages in Hong Kong and the Greater Bay Area, while providing an interdisciplinary framework to tackle post-COVID societal challenges and drive inclusive socio-cultural-economic progress.

Project Coordinator: Yike GUO (HKUST)

Co-Project Coordinator: Johnny POON (HKBU)

Co-Investigators: Wei XUE (HKUST) Jie CHEN (HKBU)

Project Overview: The Art ID Registry proposes the combined use of DID:ART and DID.ART as a decentralized blockchain platform developed to accelerate the art technology industry in Hong Kong by integrating intellectual property (IP) protection with generative AI innovations. This will be led by Dr. Daniel Chun of the HKUST EMIA and AMCC divisions as part of the Media Intelligence Research Centre project. This project harnesses Hong Kong’s strategic position as a global art hub to drive sustainable creative industries and global cultural exchange.

Key features of the Art ID Registry include:

• Integration of the Content Provenance Coalition’s C2PA protection mechanism
• Development of a set of protocols that follows W3C Decentralized Identity (DID) standards using W3C compliant DID:ART, to provide immutable, transparent provenance and metadata for all kinds of artworks (including both physical and digital art)
• Support for digital media and traditional art forms, enhancing provenance tracking, rights management, licensing, and artist authentication.
• Backing by innovative research from HKUST and support from the Smart City Consortium of Hong Kong and Art ID Standard consortium, which involves global art market stakeholders like auction houses, insurers, galleries, artists community.

The project addresses challenges in the art market such as lack of unique identification for artworks (analogous to VINs for vehicles, ISBN for books, DOI for academic publishing) and the need for secure digital media authentication to reduce fake and fraudulent use of copyrighted content. By providing a robust digital infrastructure with blockchain and digital watermarking technologies, the Art ID Registry aims to transform Hong Kong’s art ecosystem into a leading global model of art market innovation, supporting artists, collectors, galleries, and institutions worldwide through trusted digital identity and provenance verification.

Project Deliverables - Developing an Art ID Registry Prototype (W3C DID:ART) and DID.ART digital platform service endpoints and API for future growth.

Related Publication: Chun, D. (2023, August). When the NFT Hype Settles, What Is Left beyond Profile Pictures? A Critical Review on the Impact of Blockchain Technologies in the Art Market. In Arts (Vol. 12, No. 5, p. 181). MDPI.

Principal Investigator: Daniel Chun (HKUST)

Collaborators: Smart City Consortium, Education University Hong Kong, Lingnan University, Burgundy School of Business, Art ID Standard Consortium

Project Overview: The project aims to develop a virtual reality (VR) platform in the metaverse to provide an immersive and interactive environment for cross-campus teaching and learning. This initiative is grounded in a design thinking course that leverages experiential learning pedagogy, facilitating substantial interactions among instructors, students, and peers. The project seeks to address challenges identified through years of experience in higher education, such as the mismatch between pedagogical approaches and physical spaces, as well as the lack of immersive interaction in remote teaching and learning contexts. Employing a user-centered approach, we identify the genuine concerns of both students and instructors, who collaborate closely to explore the intersection of virtual and remote education with VR technologies. The metaverse classroom is implemented in actual classes and refined based on direct feedback from instructors and students. The data generated and collected throughout the project represent valuable assets for exploring the evolving landscape of education through the lens of virtual reality technologies.

PI: Rong Zhang (HKUST), Pan HUI (HKUST-GZ)