复杂系统自动建模读书会:从数据驱动到可解释性,探索系统内在规律|内附75篇领域必读文献
导语
读书会背景
读书会背景
-
面对一个真实的复杂系统,我们如何提炼底层的机制、构建简单模型?
-
这些把握住系统本质的简单规则从何而来?
-
对于更复杂的系统,例如基因调控系统、社会经济系统,我们面对庞杂的数据和复杂的系统状态,要如何做出假设?
-
随着深度学习技术的发展,有没有可能设计出一个自动化的系统,根据不同的任务需求,能够通过观察数据,或者是洞察规律,对各式复杂系统建模?
读书会框架
读书会框架
当前复杂系统建模研究主要聚焦于三个核心领域:
-
首先,时间维度上的动力学建模,旨在捕捉系统随时间演化的特性;
-
其次,空间维度上的元素交互关系建模,致力于揭示系统内部组成单元之间的相互作用机制;
-
最后,基于对复杂系统的既有认知和理解,探索系统的定向设计方法。
发起人介绍
发起人介绍
报名参与读书会
报名参与读书会
运行模式
报名方式
扫码报名(可开发票)
第二步:填写信息后,付费报名。
如需用支付宝支付,请在PC端进入读书会页面报名支付:https://pattern.swarma.org/study_group/51
加入社区后可以获得的资源:
完整权限,包括线上问答、录播回看、资料共享、社群交流、信息同步、共创任务获取积分等。
参与共创任务获取积分,共建学术社区:
读书会采用共学共研机制,成员通过内容共创获积分(字幕修改、读书会笔记、论文速递、公众号文章、集智百科、论文解读等共创任务),积分符合条件即可退费。发起人和主讲人同样遵循此机制,无额外金钱激励。
领域必读文献清单
领域必读文献清单
模块一:数据驱动的复杂系统建模
储备池计算
-
Pathak, J., Hunt, B., Girvan, M., Lu, Z., & Ott, E. (2018). Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach. Physical review letters, 120(2), 024102. -
Yan, M., Huang, C., Bienstman, P., Tino, P., Lin, W., & Sun, J. (2024). Emerging opportunities and challenges for the future of reservoir computing. Nature Communications, 15(1), 2056. -
Li, X., Zhu, Q., Zhao, C., Duan, X., Zhao, B., Zhang, X., … & Lin, W. (2024). Higher-order Granger reservoir computing: simultaneously achieving scalable complex structures inference and accurate dynamics prediction. Nature Communications, 15(1), 2506. | 本篇文献介绍可见:Nat. Commun. 前沿:数据驱动的复杂系统高阶结构推断与动力学预测
物理知识引导的神经网络架构
推荐语:第一篇文章将物理规律引入到神经网络架构之中;第二篇文章综述了动力系统中的现代Koopman理论;第三篇文章构建了数理先验信息嵌入式的机器学习框架,哈密顿神经Koopman算子。
-
Greydanus, S., Dzamba, M., & Yosinski, J. (2019). Hamiltonian neural networks. Advances in neural information processing systems, 32.
-
Brunton, S. L., Budišić, M., Kaiser, E., & Kutz, J. N. (2021). Modern Koopman theory for dynamical systems. arXiv preprint arXiv:2102.12086.
-
Zhang, J., Zhu, Q., & Lin, W. (2024). Learning Hamiltonian neural Koopman operator and simultaneously sustaining and discovering conservation laws. Physical Review Research, 6(1), L012031. | 本篇文献介绍可见:复旦大学智能复杂体系实验室团队提出了数理先验信息嵌入式机器学习框架
-
Cranmer, M., Greydanus, S., Hoyer, S., Battaglia, P., Spergel, D., & Ho, S. (2020). Lagrangian Neural Networks. In ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations.
-
Rao, C., Ren, P., Wang, Q., Buyukozturk, O., Sun, H., & Liu, Y. (2023). Encoding physics to learn reaction–diffusion processes. Nature Machine Intelligence, 5(7), 765-779.
推荐语:利用网络重整化群启发的神经网络识别大规模复杂网络的关键“骨架”,捕捉长期演化模式,进行高效且精准的动力学模拟。
-
Predicting Long-term Dynamics of Complex Networks via Identifying Skeleton in Hyperbolic Space. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
神经微分方程
-
Chen, R. T., Rubanova, Y., Bettencourt, J., & Duvenaud, D. K. (2018). Neural ordinary differential equations. Advances in neural information processing systems, 31. -
Dupont, E., Doucet, A., & Teh, Y. W. (2019). Augmented neural odes. Advances in neural information processing systems, 32. -
Zhu, Q., Guo, Y., & Lin, W. (2021). Neural delay differential equations. The Ninth International Conference on Learning Representations (ICLR 2021).
小样本下的数据驱动建模:可解释性->泛化性->鲁棒性
-
Lai, Z., Mylonas, C., Nagarajaiah, S., & Chatzi, E. (2021). Structural identification with physics-informed neural ordinary differential equations. Journal of Sound and Vibration, 508, 116196. -
Li, Q., Wang, T., Roychowdhury, V., & Jawed, M. K. (2023). Rapidly encoding generalizable dynamics in a Euclidean symmetric neural network. Extreme Mechanics Letters, 58, 101925. -
Li, Q., Wang, T., Roychowdhury, V., & Jawed, M. K. (2023). Metalearning generalizable dynamics from trajectories. Physical Review Letters, 131(6), 067301. | 本篇文献介绍可见:PRL速递:AI 学习玩弹簧玩具——从轨迹到通用动力学的元学习 -
Ji, W., & Deng, S. (2021). Autonomous discovery of unknown reaction pathways from data by chemical reaction neural network. The Journal of Physical Chemistry A, 125(4), 1082-1092. -
Li, Q., Chen, H., Koenig, B. C., & Deng, S. (2023). Bayesian chemical reaction neural network for autonomous kinetic uncertainty quantification. Physical Chemistry Chemical Physics, 25(5), 3707-3717.
复杂系统智能控制
-
D’Souza, R. M., di Bernardo, M., & Liu, Y. Y. (2023). Controlling complex networks with complex nodes. Nature Reviews Physics, 5(4), 250-262. | 本篇文献介绍可见:Nat.Rev.Phys.速递:用复杂节点控制复杂网络
-
Baggio, G., Bassett, D. S., & Pasqualetti, F. (2021). Data-driven control of complex networks. Nature communications, 12(1), 1429.| 本篇文献介绍可见:数据驱动的复杂网络控制方法 | 复杂性科学顶刊精选8篇
-
Böttcher, L., Antulov-Fantulin, N., & Asikis, T. (2022). AI Pontryagin or how artificial neural networks learn to control dynamical systems. Nature communications, 13(1), 333.
-
Chang, Y. C., Roohi, N., & Gao, S. (2019). Neural lyapunov control. Advances in neural information processing systems, 32. -
Zhang, J., Zhu, Q., & Lin, W. (2022). Neural stochastic control. Advances in neural information processing systems, 35, 9098-9110.
连续时间生成模型
-
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2020). Score-based generative modeling through stochastic differential equations. The Ninth International Conference on Learning Representations (ICLR 2021). -
Lipman, Y., Chen, R. T., Ben-Hamu, H., Nickel, M., & Le, M. (2023). Flow matching for generative modeling. The Eleventh International Conference on Learning Representations (ICLR 2023). -
Zhu, Q., & Lin, W. (2024). Switched flow matching: Eliminating singularities via switching ODEs, 41st International Conference on Machine Learning (ICML 2024). | 本篇文献介绍可见:ICML2024 |复旦大学智能复杂体系基础理论与关键技术实验室研究团队3篇论文被国际机器学习顶级学术会议录用
复杂网络降维表达
验证了低秩结构在复杂系统中的普遍存在性:
-
Thibeault, V., Allard, A., & Desrosiers, P. (2024). The low-rank hypothesis of complex systems. Nature Physics, 20(2), 294-302. | 本篇文献介绍可见:Nature Physics评论:复杂系统的内在简单性
数据驱动下对低秩结构的估计和学习:
-
Cao, W., Picci, G., & Lindquist, A. (2023). Identification of low rank vector processes. Automatica, 151, 110938.
通过优化手段提取复杂网络关键节点:
-
MacLaren, N. G., Barzel, B., & Masuda, N. (2024). Observing network dynamics through sentinel nodes. arXiv preprint arXiv:2408.00045.
模块二:复杂网络结构推断
基于Granger因果的推断方法
-
Tank, A., Covert, I., Foti, N., Shojaie, A., & Fox, E. B. (2021). Neural granger causality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8), 4267-4279.
-
Bellot, A., Branson, K., & van der Schaar, M. (2021). Neural graphical modelling in continuous-time: consistency guarantees and algorithms. The Ninth International Conference on Learning Representations (ICLR 2021).
基于稀疏辨识的结构推断方法
-
Casadiego, J., Nitzan, M., Hallerberg, S., & Timme, M. (2017). Model-free inference of direct network interactions from nonlinear collective dynamics. Nature communications, 8(1), 2192.
-
Delabays, R., De Pasquale, G., Dörfler, F., & Zhang, Y. (2024). Hypergraph reconstruction from dynamics. arXiv preprint arXiv:2402.00078.
基于常微分方程优化的方法
-
Yan, Z., Gui, L., Xu, K., & Lan, Y. (2023). Reconstructing dynamics of complex systems from noisy time series with hidden variables. New Journal of Physics, 25(8), 083011.
基于图神经网络的方法
-
Kipf, T., Fetaya, E., Wang, K. C., Welling, M., & Zemel, R. (2018, July). Neural relational inference for interacting systems. In International conference on machine learning (pp. 2688-2697). PMLR.
-
Han, Z., Fink, O., & Kammer, D. S. (2024). Collective relational inference for learning heterogeneous interactions. Nature Communications, 15(1), 3191.
基于Intervention的因果推断方法
-
Sui, Y., Wang, X., Wu, J., Lin, M., He, X., & Chua, T. S. (2022, August). Causal attention for interpretable and generalizable graph classification. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 1696-1705).
-
Zhao, J., Gan, Z., Huang, R., Guan, C., Shi, J., & Leng, S. (2024). Detecting dynamical causality via intervened reservoir computing. Communications Physics, 7(1), 232.
-
Brouillard, P., Lachapelle, S., Lacoste, A., Lacoste-Julien, S., & Drouin, A. (2020). Differentiable causal discovery from interventional data. Advances in Neural Information Processing Systems, 33, 21865-21877.
-
Tigas, P., Annadani, Y., Jesson, A., Schölkopf, B., Gal, Y., & Bauer, S. (2022). Interventions, where and how? experimental design for causal models at scale. Advances in neural information processing systems, 35, 24130-24143.
模块三:具有可解释性的复杂系统推断(动力学+网络结构)
-
系统具有超高维度、高度非线性和不确定性 -
对初始值敏感,表现出混沌特性 -
某些系统会根据预测结果进行调整和反馈,影响长期预测 -
普遍存在随机效应
基于符号回归的动力学推断方法
-
Cranmer, M. (2023). Interpretable machine learning for science with PySR and SymbolicRegression. jl. arXiv preprint arXiv:2305.01582.
-
Valipour, M., You, B., Panju, M., & Ghodsi, A. (2021). Symbolicgpt: A generative transformer model for symbolic regression. arXiv preprint arXiv:2106.14131. -
Kamienny, P. A., d’Ascoli, S., Lample, G., & Charton, F. (2022). End-to-end symbolic regression with transformers. Advances in Neural Information Processing Systems, 35, 10269-10281.
-
Mundhenk, T., Landajuela, M., Glatt, R., Santiago, C. P., & Petersen, B. K. (2021). Symbolic regression via deep reinforcement learning enhanced genetic programming seeding. Advances in Neural Information Processing Systems, 34, 24912-24923. -
Song, Z., Cameron, M. K., & Yang, H. (2023). A finite expression method for solving high-dimensional committor problems. arXiv preprint arXiv:2306.12268.
-
Sun, F., Liu, Y., Wang, J. X., & Sun, H. (2022). Symbolic physics learner: Discovering governing equations via monte carlo tree search. arXiv preprint arXiv:2205.13134.
-
Shi, H., Ding, J., Cao, Y., Liu, L., & Li, Y. (2022). Learning symbolic models for graph-structured physical mechanism. In The Eleventh International Conference on Learning Representations.
基于基函数的推断方法
-
Gao, T. T., & Yan, G. (2022). Autonomous inference of complex network dynamics from incomplete and noisy data. Nature Computational Science, 2(3), 160-168. | 本篇文献介绍可见:从非完整和含噪声的数据中推理复杂网络动力学 -
Gao, T. T., Barzel, B., & Yan, G. (2024). Learning interpretable dynamics of stochastic complex systems from experimental data. Nature Communications, 15(1), 6029.| 本篇文献介绍可见:学术争鸣丨物理科学与工程学院严钢团队在复杂系统可解释推理方面取得进展,研究成果发表于《自然·通讯》
-
Frishman, A., & Ronceray, P. (2020). Learning force fields from stochastic trajectories. Physical Review X, 10(2), 021009. -
Brückner, D., & Broedersz, C. P. (2024). Learning dynamical models of single and collective cell migration: a review. Reports on Progress in Physics.
-
Course, K., & Nair, P. B. (2023). State estimation of a physical system with unknown governing equations. Nature, 622(7982), 261-267.
-
Zhang, Y., Guo, Y., Zhang, Z., Chen, M., Wang, S., & Zhang, J. (2022). Universal framework for reconstructing complex networks and node dynamics from discrete or continuous dynamics data. Physical Review E, 106(3), 034315. | 本篇文献介绍可见:复杂系统的逆向工程——通过时间序列重构复杂网络和动力学
深度学习预测复杂系统临界点
-
Liu, Z., Zhang, X., Ru, X., Gao, T. T., Moore, J. M., & Yan, G. (2024). Early predictor for the onset of critical transitions in networked dynamical systems. Physical Review X, 14(3), 031009.
-
Harris, B., Gollo, L. L., & Fulcher, B. D. (2024). Tracking the distance to criticality in systems with unknown noise. Physical Review X, 14(3), 031021.
-
Morr, A., & Boers, N. (2024). Detection of approaching critical transitions in natural systems driven by red noise. Physical Review X, 14(2), 021037.
跨尺度系统建模
-
Chen, X., Soh, B. W., Ooi, Z. E., Vissol-Gaudin, E., Yu, H., Novoselov, K. S., … & Li, Q. (2024). Constructing custom thermodynamics using deep learning. Nature Computational Science, 4(1), 66-85.
-
Mingzhe Yang, Zhipeng Wang, Kaiwei Liu, Yingqi Rong, Bing Yuan, Jiang Zhang, Finding emergence in data by maximizing effective information, National Science Review, 2024, nwae279, https://doi.org/10.1093/nsr/nwae279 | 对本工作感兴趣的伙伴可以关注:因果涌现第三季:12.基于数据驱动的多尺度因果涌现框架
-
Medrano, J., Friston, K., & Zeidman, P. (2024). Linking fast and slow: The case for generative models. Network Neuroscience, 8(1), 24-43.
模块四:应用-超材料设计和城市系统
基于人工智能的结构反向设计:以超材料为例
-
Deng, B., Zareei, A., Ding, X., Weaver, J. C., Rycroft, C. H., & Bertoldi, K. (2022). Inverse design of mechanical metamaterials with target nonlinear response via a neural accelerated evolution strategy. Advanced Materials, 34(41), 2206238. -
Bastek, J. H., & Kochmann, D. M. (2023). Inverse design of nonlinear mechanical metamaterials via video denoising diffusion models. Nature Machine Intelligence, 5(12), 1466-1475. -
Ha, C. S., Yao, D., Xu, Z., Liu, C., Liu, H., Elkins, D., … & Zheng, X. (2023). Rapid inverse design of metamaterials based on prescribed mechanical behavior through machine learning. Nature Communications, 14(1), 5765. -
于相龙,周济著.力学超材料的构筑与超常性能[M].合肥:中国科学技术大学出版社,2021 -
Mao, Y., He, Q., & Zhao, X. (2020). Designing complex architectured materials with generative adversarial networks. Science advances, 6(17), eaaz4169. -
Liu, Z., Zhu, D., Rodrigues, S. P., Lee, K. T., & Cai, W. (2018). Generative model for the inverse design of metasurfaces. Nano letters, 18(10), 6570-6576. -
Zheng, L., Karapiperis, K., Kumar, S., & Kochmann, D. M. (2023). Unifying the design space and optimizing linear and nonlinear truss metamaterials by generative modeling. Nature Communications, 14(1), 7563. -
Wang, L., Chan, Y. C., Ahmed, F., Liu, Z., Zhu, P., & Chen, W. (2020). Deep generative modeling for mechanistic-based learning and design of metamaterial systems. Computer Methods in Applied Mechanics and Engineering, 372, 113377. -
Bastek, J. H., Kumar, S., Telgen, B., Glaesener, R. N., & Kochmann, D. M. (2022). Inverting the structure–property map of truss metamaterials by deep learning. Proceedings of the National Academy of Sciences, 119(1), e2111505119.
城市系统的自动建模
推荐语:城市也是多尺度的复杂系统,现实世界积累的海量真实数据使其成为复杂系统自动建模方法的理想应用场景。本节推荐的参考文章涵盖城市复杂系统的三类主要问题(详见第一篇综述文章)。
-
Ding, J., Liu, C., Zheng, Y., Zhang, Y., Yu, Z., Li, R., … & Li, Y. (2024). Artificial Intelligence for Complex Network: Potential, Methodology and Application. arXiv preprint arXiv:2402.16887.
预测和模拟:预测复杂系统的长期演化状态、韧性对于城市中的基础设施、生态系统至关重要,下列文献综合利用了前述提到的物理知识引导神经网络(PINN)、神经常微分方程(Neural ODE)以及扩散模型(Diffusion Generative Model)技术来克服基于真实数据的建模难题。
-
Li, R., Wang, H., & Li, Y. (2023). Learning slow and fast system dynamics via automatic separation of time scales. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 4380-4390).
-
Li, R., Wang, H., Piao, J., Liao, Q., Li, Y. (2024). Predicting Long-term Dynamics of Complex Networks via Identifying Skeleton in Hyperbolic Space. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
-
Liu, C., Ding, J., Song, Y., Li, Y. (2024). TDNetGen: Empowering Complex Network Resilience Prediction with Generative Augmentation of Topology and Dynamics. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
-
Chen, H., Ding, J., Li, Y., Wang, Y., & Zhang, X. P. (2024). Social physics informed diffusion model for crowd simulation. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 1, pp. 474-482).
-
Li, J., Wang, H., Chen, X. (2024). Physics-informed Neural ODE for Post-disaster Mobility Recovery. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
控制和优化:城市基础设施的攻击与防护是一类经典的复杂系统控制优化应用,传统的组合优化方法面临解空间巨大、搜索低效问题,数据驱动的强化学习方法具有显著的优势(下面第一篇),近期研究表明大语言模型也可用于启发式的求解这类问题(下面第二篇)。
-
Mao, J., Cao, L., Gao, C., Wang, H., Fan, H., Jin, D., & Li, Y. (2023). Detecting vulnerable nodes in urban infrastructure interdependent network. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 4617-4627).
-
Mao, J., Zou, D., Sheng, L., Liu, S., Gao, C., Wang, Y., & Li, Y. (2024). Identify Critical Nodes in Complex Network with Large Language Models. arXiv preprint arXiv:2403.03962.
规律发现:基于真实数据,利用前述复杂系统的动力学推断方法能够发现城市系统的内在演化规律,以人流移动为例,相比经典的“社会力”物理模型(Social Force),基于真实数据提炼的新物理公式能更好的模拟各种场景下的人群移动。
-
Shi, H., Ding, J., Cao, Y., Liu, L., & Li, Y. (2022). Learning symbolic models for graph-structured physical mechanism. In The Eleventh International Conference on Learning Representations.
-
Zhang, G., Yu, Z., Jin, D., & Li, Y. (2022). Physics-infused machine learning for crowd simulation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 2439-2449).
点击“阅读原文”,报名读书会