• Alvaro Sanchez-Gonzalez,Nicolas Heess,Jost Tobias Springenberg.et al.: Graph networks as learnable physics engines for inference and control ,arxiv,2018

这篇文章是用图网络方法进行多体系统动力学学习以及控制的经典论文

• Thomas Kipf,Ethan Fetaya,Kuan-Chieh Wang.et al.: Neural Relational Inference for Interacting Systems ,arXiv:1802.04687, 2018.

这篇文章首次将显示地学习网络结构与系统的动力学规则结合在了一起。

• Seungwoong Ha,Hawoong Jeong: Towards Automated Statistical Physics : Data-driven Modeling of Complex Systems with Deep Learning ,arxiv,2020

该篇将NLP中的Transformer模型中的自注意力机制应用到了多体复杂系统中的自动建模问题中来。可以学习动态的网络结构以及动力学。

• Danilo Jimenez Rezende Shakir Mohamed: Variational Inference with Normalizing Flows, arXiv:1505.05770v6

这篇文章提出了一种新型梯度计算方法，能够更加方便、快速地对概率密度函数进行梯度计算，从而进行变分推断，目前几乎已经成为了动力学学习中的一种必备方法。

• Fan Yang†, Ling Chen∗†, Fan Zhou†, Yusong Gao‡, Wei Cao：RELATIONAL STATE-SPACE MODEL FOR STOCHASTIC MULTI-OBJECT SYSTEMS, arXiv:2001.04050v1

这篇文章提出了一种基于状态空间的随机多体系统自动学习建模方法。

• Ricky T. Q. Chen*, Yulia Rubanova*, Jesse Bettencourt*, David Duvenaud: Neural Ordinary Differential Equations, arXiv:1806.07366v5

这篇文章首次提出了运用最优控制原理可微分地求解常微分方程的方法，并将深度网络连续化，并视作一种动力系统，因此对深度网络的训练也被转化为一种常微分方程的求解问题。

• Michael John Lingelbach, Damian Mrowca, Nick Haber, Li Fei-Fei, and Daniel L. K. Yamins: TOWARDS CURIOSITY-DRIVEN LEARNING OF PHYSICAL DYNAMICS, “Bridging AI and Cognitive Science” (ICLR 2020)

这是一篇提出了让机器主动干扰物理系统，从而更有效地学习物理体系规则的人工智能系统。

• Chengxi Zang and Fei Wang: Neural Dynamics on Complex Networks, AAAI 2020

AAAI 2020的best paper，将Neural ODE与图网络结合针对复杂网络的一般的动力学西问题，利用最优控制原理进行求解。该文还将半监督节点分类问题也转化为最优控制问题，从而取得了显著的效果。

Judea Pearl 如下的三篇论文是现代因果推理的必读文章

• J. Pearl, “The Seven Tools of Causal Inference with Reflections on Machine –

• Learning,” July 2018. Communications of ACM, 62(3): 54-60, March 2019 J. Pearl, “Causal and counterfactual inference,” October 2019. Forthcoming section in The Handbook of Rationality, MIT Press.

• J. Pearl, “Causal inference in statistics: An overview,” Statistics Surveys, 3:96–146, 2009.

第一篇首先指出当前强人工智能的三大主要困难，然后指出现代因果模型将会帮助解决这些困难。介绍了因果推理本质的三级因果之梯，提出一个回答因果问题的引擎，总结了因果推理当前七大工具。第二篇文章介绍了结构因果模型（SCM）的理论框架及其应用。第三篇文章是因果推理的一个全面细致的综述。

Bernhard Scholkopf 及其团队，有两篇关键论文

• Causality for machine learning, B Schölkopf – ‎2019

• Foundations of Structural Causal Models with Cycles and Latent Variables, Bongers etc -2020

如果 Judea Pearl 对于因果推理的贡献是从零到一，那么有人称 Causality for machine learning 把因果推理从1 推进到1.5，这篇文章总结和阐述了其团队在融合 machine learning 和 causal inference 多年工作成果和深刻见解。第二篇文章展示了其团队解决有环因果模型这一个根本性难题的努力尝试。

Causal Inference and Data-Fusionin Econometrics 2019 Paul Hunermund 和 Elias Bareinboim（Judea Pearl 学生）是在披着经济学的皮讲解着 Causal AI 如何解决 confounding bias, selection bias and 迁移学习这些难题的因果理论框架。该文章是现代因果理论如何结合某个具体领域的标杆文章。

• A Survey on Causal Inference, 2020 Liuyi YAO etc,

（观测数据因果推断是热点，尤其是结合机器学习）Causal inference is a critical research topic across many domains, such as statistics, computer science, education, public policy and economics, for decades. Nowadays, estimating causal effect from observational data has become an appealing research direction owing to the large amount of available data and low budget requirement, compared with randomized controlled trials. Embraced with the rapidly developed machine learning area, various causal effect estimation methods for observational data have sprung up. 本文介绍在潜结果框架下的因果推断方法，这些方法可以按照其所需要的因果假设分类，而每个类别我们会分别介绍对应的机器学习和统计方法，也会介绍其在各领域的应用，最后我们会介绍各种方法的benchmark. Pearl‘s 结构因果模型并不是当前唯一流行的因果建模框架，Potential Outcome 是另外一个主流因果建模框架，尤其是在计量经济学，流行病学等非AI领域非常流行，这篇论文是相关内容一个全面准确清晰的综述。

Kun Kuang 老师的 Stable Learning 相关论文

• Calculus For Stochastic Interventions: Causal Effect Identification and Surrogate Experiments, J. Correa, E. Bareinboim. AAAI-20.

• A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms 2019 by Yoshua Bengio etc.

• A Second Chance to Get Causal Inference Right: A Classification of Data Science Tasks 因果推断前言综述 by Migual. A recent influx of data analysts, many not formally trained in statistical theory, bring a fresh attitude that does not a priori exclude causal questions.

• 时间序列因果 Granger Causality

…..

• 1. Nalisnick, E., Mitra, B., Craswell, N., & Caruana, R. (2016, April). Improving document ranking with dual word embeddings. In Proceedings of the 25th International Conference Companion on World Wide Web (pp. 83-84).

This paper presents how IN-IN and IN-OUT vector cosine similarities models collocative and substitutive word pairs.

• 2. Levy, O., & Goldberg, Y. (2014). Neural word embedding as implicit matrix factorization. In Advances in neural information processing systems (pp. 2177-2185).

This paper proves that term-context embedding (T_iC_j) in SGNS implicitly models pointwise mutual information (PMI).

• 3. Levy, O., Goldberg, Y., & Dagan, I. (2015). Improving distributional similarity with lessons learned from word embeddings. Transactions of the Association for Computational Linguistics, 3, 211-225.

This paper proposes that the term-term vector cosine similarity model 2nd order association and the term-context vector cosine similarity model 1st order association, and suggests that adding these two vectors to obtain a combined vector improves the performance of word2vec on certain NLP tasks.

• 4. Rapp, R. (2002, August). The computation of word associations: comparing syntagmatic and paradigmatic approaches. In Proceedings of the 19th international conference on Computational linguistics-Volume 1 (pp. 1-7). Association for Computational Linguistics.

This paper called 1st and 2nd order associations “syntagmatic” and “paradigmatic” relations, respectively, following the convention created by Ferdinand de Saussure (the founding father of linguistics). This paper also proposes to measure 1st order association by co-occurrence and 2nd order association by comparing context word distribution similarity.

• 5. Teng, C. Y., Lin, Y. R., & Adamic, L. A. (2012, June). Recipe recommendation using ingredient networks. In Proceedings of the 4th Annual ACM Web Science Conference (pp. 298-307).

This paper constructed two food-ingredient networks (one links collocation ingredients as they co-used in food receipts, the other links substitutive ingredients as suggested by users), and found that “purified”, complement ingredient network can be obtained through removing substitutive parts from collocation network. And the substitutes network is more informative in predicting users preference on receipts.

• 6. Sauer, C., Haigh, A., & Rachleff, J. Cooking up Food Embeddings.

This paper analyzes two kinds of food ingredients pairs, including substitutes and complements. They define complement pairs as frequently co-used (collocation) ingredients (maximizing Ti*Cj), and substitute pairs as those similar and replaceable (maximizing Ti*Tj).

• 7. Neffke, F. M. (2019). The value of complementary co-workers. Science Advances, 5(12), eaax3370.

This paper found that having co-workers with an education degree similar to one’s own is costly, having co-workers with a complementary education degree is beneficial. How this is defined?

• 8. Dibiaggio, L., Nasiriyar, M., & Nesta, L. (2014). Substitutability and complementarity of technological knowledge and the inventive performance of semiconductor companies. Research policy, 43(9), 1582-1593.

This paper found that complementarity between knowledge components positively contributes to firms’ inventive capability, whereas the overall level of substitutability between knowledge components generally has the opposite effect. How this is define?

• 1.Brockmann, D., & Helbing, D. (2013). The hidden geometry of complex, network-driven contagion phenomena. science, 342(6164), 1337-1342.

This paper analyzed disease spread via the “effective distance” rather than geographical distance, wherein two locations that are connected by a strong link are effectively close. The approach was successfully applied to predict disease arrival times or disease sources.

• 2.Tshitoyan, V., Dagdelen, J., Weston, L., Dunn, A., Rong, Z., Kononova, O., … & Jain, A. (2019). Unsupervised word embeddings capture latent knowledge from materials science literature. Nature, 571(7763), 95-98.

统计力学的核心问题之一是如何准确地计算多粒子系统的自由能、热力学量。自上个世纪以来，物理学家发展了各种各样的理论和方法来解决这类问题。其中，平均场理论和相关消息传递算法在某些情况下能够给出非常好的系统变分自由能结果，下面这篇综述文章详细介绍了相关消息传递算法。

• Understanding belief propagation and its generalizations J. Yedidia, W. Freeman, Y. Weiss International Joint Conference on Artificial Intelligence (IJCAI), 2001

统计物理的基本问题和机器学习中的非监督学习具有天然的联系：统计物理中的玻尔兹曼分布对应于贝叶斯推断的后验概率；最小化自由能原理等价于变分推断；寻找统计物理系统的基态等价于最大似然学习等。那么，如何利用近年来快速发展的深度学习方法老帮助我们解决统计力学问题？这篇文章提出了变分自回归神经网络(Variational Autoregressive Networks)，拓展了传统意义上的平均场方法。

• Solving Statistical Mechanics Using Variational Autoregressive Networks D. Wu, L. Wang, P. Zhang Phys. Rev. Lett., 122, 080602 (2019)

张量网络方法是另一类广泛用于计算配分函数和自由能的方法，但通常受限于格点系统，因为张量网络的缩并属于#-P难问题。这篇文章，提出了一种近似缩并任意拓扑结构张量网络的方法。

• Contracting Arbitrary Tensor Networks: general approximate algorithm and applications in graphical models and quantum circuit simulations F. Pan, P. Zhou, S. Li, P. Zhang arXiv preprint arXiv:1912.03014

”Quantum State Tomography”

[1] Juan Carrasquilla, Ciacomo Torlai, Roger Melko, Leandro Aolita. Reconstructing quantum states with generative models. arXiv: 1810.10584

[2] Peter Cha, Paul Ginsparg, Felix Wu, Juan Carrasquilla, Peter L. McMahon, Eun-Ah Kim. Attentioned-based quantum tomography. arXiv: 2006.12469

”Autoregressive generative model”

[3] Dian Wu, Lei Wang, Pan Zhang. Solving Statistical Mechanics Using Variational Autoregressive Networks. arXiv:1809.10606

[4] Or Sharir, Yoav Levine, Noam Wies, Giuseppe Carleo, Amnon Shashua. Deep autoregressive models for the efficient variational simulation of many-body quantum systems. arXiv:1902.04057

”Flow based model”

[5] Shuo-Hui Li, Chen-Xiao Dong, Linfeng Zhang, Lei Wang. Neural Canonical Transformation with Symplectic Flows. arXiv: 1910.00024.