1

1. Simon, H. A. The Sciences of the Artifcial (MIT Press, Cambridge, 1969).

Simon asks whether there can be a science of the ‘artifcial’ that produces

knowledge about artifcial objects and phenomena.

2. Milner, R. A modal characterisation of observable machine-behaviour. In Trees

in Algebra and Programming, 6th Colloquium 25–34 (Springer, 1981).

In this invited lecture, Robin Milner outlines the idea of studying machine

behaviour using formal logic.

3. Thomaz, A. L. & Breazeal, C. Teachable robots: understanding human teaching

behavior to build more efective robot learners. Artif. Intell. 172, 716–737 (2008).

4. Stone, P. et al. Artifcial Intelligence and Life in 2030. One Hundred Year Study on

Artifcial Intelligence: Report of the 2015–2016 Study Panel https://

ai100.stanford.edu/2016-report (Stanford University, 2016).

5. O’Neil, C. Weapons of Math Destruction: How Big Data Increases Inequality and

Threatens Democracy (Broadway Books, 2016).

This book articulates some of the risks posed by the uncritical use of algorithms

in society and provides motivation for the study of machine behaviour.

6. Future of Life Institute. Autonomous weapons: an open letter from AI &

robotics researchers. https://futureofife.org/open-letter-autonomousweapons/?cn-reloaded=1(2015).

7. Dressel, J. & Farid, H. The accuracy, fairness, and limits of predicting

recidivism. Sci. Adv. 4, eaao5580 (2018).

8. Binns, R. et al. ‘It’s reducing a human being to a percentage’: perceptions of

justice in algorithmic decisions. In Proc. 2018 CHI Conference on Human

Factors in Computing Systems 377 (ACM, 2018).

9. Hudson, L., Owens, C. S. & Flannes, M. Drone warfare: blowback from the new

American way of war. Middle East Policy 18, 122–132 (2011).

10. Kahneman, D., Rosenfeld, A. M., Gandhi, L. & Blaser, T. Noise: how to overcome

the high, hidden cost of inconsistent decision making. Harvard Business Review

https://hbr.org/2016/10/noise (2016).

11. Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J. & Mullainathan, S. Human

decisions and machine predictions. Q. J. Econ. 133, 237–293 (2018).

12. Crawford, K. et al. The AI Now report: The Social and Economic Implications of

Artifcial Intelligence Technologies in the Near-term. https://ainowinstitute.org/

AI_Now_2016_Report.pdf (2016).

13. Amodei, D. et al. Concrete problems in AI safety. Preprint at https://arxiv.org/

abs/1606.06565 (2016).

14. Bakshy, E., Messing, S. & Adamic, L. A. Exposure to ideologically diverse news

and opinion on Facebook. Science 348, 1130–1132 (2015).

15. Bessi, A. & Ferrara, E. Social bots distort the 2016 U.S. Presidential election

online discussion. First Monday 21, 11 (2016).

16. Ferrara, E., Varol, O., Davis, C., Menczer, F. & Flammini, A. The rise of social bots.

Commun. ACM 59, 96–104 (2016).

17. Lazer, D. The rise of the social algorithm. Science 348, 1090–1091 (2015).

18. Tufekci, Z. Engineering the public: big data, surveillance and computational

politics. First Monday 19, 7 (2014).

19. Lee, T.-S. & Chen, I.-F. A two-stage hybrid credit scoring model using artifcial

neural networks and multivariate adaptive regression splines. Expert Syst. Appl.

28, 743–752 (2005).

20. Roszbach, K. Bank lending policy, credit scoring, and the survival of loans.

Rev. Econ. Stat. 86, 946–958 (2004).

21. Huang, C.-L., Chen, M.-C. & Wang, C.-J. Credit scoring with a data mining

approach based on support vector machines. Expert Syst. Appl. 33, 847–856

(2007).

22. Tsai, C.-F. & Wu, J.-W. Using neural network ensembles for bankruptcy

prediction and credit scoring. Expert Syst. Appl. 34, 2639–2649 (2008).

23. Chen, L. & Wilson, C. Observing algorithmic marketplaces in-the-wild. SIGecom

Exch. 15, 34–39 (2017).

24. Chen, L., Mislove, A. & Wilson, C. An empirical analysis of algorithmic pricing

on Amazon marketplace. In Proc. 25th International Conference on World Wide

Web 1339–1349 (International World Wide Web Conferences Steering

Committee, 2016).

25. Hannák, A. et al. Bias in Online freelance marketplaces: evidence from

TaskRabbit and Fiverr. In Proc. ACM Conference on Computer Supported

Cooperative Work and Social Computing 1914–1933 (2017).

26. Cartlidge, J., Szostek, C., De Luca, M. & Clif, D. Too fast too furious—faster

fnancial-market trading agents can give less efcient markets. In Proc. 4th

International Conference on Agents and Artifcial Intelligence 126–135 (2012).

27. Kearns, M., Kulesza, A. & Nevmyvaka, Y. Empirical limitations on highfrequency trading proftability. J. Trading 5, 50–62 (2010).

28. Wellman, M. P. & Rajan, U. Ethical issues for autonomous trading agents. Minds

Mach. 27, 609–624 (2017).

29. Farmer, J. D. & Skouras, S. An ecological perspective on the future of computer

trading. Quant. Finance 13, 325–346 (2013).

30. Perry, W. L., McInnis, B., Price, C. C., Smith, S. & Hollywood, J. S. Predictive

Policing: The Role of Crime Forecasting in Law Enforcement Operations (RAND,

2013).

31. Bonnefon, J.-F., Sharif, A. & Rahwan, I. The social dilemma of autonomous

vehicles. Science 352, 1573–1576 (2016).

32. Kooti, F. et al. Analyzing Uber’s ride-sharing economy. In Proc. 26th

International Conference on World Wide Web 574–582 (International World

Wide Web Conferences Steering Committee, 2017).

33. Zeng, X., Fapojuwo, A. O. & Davies, R. J. Design and performance evaluation of

voice activated wireless home devices. IEEE Trans. Consum. Electron. 52,

983–989 (2006).

34. Hendriks, B., Meerbeek, B., Boess, S., Pauws, S. & Sonneveld, M. Robot vacuum

cleaner personality and behavior. Int. J. Soc. Robot. 3, 187–195 (2011).

35. Hitsch, G. J., Hortaçsu, A. & Ariely, D. Matching and sorting in online dating.

Am. Econ. Rev. 100, 130–163 (2010).

36. Finkel, E. J., Eastwick, P. W., Karney, B. R., Reis, H. T. & Sprecher, S. Online

dating: a critical analysis from the perspective of psychological science.

Psychol. Sci. Public Interest 13, 3–66 (2012).

37. Park, H. W., Rosenberg-Kima, R., Rosenberg, M., Gordon, G. & Breazeal, C.

Growing growth mindset with a social robot peer. In Proc. 2017 ACM/IEEE

International Conference on Human–Robot Interaction 137–145 (ACM, 2017).

38. Bemelmans, R., Gelderblom, G. J., Jonker, P. & de Witte, L. Socially assistive

robots in elderly care: a systematic review into efects and efectiveness. J. Am.

Med. Dir. Assoc. 13, 114–120 (2012).

39. Shirado, H. & Christakis, N. A. Locally noisy autonomous agents improve global

human coordination in network experiments. Nature 545, 370–374 (2017).

In this human–machine hybrid study, the authors show that simple

algorithms injected into human gameplay can improve coordination

outcomes among humans.

40. Pichai, S. AI at Google: Our Principles. Google Blog https://blog.google/topics/

ai/ai-principles/ (2018).

41. Rof, H. M. The strategic robot problem: lethal autonomous weapons in war.

J. Mil. Ethics 13, 211–227 (2014).

42. Krishnan, A. Killer Robots: Legality and Ethicality of Autonomous Weapons

(Routledge, 2016).

43. Voosen, P. The AI detectives. Science 357, 22–27 (2017).

44. Szegedy, C. et al. Intriguing properties of neural networks. Preprint at https://

arxiv.org/abs/1312.6199 (2013).

45. Zhang, Q.-S. & Zhu, S.-C. Visual interpretability for deep learning: a survey.

Front. Inf. Technol. Electronic Eng. 19, 27–39 (2018).

46. Doshi-Velez, F. & Kim, B. Towards a rigorous science of interpretable machine

learning. Preprint at https://arxiv.org/abs/1702.08608(2017).

47. Gebru, T. et al. Datasheets for datasets. Preprint at https://arxiv.org/

abs/1803.09010 (2018).

48. Mitchell, M. et al. Model cards for model reporting. Preprint at https://

arxiv.org/abs/1810.03993 (2018).

49. Lakkaraju, H., Kamar, E., Caruana, R. & Horvitz, E. Identifying unknown

unknowns in the open world: representations and policies for guided

exploration. In Proc. 31st Association for the Advancement of Artifcial

Intelligence Conference on Artifcial Intelligence 2 (2017).

50. Johnson, N. et al. Abrupt rise of new machine ecology beyond human

response time. Sci. Rep. 3, 2627 (2013).

51. Appel, K., Haken, W. & Koch, J. Every planar map is four colorable. Part II:

reducibility. Illinois J. Math. 21, 491–567 (1977)

52. Appel, K. & Haken, W. Every planar map is four colorable. Part I: discharging.

Illinois J. Math. 21, 429–490 (1977).

53. Westlund, J. M. K., Park, H. W., Williams, R. & Breazeal, C. Measuring young

children’s long-term relationships with social robots. In Proc. 17th ACM

Conference on Interaction Design and Children 207–218 (ACM, 2018).

54. Lorenz, T., Weiss, A. & Hirche, S. Synchrony and reciprocity: key mechanisms

for social companion robots in therapy and care. Int. J. Soc. Robot. 8, 125–143

(2016).

55. Vosoughi, S., Roy, D. & Aral, S. The spread of true and false news online.

Science 359, 1146–1151 (2018).

This study examines the complex hybrid ecology of bots and humans on

Twitter and fnds that humans spread false information at higher rates

than bots.

56. Lazer, D. M. J. et al. The science of fake news. Science 359, 1094–1096 (2018).

57. Roberts, M. E. Censored: Distraction and Diversion Inside China’s Great Firewall

(Princeton Univ. Press, 2018).

58. Corbett-Davies, S., Pierson, E., Feller, A., Goel, S. & Huq, A. Algorithmic decision

making and the cost of fairness. In Proc. 23rd ACM SIGKDD International

Conference on Knowledge Discovery and Data Mining 797–806 (ACM, 2017).

59. Kleinberg, J., Mullainathan, S. & Raghavan, M. Inherent trade-ofs in the fair

determination of risk scores. Preprint at https://arxiv.org/abs/1609.05807

(2016).

60. Buolamwini, J. & Gebru, T. Gender shades: intersectional accuracy disparities

in commercial gender classifcation. In Proc. 1st Conference on Fairness,

Accountability and Transparency (eds Friedler, S. A. & Wilson, C.) 81, 77–91

(PMLR, 2018).

61. Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V. & Kalai, A. T. Man is to

computer programmer as woman is to homemaker? Debiasing word

embeddings. In Proc. Advances in Neural Information Processing Systems

4349–4357 (2016).

62. Caliskan, A., Bryson, J. J. & Narayanan, A. Semantics derived automatically

from language corpora contain human-like biases. Science 356, 183–186

(2017).

63. Sweeney, L. Discrimination in online ad delivery. Queueing Syst. 11, 10

(2013).

64. Ensign, D., Friedler, S. A., Neville, S., Scheidegger, C. & Venkatasubramanian, S.

Runaway feedback loops in predictive policing. Preprint at https://arxiv.org/

abs/1706.09847 (2017).

65. Angwin, J., Larson, J., Mattu, S. & Kirchner, L. Machine bias. ProPublica https://

www.propublica.org/article/machine-bias-risk-assessments-in-criminalsentencing(2016).

66. Chouldechova, A., Benavides-Prado, D., Fialko, O. & Vaithianathan, R. A case

study of algorithm-assisted decision making in child maltreatment hotline

screening decisions. In Proc. 1st Conference on Fairness, Accountability and

Transparency (eds Friedler, S. A. & Wilson, C.) 81, 134–148 (PMLR, 2018).

67. Jennings, N. R. et al. Human–agent collectives. Commun. ACM 57, 80–88

(2014).

68. Campbell, M., Hoane, A. J. & Hsu, F.-H. Deep blue. Artif. Intell. 134, 57–83

(2002).

69. Schaefer, J. et al. Checkers is solved. Science 317, 1518–1522 (2007).

70. Silver, D. et al. Mastering the game of Go with deep neural networks and tree

search. Nature 529, 484–489 (2016).

71. Silver, D. et al. Mastering the game of Go without human knowledge. Nature

550, 354–359 (2017).

72. Bowling, M., Burch, N., Johanson, M. & Tammelin, O. Heads-up limit hold’em

poker is solved. Science 347, 145–149 (2015).

73. Bellemare, M. G., Naddaf, Y., Veness, J. & Bowling, M. The arcade learning

environment: an evaluation platform for general agents. J. Artif. Intell. Res. 47,

253–279 (2013).

74. Wellman, M. P. et al. Designing the market game for a trading agent

competition. IEEE Internet Comput. 5, 43–51 (2001).

75. Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I. & Osawa, E. RoboCup: the robot

world cup initiative. In Proc. 1st International Conference on Autonomous Agents

340–347 (ACM, 1997).

76. Russakovsky, O. et al. ImageNet large scale visual recognition challenge. Int. J.

Comput. Vis. 115, 211–252 (2015).

77. Lin, T.-Y. et al. Microsoft COCO: common objects in context. In Proc. European

Conference on Computer Vision (eds Fleet, D. et al.) 8693, 740–755 (Springer

International Publishing, 2014).

78. Davis, J. & Goadrich, M. The relationship between precision–recall and ROC

curves. In Proc. 23rd International Conference on Machine Learning 233–240

(ACM, 2006).

79. van de Sande, K. E. A., Gevers, T. & Snoek, C. G. M. Evaluating color descriptors

for object and scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 32,

1582–1596 (2010).

80. Papineni, K., Roukos, S., Ward, T. & Zhu, W.-J. BLEU: a method for automatic

evaluation of machine translation. In Proc. 40th Annual Meeting on Association

for Computational Linguistics 311–318 (Association for Computational

Linguistics, 2002).

81. Zhou, Z., Zhang, W. & Wang, J. Inception score, label smoothing, gradient

vanishing and -log(D(x)) alternative. Preprint at https://arxiv.org/

abs/1708.01729 (2017).

82. Epstein, Z. et al. Closing the AI knowledge gap. Preprint at https://arxiv.org/

abs/1803.07233 (2018).

83. Tinbergen, N. On aims and methods of ethology. Ethology 20, 410–433 (1963).

84. Nesse, R. M. Tinbergen’s four questions, organized: a response to Bateson and

Laland. Trends Ecol. Evol. 28, 681–682 (2013).

85. Das, R., Hanson, J. E., Kephart, J. O. & Tesauro, G. Agent–human interactions in

the continuous double auction. In Proc. 17th International Joint Conference on

Artifcial Intelligence 1169–1178 (Lawrence Erlbaum, 2001).

86. Deng, Y., Bao, F., Kong, Y., Ren, Z. & Dai, Q. Deep direct reinforcement learning

for fnancial signal representation and trading. IEEE Trans. Neural Netw. Learn.

Syst. 28, 653–664 (2017).

87. Galceran, E., Cunningham, A. G., Eustice, R. M. & Olson, E. Multipolicy

decision-making for autonomous driving via changepoint-based behavior

prediction: theory and experiment. Auton. Robots 41, 1367–1382 (2017).

88. Ribeiro, M. T., Singh, S. & Guestrin, C. Why should I trust you? Explaining the

predictions of any classifer. In Proc. 22nd ACM SIGKDD International

Conference on Knowledge Discovery and Data Mining 1135–1144 (ACM, 2016).

89. Smilkov, D., Thorat, N., Kim, B., Viégas, F. & Wattenberg, M. SmoothGrad:

removing noise by adding noise. Preprint at https://arxiv.org/

abs/1706.03825 (2017).

90. Nevmyvaka, Y., Feng, Y. & Kearns, M. reinforcement learning for optimized

trade execution. In Proc. 23rd International Conference on Machine Learning

673–680 (ACM, 2006).

91. Nguyen, T. T., Hui, P.-M., Harper, F. M., Terveen, L. & Konstan, J. A. Exploring the

flter bubble: the efect of using recommender systems on content diversity. In

Proc. 23rd International Conference on World Wide Web 677–686 (ACM, 2014).

92. Dalvi, N. & Domingos, P. Mausam, Sanghai, S. & Verma, D. Adversarial

classifcation. In Proc. Tenth ACM SIGKDD International Conference on

Knowledge Discovery and Data Mining 99–108 (ACM, 2004).

93. Globerson, A. & Roweis, S. Nightmare at test time: robust learning by feature

deletion. In Proc. 23rd International Conference on Machine Learning 353–360

(ACM, 2006).

94. Biggio, B. et al. Evasion attacks against machine learning at test time. In Proc.

Joint European Conference on Machine Learning and Knowledge Discovery in

Databases 387–402 (Springer, 2013).

95. Tramèr, F. et al. Ensemble adversarial training: attacks and defenses. Preprint

at https://arxiv.org/abs/1705.07204(2017).

96. Parkes, D. C. & Wellman, M. P. Economic reasoning and artifcial intelligence.

Science 349, 267–272 (2015).

97. Wagner, A. Robustness and Evolvability in Living Systems (Princeton Univ. Press,

2013).

98. Edwards, H. & Storkey, A. Censoring representations with an adversary.

Preprint at https://arxiv.org/abs/1511.05897(2015).

99. Zemel, R., Wu, Y., Swersky, K., Pitassi, T. & Dwork, C. learning fair

representations. In Proc. International Conference on Machine Learning

325–333 (2013).

100. Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C. & Venkatasubramanian,

S. Certifying and removing disparate impact. In Proc. 21th ACM SIGKDD

International Conference on Knowledge Discovery and Data Mining 259–268

(ACM, 2015).

101. Cully, A., Clune, J., Tarapore, D. & Mouret, J.-B. Robots that can adapt like

animals. Nature 521, 503–507 (2015).

This study characterizes a robot driven by an adaptive algorithm that mimics

the adaptation and behaviours of animals.

102. Bongard, J., Zykov, V. & Lipson, H. Resilient machines through continuous

self-modeling. Science 314, 1118–1121 (2006).

103. Leibo, J. Z. et al. Psychlab: a psychology laboratory for deep reinforcement

learning agents. Preprint at https://arxiv.org/abs/1801.08116(2018).

In this study, the authors use behavioural tools from the life sciences in the

study of machine behaviours.

104. Subrahmanian, V. S. et al. The DARPA Twitter bot challenge. Preprint at

https://arxiv.org/abs/1601.05140(2016).

105. Carrascosa, J. M., Mikians, J., Cuevas, R., Erramilli, V. & Laoutaris, N. I. Always

feel like somebody’s watching me: measuring online behavioural advertising.

In Proc. 11th ACM Conference on Emerging Networking Experiments and

Technologies 13 (ACM, 2015).

106. Datta, A., Tschantz, M. C. & Datta, A. Automated Experiments on Ad Privacy

Settings. Proc. Privacy Enhancing Technologies 2015, 92–112 (2015).

107. Giusti, A. et al. A machine learning approach to visual perception of forest trails

for mobile robots. IEEE Robot. Autom. Lett. 1, 661–667 (2016).

108. Berdahl, A., Torney, C. J., Ioannou, C. C., Faria, J. J. & Couzin, I. D. Emergent

sensing of complex environments by mobile animal groups. Science 339,

574–576 (2013).

109. Couzin, I. D. et al. Uninformed individuals promote democratic consensus in

animal groups. Science 334, 1578–1580 (2011).

110. Rubenstein, M., Cornejo, A. & Nagpal, R. Programmable self-assembly in a

thousand-robot swarm. Science 345, 795–799 (2014).

111. Kernbach, S., Thenius, R., Kernbach, O. & Schmickl, T. Re-embodiment of

honeybee aggregation behavior in an artifcial micro-robotic system. Adapt.

Behav. 17, 237–259 (2009).

112. Bak, P., Chen, K. & Creutz, M. Self-organized criticality in the ‘Game of Life’.

Nature 342, 780–782 (1989).

113. Tsvetkova, M., García-Gavilanes, R., Floridi, L. & Yasseri, T. Even good bots fght:

the case of Wikipedia. PLoS ONE 12, e0171774 (2017).

114. Lazaridou, A., Peysakhovich, A. & Baroni, M. Multi-agent cooperation and the

emergence of (natural) language. Preprint at https://arxiv.org/

abs/1612.07182 (2016).

115. Budish, E., Cramton, P. & Shim, J. The high-frequency trading arms race:

frequent batch auctions as a market design response. Q. J. Econ. 130,

1547–1621 (2015).

116. Kirilenko, A. A. & Lo, A. W. Moore’s law versus Murphy’s law: algorithmic

trading and its discontents. J. Econ. Perspect. 27, 51–72 (2013).

117. Menkveld, A. J. The economics of high-frequency trading: taking stock. Annu.

Rev. Financ. Econ. 8, 1–24 (2016).

118. Mønsted, B., Sapieżyński, P., Ferrara, E. & Lehmann, S. Evidence of complex

contagion of information in social media: an experiment using Twitter bots.

PLoS ONE 12, e0184148 (2017).

This study presents an experimental intervention on Twitter using bots and

provides evidence that information difusion is most accurately described by

complex contagion.

119. Bainbridge, L. Ironies of automation. Automatica 19, 775–779 (1983).

120. Jeong, S., Breazeal, C., Logan, D. & Weinstock, P. Huggable: the impact of

embodiment on promoting socio-emotional interactions for young pediatric

inpatients. In Proc. 2018 CHI Conference on Human Factors in Computing

Systems 495 (ACM, 2018).

121. Kory Westlund, J. M. et al. Flat vs. expressive storytelling: young children’s

learning and retention of a social robot’s narrative. Front. Hum. Neurosci. 11,

295 (2017).

122. Salisbury, E., Kamar, E. & Morris, M. R. Toward scalable social alt text:

conversational crowdsourcing as a tool for refning vision-to-language

technology for the blind. Proc. 5th AAAI Conference on Human Computation and

Crowdsourcing (2017).

123. Awad, E. et al. The Moral Machine experiment. Nature 563, 59–64 (2018).

124. Dietvorst, B. J., Simmons, J. P. & Massey, C. Algorithm aversion: people

erroneously avoid algorithms after seeing them err. J. Exp. Psychol. Gen. 144,

114–126 (2015).

125. Gray, K. & Wegner, D. M. Feeling robots and human zombies: mind perception

and the uncanny valley. Cognition 125, 125–130 (2012).

126. Brynjolfsson, E. & Mitchell, T. What can machine learning do? Workforce

implications. Science 358, 1530–1534 (2017).

127. Christiano, P. F. et al. Deep reinforcement learning from human preferences. In

Proc. Advances in Neural Information Processing Systems 30 (eds Guyon, I. et

al.) 4299–4307 (Curran Associates, 2017).

128. Tsvetkova, M. et al. Understanding human–machine networks: a crossdisciplinary survey. ACM Comput. Surv. 50, 12:1–12:35 (2017).

129. Hilbert, M., Ahmed, S., Cho, J., Liu, B. & Luu, J. Communicating with

algorithms: a transfer entropy analysis of emotions-based escapes from

online echo chambers. Commun. Methods Meas. 12, 260–275 (2018).

130. Kramer, A. D. I., Guillory, J. E. & Hancock, J. T. Experimental evidence of

massive-scale emotional contagion through social networks. Proc. Natl Acad.

Sci. USA 111, 8788–8790 (2014).

131. Kamar, E., Hacker, S. & Horvitz, E. Combining human and machine intelligence

in large-scale crowdsourcing. in Proc. 11th International Conference on

Autonomous Agents and Multiagent Systems 467–474 (International

Foundation for Autonomous Agents and Multiagent Systems, 2012).

132. Jackson, M. The Human Network: How Your Social Position Determines Your

Power, Beliefs, and Behaviors (Knopf Doubleday, 2019).

133. Crandall, J. W. et al. Cooperating with machines. Nat. Commun. 9, 233 (2018).

This study examines algorithmic cooperation with humans and provides an

example of methods that can be used to study the behaviour of human–

machine hybrid systems.

134. Wang, D., Khosla, A., Gargeya, R., Irshad, H. & Beck, A. H. Deep learning for

identifying metastatic breast cancer. Preprint at https://arxiv.org/

abs/1606.05718 (2016).

135. Pentland, A. Social Physics: How Social Networks Can Make Us Smarter

(Penguin, 2015).

136. Lazer, D. et al. Computational social science. Science 323, 721–723 (2009).

137. Aharony, N., Pan, W., Ip, C., Khayal, I. & Pentland, A. Social fMRI: investigating

and shaping social mechanisms in the real world. Pervasive Mobile Comput. 7,

643–659 (2011).

138. Ledford, H. How to solve the world’s biggest problems. Nature 525, 308–311

(2015).

139. Bromham, L., Dinnage, R. & Hua, X. Interdisciplinary research has consistently

lower funding success. Nature 534, 684–687 (2016).

140. Kleinberg, J. & Oren, S. Mechanisms for (mis)allocating scientifc credit.

In Proc. 43rd Annual ACM Symposium on Theory of Computing 529–538

(ACM, 2011).

141. Kannel, W. B. & McGee, D. L. Diabetes and cardiovascular disease. The

Framingham study. J. Am. Med. Assoc. 241, 2035–2038 (1979).

142. Kraft, P. M., Macy, M. & Pentland, A. Bots as virtual confederates: design and

ethics. In Proc. 2017 ACM Conference on Computer Supported Cooperative Work

and Social Computing 183–190 (ACM, 2017).

143. Meyer, M. N. Two cheers for corporate experimentation: The A/B illusion

and the virtues of data-driven innovation. Colorado Technol. Law J. 13, 273

(2015).

144. Xing, X. et al. Take this personally: pollution attacks on personalized services.

In Proc. 22nd USENIX Security Symposium 671–686 (2013).

145. Patel, K. Testing the limits of the First Amendment: how a CFAA prohibition on

online antidiscrimination testing infringes on protected speech activity.

Columbia Law Rev. https://doi.org/10.2139/ssrn.3046847(2017).