Professor Jacek Mańdziuk: Despite being very effective and outperforming the best human players, artificial intelligence playing Go is not the ultimate achievement in abstract strategy games
Dynamically developing artificial intelligence has found its way into games. The most noteworthy example of progress in the application of AI in games includes the achievements of Google DeepMind in Go, a game which for decades has remained a bastion of supremacy of people over machines in the field of classic abstract games.
While the earlier achievement of IBM (Deep Blue computer machine) was only a psychological breakthrough (that phenomenal engineering solution did not offer any groundbreaking solutions to artificial intelligence methods), the AlphaGo system by Google DeepMind, and, to an even greater extent, AlphaGo Zero, have proven that AI methods (deep learning, reinforcement learning and Monte Carlo simulation) may be used on a large scale to solve issues that until recently remained beyond the reach of artificial intelligent systems.
Despite being very effective and outperforming the best human players, artificial intelligence playing Go is not the ultimate achievement in abstract strategy games. It forms a basis for further considerations on the use of AI and machine learning methods in matters of a more universal nature.
Research and challenges
Until now, research in the field of games was mostly concentrated on improving the play level of artificial players. There was no focus on how that high level was achieved. The main goal was to improve playing skills in order to beat the best (human) players in the world and, after that happened, to compete with other machine players for the championship title.
Loss of doctor Marion Tinsley [Editor’s note: American and best checkers player in history] to the Chinook program (checkers), followed by the solution of that game (i.e. prediction of the outcome of a play, assuming both players play perfectly) by a group of researchers from the University of Alberta led by professor Jonathan Schaeffer; Deep Blue’s victory over Garry Kasparov, followed by the launch of programs with player skill level exceeding 3,000 ELO on a single-processor PC (Komodo, Houdini, Stockfish, Deep Fritz etc.), or defeat of Lee Sedola in the clash with AlphaGo – all that has led to a reassessment of priorities in classic two-player games with full information.
Most notably, researchers renewed their interest in the paradigm of human-like playing (playing in a similar way to people), especially in the aspect of multigame playing (ability to play many games, usually within a certain class of games).
Poland: three groups
There are three scientific groups at Polish universities that conduct research in the domain of games.
Group led by professor Krzysztof Krawiec from Poznan University of Technology
Works of the research team led by professor Krawiec (collaborators: Wojciech Jaśkowski, PhD; Paweł Liskowski, PhD; Marcin Szubert, PhD; Bartosz Wieloch, PhD) in the field of games are focused on learning game strategies with the use of evolutionary algorithms, co-evolutionary algorithms, reinforcement learning and, recently, neural networks, as well as hybridization of the above approaches.
In their works, different representations of strategies/players and different learning algorithms (i.a. single-population and dual-population co-evolutionary algorithms, temporal difference learning, gradient methods) are used. The works have resulted in development of strategies which got very high in the rankings for such games as Othello (Reversi) and SZ-Tetris, often staying in the lead.
Until now, research in the field of games was mostly concentrated on improving the play level of artificial players. There was no focus on how that high level was achieved
Furthermore, the team led by professor Krawiec also conducted research on theoretical aspects of games, e.g. on precise and approximate methods of estimating games’ dimensionality and on the role of shaping in co evolutionary learning.
Group led by professor Dominik Ślęzak from Warsaw University
As far as games are concerned, two main areas, on which research is focused, include development of tools supporting the implementation of AI mechanisms in video games (Grail project carried out in cooperation with Silver Bullet) and construction of a coaching system supporting the improvement of individual skills of players in eSport games (SENSEI project carried out in cooperation with eSensei).
Both projects are co-financed by the National Center for Research and Development (within the Smart Growth Operational Program (SGOP) – GameINN competition) and are very good examples of harmonious combination of basic research and implementation actions.
Until now, research in the field of games was mostly concentrated on improving the level of artificial players. There was no focus on how that high level was achieved.
From the methodological perspective, the focus of both of those projects is on various incarnations of the MCTS/UCT method in combination with granular computing carried out at pre-selected levels of information granulation. Proposed solutions are chiefly designed and implemented by professor Ślęzak and by my former doctoral student Maciej Świechowski, PhD.
Research group at the Faculty of Mathematics and Information Sciences; Warsaw University of Technology
The main areas of scientific research of my of artificial intelligence and computational intelligence team consisting of 12 people (one full professor, one Doctor of Science, three Doctors of Philosophy, one research assistant, six doctoral students) include:
• artificial intelligence methods, machine learning and artificial neural networks;
• metaheuristic population methods (genetic algorithms, mathematical methods, particle swarm optimization algorithms and ant colony optimization algorithms).
The team conducts both basic research pertaining to theoretical characteristics of the above metaheuristic methods and application research using the above methods to solve practical problems in many application fields.
In particular, the research is focused on:
• using artificial general intelligence (AGI) methods in dynamic multiagent environments (e.g. synchronized multiplayer games);
• developing effective (efficient and scalable) methods of finding Stackelberg’s equilibrium in multistep games (Stackelberg’s defense games);
• applications of self-adapting metaheuristic algorithms to solve, for example, difficult optimization problems (mainly dynamic transport problems);
• application of knowledge-free learning in general game playing;
• effective knowledge representation methods in autonomous systems.
Over the past several years the team has been working on extensive development of research in the field of cognitively inspired methods of knowledge acquisition and representation in intelligent systems as well as autonomous learning methods that do not require any external trainers.
The above issues are covered in my monograph entitled “Knowledge-Free and Learning-Based Methods in Intelligent Game Playing”, published by Springer-Verlag in 2010; they were also discussed at IEEE annual international scientific symposiums on Computational Intelligence for Human-like Intelligence (2013 – Singapore, 2014 Orlando/USA, 2015 – Cape Town/RSA, 2016 – Athens, 2017 – Honolulu/USA, 2018 – Bangalore/India), which I organized together with Włodzisław Duch and Janusz Starzyk (Ohio University, USA).
Our research is financed by both Warsaw University of Technology (statutory and dean’s grants) and other third-party organizations and institutions (grant awarded by National Science Center (2018-2019), grant awarded by National Science Center (2013-2016), grant awarded by Foundation for Polish Science within the international PhD projects (2010-2015)). Research is conducted in collaboration with scientists from renowned research centers, e.g. Nanyang Technological University in Singapore, System Research Institute of the Polish Academy of Sciences, University of Alberta in Canada, Nicolaus Copernicus University in Toruń, University of New South Wales in Australia and National University of Tainan, Taiwan.
Research on games is conducted not only by the abovementioned teams but also by individual researchers from other centers, namely Piotr Beling, PhD (University of Lodz), who focuses on research pertaining to bridge AI-supported process of defining and developing games (Grail project), and Jakub Kowalski, PhD (University of Wrocław), who has been developing ideas about General Game Playing in collaboration with professor Andrzej Kisielewicz.
Three research currents
In recent years the domain of games has been developing very dynamically, mainly due to promotion of new search methods (Monte Carlo Tree Search/UCT) as well as analysis and classification of images (deep neural networks). At the same time, with gaming skills of artificial agents being greatly improved, there has been a gradual shift of emphasis from further improvement of games to the issue of training usability of AI agents.
• Particular focus is given to development of the coaching approach consisting in the most effective implementation of the learning process implemented by the AI agent in the context of achieving an intended learning goal by a human – observer, e.g. playing a given game, solving a specific type of problems or effectively dealing with specific situations).
• The second key research current involves the implementation of the paradigm of human like playing and problem solving based on the idea of imitating human cognitive skills and their application in the game (speaking more generally: when solving decision making problems). This area also covers the issue of believability of a playing bot, considered among others in the context of the game bot Turing test.
• The third dynamically developing research current is human-machine co-learning and problem solving, i.e. the problem of joint, synergistic problem solving (often in the form of an iteration loop: human-in-the-loop) and an approach to the learning / problem solving process that would make it possible for a human to learn effectively, with an AI agent (robot) as a partner – teacher.
As a matter of fact, all research areas related to games, including, of course, the abovementioned key directions, lead to theoretical considerations and practical solutions that may be applicable not only in the domain in which they are generated but also in a wide variety of other fields. A vast majority of methods and algorithms related to games are of universal importance in the context of solving decision-making problems and building operational strategies in a long-term perspective.
A great majority of ideas and algorithms developed in the course of research conducted in the domain of games can be generically applied in many other areas/decision-making problems. Therefore, such ideas, algorithms and metamethods mean much more than just optimization of the behavior of AI players.
The future: three recommendations
What are the prospects for AI development in Poland? In my opinion, there are three absolutely fundamental aspects of education and scientific policy that need to be significantly modified and strengthened:
• Enhancing the level of internationalization of Polish science through institutional and financial support for cooperation with foreign entities and for sharing human resources with the best AI centers in the world.
• Establishing dedicated doctoral schools covering several scientific institutions with the requirement to have a co-mentor/co-supervisor from abroad (this recommendation would naturally support scientific exchange of doctoral students).
• Promoting, in a special way and within parametric assessment of scientific units, publications in the most prestigious periodicals and at scientific conferences. If we want to be among the best, we should definitely rely on the quality rather than quantity of publications and on their real (and not only formal) impact on global science.
In my opinion, the above recommendations are the bare minimum of necessary actions Polish science has to take to follow global/European leaders in the domain of artificial intelligence. The exhaustive list of such actions would obviously be much longer.
The article is an adaptation of the paper by professor Jacek Mańdziuk entitled “Development of artificial intelligence and machine learning in the area of games” (Warsaw 2018) prepared for the National Information Processing Institute.