Trimming the ML Bot:⋆ A Study on the Effects of

Code Simplification on AI Efficiency in Schnapsen

Anonymous Submission

Vrije Universiteit (VU), De Boelelaan 1105, 1081 HV Amsterdam

Abstract. This research extends the investigation of a modified machine

learning bot designed for the card game Schnapsen, focusing on practical
implementation and performance evaluation within a Python program-
ming environment. The modified bot, optimized through strategic feature
sets reduction, is assessed alongside the original machine learning bot to
determine its efficacy in terms of similar game outcomes and computa-
tional efficiency. The study leverages a Python-based framework to execute
and evaluate the bots in a controlled environment. The Python program
facilitates consistent testing conditions and enables measurement of key
performance metrics, such as decision-making speed, memory utilization,
and overall computational efficiency. This empirical approach emphasizes
the importance of seamless integration into practical applications. Results
obtained from extensive Python-based simulations are analyzed to assess
the modified bot’s performance and the performance of the original bot
against another opponent. The findings not only validate the effectiveness
of the feature reduction strategy in maintaining competitive gameplay
outcomes but also demonstrate the practicality of the modified bot in
terms of computational resource requirements.This research contributes
valuable insights into the implementation of machine learning bots for
gaming applications, in this case Schnapsen, emphasizing the significance
of testing within a Python environment. The Python-based evaluation
provides a tangible and accessible framework for assessing the practical
implications of the modified bot.

Keywords: Schnapsen · Machine learning · Code Simplification.

1 Introduction

1.1 Problem Statement

The need for processing capacity is a major barrier to the widespread use of
AI-driven bots in the context of machine learning-driven games. This problem
is noticeable in games like Schnapsen, where difficult decision making processes
lead to large computing difficulties.(Salzer, n.d.) It is interesting to see what
difficulties the application of a machine learning bot could get in this game, which
has incomplete information.(Application of Machine Learning in Games With
⋆
Intelligent Systems 2023
2 Anonymous Submission

Incomplete Information, 2021) Even though it works well, the present Schnapsen
machine learning bot needs a lot of processing power to function at its best.
Solving this issue is important for improving the usability and accessibility of
these kinds of bots in a range of computer contexts.

1.2 Motivation

This research focuses on finding a balance between the limitations of computer

resources and the efficiency of ML bots in games like Schnapsen. Our main goal is
to keep the same functionality but less computational thinking. This way, the bot
remains the same, but the computation time declines. This is because parts of the
code are removed.(Winning Is Not Everything: Enhancing Game Development
With Intelligent Agents, 2020) In addition, the research question was proposed
by a TA, and we found it to be an excellent idea. We thoroughly examined
and implemented it by conducting our own investigation into what elements to
remove and what to retain. Moreover, as AI in gaming develops further, it will
become more and more necessary to improve ML models for useful, real-world
applications. The results of this study might help clarify how simplifying, by
removing lines of code, can improve MLBots performance. This could make it
easier for these bots to blend in with different gaming configurations that have
various levels of processing power. The main goal of the research is to increase
the sustainability and adaptability of AI-driven gaming solutions, especially in
certain scenarios with constrained computer resources.(Lee and Kim, 2016)

1.3 Research Question

In our research we look at the following question: Can simplifying the feature set
of the Schnapsen bot’s machine learning model, by removing certain features,
maintain comparable win percentages while enhancing computational efficiency?

1.4 Hypothesis

We believe that by making the ML bot for playing Schnapsen easier and more
focused, it could perform just as well as the current version, but will use less
computation power. Our idea is to identify and remove parts of the bot that do not
really help it make better decisions. This should make the bot’s decision-making
process more efficient, allowing it to play the game just as effectively without
needing as much computer power. We also think that by getting rid of unnecessary
parts, the bot’s decisions will be easier to understand. This simplification not
only reduces the work the computer has to do but also helps the bot recognize
patterns better, possibly making it even better at playing the game. Overall, our
goal is to show that we can make the bot more efficient without sacrificing its
ability to play well, making it more practical for use in gaming situations where
computer resources are limited.
 Title Suppressed Due to Excessive Length 3

1.5 Approach

We apply the following steps to address our research question: 1. We look for
literature research about our question 2. We compare the results of our two bots
and their computation power. 3. We evaluate the results of our experiments into
the conclusion.

2 Experimental Setup

2.1 Overview

The purpose of the experimental setting was to analyze how code pruning affected
the MLBot’s performance when playing Schnapsen.(Zhu, 2017) The main goal
was to eliminate unnecessary segments of code so that we could compare the
modified bot to the original ML bot.(Minimizing Computational Overhead While
Retaining Gameplay Effectiveness in Starcraft Bots, 2022) In an effort to measure
the effects of code pruning, 20000 games against the RandBot were played in
total, 10000 with the pruned ML bot and 10000 with the original ML bot being
compared. We ran through assignment 11 of project intelligent system twice and
modified task 6 so both bots can play the 10000 games against Randbot one
after another. In the first run we let MLBot play against Randbot and in the
second run we let the modified MLBot play against Randbot.

2.2 Bots

ML Bot Modified (Pruned) The modified machine learning bot, also known
as the pruned version, was made by removing code parts that we considered
unnecessary or did not significantly affect the function of the bot. Pruning focused
on parts of the machine learning bot’s code that were found by studying how the
algorithm operated.(Zhu, 2017)
Here’s a detailed breakdown of the lines we removed:

1. Removed unnecessary comments and docstrings for brevity.

2. Removed the MLDataBot class’s docstring for brevity.
3. Removed replay memory file path from the MLDataBot class’s initialization
because it wasn’t used.
4. Removed the notify game start method from the MLDataBot class as it
wasn’t used.
5. Removed the MLDataBot class’s notify game start and notify trick end
methods for brevity.
6. Removed the train ML model function’s overwrite parameter, as it was not
used.
7. Removed the train ML model function’s from memory file and to memory
file parameters, as they were not used.
8. Removed the train ML model function’s unused import statement for random.
4 Anonymous Submission

9. Removed the train ML model function’s redundant line for converting data
and targets to NumPy arrays.
10. Removed the train ML model function’s unnecessary type hints for function
parameters.
11. Removed the train ML model function’s model class check for ’LR’, as it was
unnecessary.
12. Removed the train ML model function’s unused train indices and test indices
variables.
13. Removed the train ML model function’s unused classification report import.
14. Removed the train ML model function’s unused accuracy and precision
variables.
15. Removed the train ML model function’s unnecessary type parameter when
loading the model.

Our overall goal was to keep the essential functionality of the code while
removing unnecessary comments, docstrings, and code segments that weren’t
contributing to the main process. (A Survey of Feature Selection and Feature
Extraction Techniques in Machine Learning, 2014)

2.3 ML Bot

As the control group, the original machine learning bot stayed the same and
provided a standard for comparison. This version did not include any pruning
and contained the whole code.

2.4 RandBot

For both versions of the ML bot, the Randbot was chosen as the opponent. In
order to compare the performance of the two ML bot types, an identical and
randomized gameplay approach was provided by the RandBot, which functioned
as a standard baseline.

3 Experimental Procedure

The experiment consisted of two primary conditions: Modified ML Bot vs. Rand-
Bot: The ML bot, after it was pruned, played the RandBot in a 10000 game series.
The purpose of this condition was to evaluate the impact of code trimming on
the ML bot’s performance against a randomly selected opponent in a particular
matchup. Original ML Bot vs. RandBot: In a different set of 10000 games, the
original ML bot played against RandBot. By using this condition as the control
group, it was possible to compare the performance of the modified ML bot to
that of the original ML bot under the same gaming settings. The win/loss ratio
served as the main performance measurement. We looked at the win/loss ratio
to measure each bot’s relative performance in the chosen games by keeping track
of the games it won and lost. We also used Randomization in the experiment
 Title Suppressed Due to Excessive Length 5

to minimize any potential bias caused by the order of the games or external
factors. To maintain statistical stability, the 20000 games’ starting positions, play
order, and other game-related features were randomized and not fixed. All game
results, including the winner of each match, were recorded for more analysis. The
gathered information served as a basis for comparing the performance of the
original and modified ML bot to the Randbot.

4 Findings

4.1 Results and Analysis

The results of the experiment indicate that the modified ML bot maintained
a comparable win/loss ratio when playing against the RandBot. The win per-
centages for both the modified and original ML bots were almost the identical,
suggesting that the feature reduction strategy did not significantly impact the
ML bot’s ability to win games. Furthermore, we can say that the computational
efficiency of the modified ML bot was improved. We saw a difference in execution
time. The modified ML bot was slightly faster done with the 10000 games then
the original ML bot. (Preuveneers et al., 2020) The win/loss ratio for both the
modified and original ML bots against the RandBot was consistently high, with
no significant difference between the two bots. We can say that the pruning was
successfully executed, because there is no significant difference in win/loss ratio
between the two bots. The removal of unnecessary docstrings, comments and
parts of code allowed the modified bot to achieve these comparable outcomes
with less computational time. The simplification of the bot did not compromise
its overall performance in Schnapsen.

Fig. 1. Results of the ML Bot vs the RandBot.

6 Anonymous Submission

Fig. 2. Results of the modified ML Bot vs the RandBot.

5 Conclusion

The study’s findings support the hypothesis that simplifying the feature set of
the Schnapsen bot’s machine learning model, through strategic code pruning, can
maintain comparable win percentages while enhancing computational efficiency.
The modified ML bot, with reduced code complexity, demonstrated equivalent
performance to the original ML bot in terms of game outcomes against the
RandBot. The implications of this research extend to the broader field of AI-
driven gaming solutions, emphasizing the importance of optimizing machine
learning models for practical applications with limited computational resources.
The study contributes to the ongoing efforts to make AI-driven bots more
accessible, adaptable, and sustainable in various gaming environments.

6 Future Work

6.1 Real-World Implementation and Practical Advantages

The exploration of future directions is essential for extending and building

upon the contributions presented in this paper. So is the performance against
different opponents not tested. In-depth investigation into the modified ML bot’s
performance against a more diverse array of opponents is warranted. This includes
opponents characterized by various playstyles, such as aggressive, defensive, or
adaptive strategies. Methodologically, understanding how the bot adapts to
distinct opponent profiles, identifying specific challenges posed by each playstyle,
and assessing the transferability of learned strategies between opponents will form
the crux of this follow-up study. Also the generalizability to other ML-Driven
games is important. The current study focuses on Schnapsen, but future research
should look into the generalizability of code pruning strategies to more ML-driven
games. Investigating the effectiveness of the approach across different game types
will provide insights into the robustness and adaptability of code simplification
strategies. This could involve evaluating commonalities and differences in the
 Title Suppressed Due to Excessive Length 7

impact of feature reduction on diverse gaming environments. We only looked at

the win/loss ratio and focused on computational efficiency , but future research in
exploration of additional metrics should provide a more comprehensive evaluation
of the performance of the modified ML bot. (A Survey of Feature Selection and
Feature Extraction Techniques in Machine Learning, 2014) A crucial next step is
the practical implementation of the optimized bot in real-world gaming scenarios.
Evaluating its flexibility and benefits in particular gaming contexts will shed light
on how broadly applicable our findings are. This can entail assessing the way the
modified bot communicates with human players, possible difficulties in real-world
gaming environments, and the effects on user happiness and experience.

References
1. Zhu, M. (2017, October 5). To prune, or not to prune: exploring the efficacy of
pruning for model compression. arXiv.org. https://arxiv.org/abs/1710.01878
2. Preuveneers, D., Tsingenopoulos, I., and Joosen, W. (2020). Resource usage and
performance trade-offs for machine learning models in smart environments. Sensors,
20(4), 1176. https://doi.org/10.3390/s20041176
3. A survey of feature selection and feature extraction techniques in machine
learning. (2014, August 1). IEEE Conference Publication — IEEE Xplore.
https://ieeexplore.ieee.org/abstract/document/6918213
4. Application of Machine Learning in Games with incomplete informa-
tion. (2021, December 1). VDE Conference Publication — IEEE Xplore.
https://ieeexplore.ieee.org/abstract/document/9736720
5. Salzer, T. (n.d.). Reinforcement learning for games with imperfect infor-
mation – Teaching an agent the game of Schnapsen. repositum.tuwien.at.
https://doi.org/10.34726/hss.2023.86961
6. Lee, G. S., and Kim, I. (2016). A study on Simplification of Machine learning model.
The Journal of the Institute of Internet, Broadcasting and Communication, 16(4),
147–152. https://doi.org/10.7236/jiibc.2016.16.4.147
7. Minimizing Computational Overhead while Retaining Gameplay Effectiveness in
Starcraft Bots. (2022, December 1). IEEE Conference Publication — IEEE Xplore.
https://ieeexplore.ieee.org/abstract/document/10216577
8. Winning Is Not Everything: Enhancing Game Development With Intelli-
gent Agents. (2020, June 1). IEEE Journals and Magazine — IEEE Xplore.
https://ieeexplore.ieee.org/abstract/document/9104019

7 Appendices
7.1 Appendix A
This is the code used for the ML Bot:

Listing 1.1. ML Bot code Python-code

from t y p i n g import O p t i o n a l , c a s t , L i t e r a l
import random
8 Anonymous Submission

import c l i c k
import t i m e
import p a t h l i b

import j o b l i b
from s k l e a r n . n e u r a l n e t w o r k import M L P C l a s s i f i e r
from s k l e a r n . l i n e a r m o d e l import L o g i s t i c R e g r e s s i o n

from s c h n a p s e n . d e c k import S u i t , Rank

from s c h n a p s e n . game import (
Bot ,
PlayerPerspective ,
SchnapsenDeckGenerator ,
Move ,
Trick ,
GamePhase ,
GamePlayEngine ,
SchnapsenGamePlayEngine ,
TrumpExchange ,
)
from s c h n a p s e n . a l t e r n a t i v e e n g i n e s . a c e o n e e n g i n e import AceOneGamePlayEngine
from s c h n a p s e n . b o t s import RandBot , RdeepBot , MiniMaxBot , AlphaBetaBot
from s c h n a p s e n . b o t s . e x a m p l e b o t import ExampleBot
from s c h n a p s e n . a l t e r n a t i v e e n g i n e s . t w e n t y f o u r c a r d s c h n a p s e n import (
TwentyFourSchnapsenGamePlayEngine ,
)

c l a s s MLPlayingBot ( Bot ) :
”””
T h i s c l a s s l o a d s a t r a i n e d ML model and u s e s i t t o p l a y
”””

def init (
s e l f , m o d e l l o c a t i o n : p a t h l i b . Path , name : O p t i o n a l [ s t r ] = None
) −> None :
”””
C r e a t e a new MLPlayingBot w h i c h u s e s t h e model s t o r e d i n t h e m o f e l l o c a

: param m o d e l l o c a t i o n : The f i l e c o n t a i n i n g t h e model .

”””
super ( ) . i n i t ( name )
model location = model location
a s s e r t m o d e l l o c a t i o n . e x i s t s ( ) , f ” Model c o u l d n o t b e f o u n d a t : { m o d e l
 Title Suppressed Due to Excessive Length 9

# l o a d model
s e l f . model = j o b l i b . load ( model location )

def g e t m o v e (
s e l f , p e r s p e c t i v e : P l a y e r P e r s p e c t i v e , l e a d e r m o v e : O p t i o n a l [ Move ]
) −> Move :
# get the sate feature representation
state representation = get state feature vector ( perspective )
# g e t t h e l e a d e r ’ s move r e p r e s e n t a t i o n , e v e n i f i t i s None
leader move representation = get move feature vector ( leader move )
# g e t a l l my v a l i d moves
my valid moves = p e r s p e c t i v e . valid moves ()
# g e t t h e f e a t u r e r e p r e s e n t a t i o n s f o r a l l my v a l i d moves
my move representations : l i s t [ l i s t [ int ] ] = [ ]
for my move in m y v a l i d m o v e s :
m y m o v e r e p r e s e n t a t i o n s . append ( g e t m o v e f e a t u r e v e c t o r ( my move ) )

# c r e a t e a l l model i n p u t s , f o r a l l b o t ’ s v a l i d moves
a c t i o n s t a t e r e p r e s e n t a t i o n s : l i s t [ l i s t [ int ] ] = [ ]

if perspective . am i leader ():

f o l l o w e r m o v e r e p r e s e n t a t i o n = g e t m o v e f e a t u r e v e c t o r ( None )
for m y m o v e r e p r e s e n t a t i o n in m y m o v e r e p r e s e n t a t i o n s :
a c t i o n s t a t e r e p r e s e n t a t i o n s . append (
state representation
+ my move representation
+ follower move representation
)
else :
for m y m o v e r e p r e s e n t a t i o n in m y m o v e r e p r e s e n t a t i o n s :
a c t i o n s t a t e r e p r e s e n t a t i o n s . append (
state representation
+ leader move representation
+ my move representation
)

model output = s e l f . model . predict proba ( a c t i o n s t a t e r e p r e s e n t a t i o n s

winning probabilities of moves = [
o u t c o m e p r o b [ 1 ] for o u t c o m e p r o b in m o d e l o u t p u t
]
h i g h e s t v a l u e : f l o a t = −1
b e s t m o v e : Move
for i n d e x , v a l u e in enumerate ( w i n n i n g p r o b a b i l i t i e s o f m o v e s ) :
if value > highest value :
highest value = value
10 Anonymous Submission

best move = my valid moves [ index ]

a s s e r t b e s t m o v e i s not None
return b e s t m o v e

c l a s s MLDataBot ( Bot ) :
”””
This c l a s s i s d e f i n e d to a l l o w t h e c r e a t i o n o f a t r a i n i n g schnapsen b o t d a
P r a c t i c a l l y , i t h e l p s us r e c o r d how t h e game p l a y s o u t a c c o r d i n g t o a p r o v
In more d e t a i l , we c r e a t e one t r a i n i n g s a m p l e f o r e a c h d e c i s i o n t h e b o t ma
Then we r e l a t e e a c h d e c i s i o n w i t h t h e outcome o f t h e game , i . e . w h e t h e r t h
T h i s way we can t h e n t r a i n a b o t a c c o r d i n g t o t h e a s s u m p t i o n t h a t :
” d e c i s i o n s i n e a r l i e r games t h a t ended up i n v i c t o r i e s s h o u l d b e p r e f e r
T h i s c l a s s o n l y r e c o r d s t h e d e c i s i o n s and game outcomes o f t h e p r o v i d e d b o
”””

def i n i t ( s e l f , b o t : Bot , r e p l a y m e m o r y l o c a t i o n : p a t h l i b . Path ) −> None

”””
: param b o t : t h e p r o v i d e d b o t t h a t w i l l a c t u a l l y p l a y t h e game and make
: param r e p l a y m e m o r y l o c a t i o n : t h e f i l e n a m e under w h i c h t h e r e p l a y mem
”””

s e l f . b o t : Bot = b o t
s e l f . r e p l a y m e m o r y f i l e p a t h : p a t h l i b . Path = r e p l a y m e m o r y l o c a t i o n

def g e t m o v e (
s e l f , p e r s p e c t i v e : P l a y e r P e r s p e c t i v e , l e a d e r m o v e : O p t i o n a l [ Move ]
) −> Move :
”””
This f u n c t i o n simply c a l l s t h e get move o f t h e p r o v i d e d b o t
”””
return s e l f . b o t . g e t m o v e ( p e r s p e c t i v e=p e r s p e c t i v e , l e a d e r m o v e=l e a d e r m o

def n o t i f y g a m e e n d ( s e l f , won : bool , p e r s p e c t i v e : P l a y e r P e r s p e c t i v e ) −> No

”””
When t h e game ends , t h i s f u n c t i o n r e t r i e v e s t h e game h i s t o r y and more
b e d e r i v e d from i t , and s t o r e s them i n t h e form o f s t a t e −a c t i o n s v e c t o r

: param won : Did t h i s b o t win t h e game?

: param s t a t e : The f i n a l s t a t e o f t h e game .
”””
# we r e t r i e v e t h e game h i s t o r y w h i l e a c t u a l l y d i s c a r d i n g t h e l a s t u s e l e
# we know none o f t h e T r i c k s can b e None b e c a u s e t h a t i s o n l y f o r t h e
game history : l i s t [ tuple [ P l a y e r P e r s p e c t i v e , Trick ] ] = c a s t (
l i s t [ tuple [ P l a y e r P e r s p e c t i v e , Trick ] ] , p e r s p e c t i v e . g e t g a m e h i s t o r y
 Title Suppressed Due to Excessive Length 11

)
# we a l s o s a v e t h e t r a i n i n g l a b e l ”won or l o s t ”
w o n l a b e l = won

# we i t e r a t e o v e r a l l t h e r o u n d s o f t h e game
for r o u n d p l a y e r p e r s p e c t i v e , r o u n d t r i c k in g a m e h i s t o r y :
if round trick . is trump exchange () :
leader move = r o u n d t r i c k . exchange
f o l l o w e r m o v e = None
else :
leader move = round trick . leader move
follower move = round trick . follower move

# we do n o t want t h i s r e p r e s e n t a t i o n t o i n c l u d e a c t i o n s t h a t f o l l o w
if round player perspective . am i leader ():
f o l l o w e r m o v e = None

state actions representation = (

create state and actions vector representation (
p e r s p e c t i v e=r o u n d p l a y e r p e r s p e c t i v e ,
l e a d e r m o v e=l e a d e r m o v e ,
f o l l o w e r m o v e=f o l l o w e r m o v e ,
)
)

# append r e p l a y memory t o f i l e
w i t h open (
f i l e= s e l f . r e p l a y m e m o r y f i l e p a t h , mode=” a ”
) as r e p l a y m e m o r y f i l e :
# r e p l a y m e m o r y l i n e : l i s t [ t u p l e [ l i s t , number ] ] = [ s t a t e a c t i o n
# w r i t i n g t o r e p l a y memory f i l e i n t h e form ” [ f e a t u r e l i s t ] | |
replay memory file . write (
f ”{ s t r ( s t a t e a c t i o n s r e p r e s e n t a t i o n )[1: −1]} | | { i n t ( w o n l a
)

def t r a i n M L m o d e l (
r e p l a y m e m o r y l o c a t i o n : O p t i o n a l [ p a t h l i b . Path ] ,
m o d e l l o c a t i o n : O p t i o n a l [ p a t h l i b . Path ] ,
m o d e l c l a s s : L i t e r a l [ ”NN” , ”LR” ] = ”LR” ,
) −> None :
”””
Train t h e ML model f o r t h e MLPlayingBot b a s e d on r e p l a y memory s t o r e d b y t
T h i s i m p l e m e n t a t i o n h a s t h e o p t i o n t o t r a i n a n e u r a l n e t w o r k model or a mo
The model c l a s s e s u s e d i n t h i s i m p l e m n t a t i o n a r e n o t n e c e s a r i l y o p t i m a l .
12 Anonymous Submission

: param r e p l a y m e m o r y l o c a t i o n : L o c a t i o n o f t h e games s t o r e d by MLDataBot ,

: param m o d e l l o c a t i o n : L o c a t i o n where t h e model w i l l b e s t o r e d , d e f a u l t p a
: param m o d e l c l a s s : The machine l e a r n i n g model c l a s s t o b e used , e i t h e r ’NN
: param o v e r w r i t e : Whether t o o v e r w r i t e a p o s s i b l y e x i s t i n g model .
”””
i f r e p l a y m e m o r y l o c a t i o n i s None :
replay memory location = (
p a t h l i b . Path ( ” M L r e p l a y m e m o r i e s ” ) / ” t e s t r e p l a y m e m o r y ”
)
i f m o d e l l o c a t i o n i s None :
m o d e l l o c a t i o n = p a t h l i b . Path ( ” ML models ” ) / ” t e s t m o d e l ”
a s s e r t m o d e l c l a s s == ”NN” or m o d e l c l a s s == ”LR” , ”Unknown model c l a s s ”

# c h e c k t h a t t h e r e p l a y memory d a t a s e t i s f o u n d a t t h e s p e c i f i e d l o c a t i o n
i f not r e p l a y m e m o r y l o c a t i o n . e x i s t s ( ) :
r a i se V a l u e E r r o r ( f ” D a t a s e t was n o t f o u n d a t : { r e p l a y m e m o r y l o c a t i o n }

# Check i f model e x i s t s a l r e a d y
if model location . exists ():
r a i se V a l u e E r r o r (
f ” Model a t { m o d e l l o c a t i o n } e x i s t s a l r e a d y and o v e r w r i t e i s s e t t o
)

# c h e c k i f d i r e c t o r y e x i s t s , and i f not , t h e n c r e a t e i t
m o d e l l o c a t i o n . p a r e n t . mkdir ( p a r e n t s=True , e x i s t o k =True )

data : l i s t [ l i s t [ int ] ] = [ ]
t a r g e t s : l i s t [ int ] = [ ]
w i t h open ( f i l e=r e p l a y m e m o r y l o c a t i o n , mode=” r ” ) a s r e p l a y m e m o r y f i l e :
for l i n e in r e p l a y m e m o r y f i l e :
feature string , won label str = line . s p l i t (” | | ”)
f e a t u r e l i s t s t r i n g s : l i s t [ str ] = f e a t u r e s t r i n g . s p l i t ( ” , ” )
f e a t u r e l i s t = [ i n t ( f e a t u r e ) for f e a t u r e in f e a t u r e l i s t s t r i n g s ]
w o n l a b e l = int ( w o n l a b e l s t r )
d a t a . append ( f e a t u r e l i s t )
t a r g e t s . append ( w o n l a b e l )

print ( ” D a t a s e t S t a t i s t i c s : ” )
s a m p l e s o f w i n s = sum( t a r g e t s )
s a m p l e s o f l o s s e s = len ( t a r g e t s ) − s a m p l e s o f w i n s
print ( ” Samples o f w i n s : ” , s a m p l e s o f w i n s )
print ( ” Samples o f l o s s e s : ” , s a m p l e s o f l o s s e s )

# What t y p e o f model w i l l b e u s e d d e p e n d s on t h e v a l u e o f t h e p a r a m e t e r u s
 Title Suppressed Due to Excessive Length 13

i f m o d e l c l a s s == ”NN” :
#############################################
# N e u r a l Network model p a r a m e t e r s :
# l e a r n more a b o u t t h e model or how t o u s e b e t t e r u s e i t by c h e c k i n g o
# h t t p s : / / s c i k i t −l e a r n . o r g / s t a b l e / modules / g e n e r a t e d / s k l e a r n . n e u r a l n e t w
# P l a y around w i t h t h e model p a r a m e t e r s b e l o w
print ( ” T r a i n i n g a Complex ( N e u r a l Network ) model . ” )

# F e e l f r e e t o e x p e r i m e n t w i t h d i f f e r e n t number o f n e u r a l l a y e r s or d i
# T i p s : more n e u r o n s or more l a y e r s o f n e u r o n s c r e a t e a more c o m p l i c a t e
# n e e d s a b i g g e r d a t a s e t , b u t i f you f i n d t h e c o r r e c t c o m b i n a t i o n o f n

# one l a y e r o f 30 n e u r o n s
h i d d e n l a y e r s i z e s = 30
# two l a y e r s o f 30 and 5 n e u r o n s r e s p e c t i v e l y
# h i d d e n l a y e r s i z e s = (30 , 5)

# The l e a r n i n g r a t e d e t e r m i n e s how f a s t we move t o w a r d s t h e o p t i m a l s o

# A l o w l e a r n i n g r a t e w i l l c o n v e r g e s l o w l y , b u t a l a r g e one m i g h t o v e r
l e a r n i n g r a t e = 0.0001

# The r e g u l a r i z a t i o n term aims t o p r e v e n t o v e r − f i t t i n g , and we can t w e a

r e g u l a r i z a t i o n s t r e n g t h = 0.0001

# Train a n e u r a l n e t w o r k
learner = MLPClassifier (
h i d d e n l a y e r s i z e s=h i d d e n l a y e r s i z e s ,
l e a r n i n g r a t e i n i t =l e a r n i n g r a t e ,
a l p h a=r e g u l a r i z a t i o n s t r e n g t h ,
v e r b o s e=True ,
e a r l y s t o p p i n g=True ,
n i t e r n o c h a n g e =6 ,
a c t i v a t i o n=” t a n h ” ,
)
e l i f m o d e l c l a s s == ”LR” :
# Train a s i m p l e r L i n e a r L o g i s t i c R e g r e s s i o n model
# l e a r n more a b o u t t h e model or how t o u s e b e t t e r u s e i t by c h e c k i n g o
# h t t p s : / / s c i k i t −l e a r n . o r g / s t a b l e / modules / g e n e r a t e d / s k l e a r n . l i n e a r m o d
print ( ” T r a i n i n g a S i m p l e ( L i n e a r L o g i s t i c R e g r e s s i o n model ) ” )

# U s u a l l y t h e r e i s no r e a s o n t o c h a n g e t h e h y p e r p a r a m e t e r s o f s u c h a s i
l e a r n e r = L o g i s t i c R e g r e s s i o n ( m a x i t e r =1000)
else :
r a i se A s s e r t i o n E r r o r ( ”Unknown model c l a s s ” )
14 Anonymous Submission

s t a r t = time . time ()
print ( ” S t a r t i n g t r a i n i n g p h a s e . . . ” )

model = l e a r n e r . f i t ( data , t a r g e t s )
# Save t h e model i n a f i l e
j o b l i b . dump ( model , m o d e l l o c a t i o n )
end = t i m e . t i m e ( )
print ( ”The model was t r a i n e d i n ” , ( end − s t a r t ) / 6 0 , ” m i n u t e s . ” )

def c r e a t e s t a t e a n d a c t i o n s v e c t o r r e p r e s e n t a t i o n (
perspective : PlayerPerspective ,
l e a d e r m o v e : O p t i o n a l [ Move ] ,
f o l l o w e r m o v e : O p t i o n a l [ Move ] ,
) −> l i s t [ i n t ] :
”””
T h i s f u n c t i o n t a k e s a s i n p u t a P l a y e r P e r s p e c t i v e v a r i a b l e , and t h e two mov
and r e t u r n s a l i s t o f c o m p l e t e f e a t u r e r e p r e s e n t a t i o n t h a t c o n t a i n s a l l i n f
”””
player game state representation = get state feature vector ( perspective )
leader move representation = get move feature vector ( leader move )
follower move representation = get move feature vector ( follower move )

return (
player game state representation
+ leader move representation
+ follower move representation
)

def g e t o n e h o t e n c o d i n g o f c a r d s u i t ( c a r d s u i t : S u i t ) −> l i s t [ i n t ] :
”””
T r a n s l a t i n g t h e s u i t o f a c a r d i n t o one h o t v e c t o r e n c o d i n g o f s i z e 4 .
”””
c a r d s u i t o n e h o t : l i s t [ int ]
i f c a r d s u i t == S u i t .HEARTS:
card suit one hot = [0 , 0 , 0 , 1]
e l i f c a r d s u i t == S u i t . CLUBS :
card suit one hot = [0 , 0 , 1 , 0]
e l i f c a r d s u i t == S u i t . SPADES :
card suit one hot = [0 , 1 , 0 , 0]
e l i f c a r d s u i t == S u i t .DIAMONDS:
card suit one hot = [1 , 0 , 0 , 0]
else :
r a i se V a l u e E r r o r ( ” S u i t o f c a r d was n o t f o u n d ! ” )
 Title Suppressed Due to Excessive Length 15

return c a r d s u i t o n e h o t

def g e t o n e h o t e n c o d i n g o f c a r d r a n k ( c a r d r a n k : Rank ) −> l i s t [ i n t ] :

”””
T r a n s l a t i n g t h e r an k o f a c a r d i n t o one h o t v e c t o r e n c o d i n g o f s i z e 1 3 .
”””
card rank one hot : l i s t [ int ]
i f c a r d r a n k == Rank .ACE:
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1]
e l i f c a r d r a n k == Rank .TWO:
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 0]
e l i f c a r d r a n k == Rank .THREE:
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 0 , 0]
e l i f c a r d r a n k == Rank .FOUR:
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank . FIVE :
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank . SIX :
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank .SEVEN:
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank . EIGHT :
card rank one hot = [0 , 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank . NINE :
card rank one hot = [0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank .TEN:
card rank one hot = [0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank .JACK:
card rank one hot = [0 , 0 , 1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank .QUEEN:
card rank one hot = [0 , 1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank . KING :
card rank one hot = [1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
else :
r a i se A s s e r t i o n E r r o r ( ” P r o v i d e d c a r d Rank d o e s n o t e x i s t ! ” )
return c a r d r a n k o n e h o t

def g e t m o v e f e a t u r e v e c t o r ( move : O p t i o n a l [ Move ] ) −> l i s t [ i n t ] :

”””
In c a s e t h e r e i s n ’ t any move p r o v i d e d move t o encode , we s t i l l need t o c r e
f i l l e d w i t h 0 s , s i n c e t h e ML m o d e l s need t o r e c e i v e i n p u t o f t h e same d i m e
O t h e r w i s e , we c r e a t e a l l t h e i n f o r m a t i o n o f t h e move i ) move t y p e , i i ) p l a y
16 Anonymous Submission

t r a n s l a t e t h i s i n f o r m a t i o n i n t o one−h o t v e c t o r s r e s p e c t i v e l y , and c o n c a t e n
”””

i f move i s None :
move type one hot encoding numpy array = [0 , 0 , 0]
card rank one hot encoding numpy array = [0 , 0 , 0 , 0]
card suit one hot encoding numpy array = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0

else :
move type one hot encoding : l i s t [ int ]
# i n c a s e t h e move i s a m a r r i a g e move
i f move . i s m a r r i a g e ( ) :
move type one hot encoding = [0 , 0 , 1]
c a r d = move . q u e e n c a r d
# i n c a s e t h e move i s a trump e x c h a n g e move
e l i f move . i s t r u m p e x c h a n g e ( ) :
move type one hot encoding = [0 , 1 , 0]
c a r d = move . j a c k
# i n c a s e i t i s a r e g u l a r move
else :
move type one hot encoding = [1 , 0 , 0]
c a r d = move . c a r d
move type one hot encoding numpy array = move type one hot encoding
card rank one hot encoding numpy array = get one hot encoding of card r
c a r d . ra n k
)
card suit one hot encoding numpy array = get one hot encoding of card
card . s u i t
)

return (
move type one hot encoding numpy array
+ card rank one hot encoding numpy array
+ card suit one hot encoding numpy array
)

def g e t s t a t e f e a t u r e v e c t o r ( p e r s p e c t i v e : P l a y e r P e r s p e c t i v e ) −> l i s t [ i n t ] :
”””
This f u n c t i o n g a t h e r s a l l s u b j e c t i v e i n f o r m a t i o n t h a t t h i s b o t has a c c e s s
− points of t h i s player ( int )
− p o i n t s of the opponent ( i n t )
− pending p oi n t s of t h i s player ( i n t )
− pending p o i n t s of opponent ( i n t )
− t h e trump s u i t (1− h o t e n c o d i n g )
 Title Suppressed Due to Excessive Length 17

− p h a s e o f game (1− hoy e n c o d i n g )

− talon size ( int )
− i f t h i s p l a y e r i s l e a d e r (1− h o t e n c o d i n g )
− What i s t h e s t a t u s o f e a c h c a r d o f t h e d e c k ( where i t i s , or i f its loca

I m p o r t a n t : T h i s f u n c t i o n s h o u l d n o t i n c l u d e t h e move o f t h i s a g e n t .
I t s h o u l d o n l y i n c l u d e any e a r l i e r a c t i o n s o f o t h e r a g e n t s ( s o t h e a c t i o n
”””
# a l i s t o f a l l t h e f e a t u r e s t h a t c o n s i s t t h e s t a t e f e a t u r e s e t , o f t y p e np
s t a t e f e a t u r e l i s t : l i s t [ int ] = [ ]

player score = perspective . get my score ()

# − points of t h i s player ( int )
player points = player score . direct points
# − pending p oi n t s of t h i s player ( i n t )
player pending points = player score . pending points

# add t h e f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += [ p l a y e r p o i n t s ]
s t a t e f e a t u r e l i s t += [ p l a y e r p e n d i n g p o i n t s ]

opponents score = perspective . get opponent score ()

# − p o i n t s of the opponent ( i n t )
opponents points = opponents score . direct points
# − pending p o i n t s of opponent ( i n t )
opponents pending points = opponents score . pending points

# add t h e f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += [ o p p o n e n t s p o i n t s ]
s t a t e f e a t u r e l i s t += [ o p p o n e n t s p e n d i n g p o i n t s ]

# − t h e trump s u i t (1− h o t e n c o d i n g )
trump suit = perspective . get trump suit ()
trump suit one hot = get one hot encoding of card suit ( trump suit )
# add t h i s f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += t r u m p s u i t o n e h o t

# − p h a s e o f game (1− h o t e n c o d i n g )
g a m e p h a s e e n c o d e d = [ 1 , 0 ] i f p e r s p e c t i v e . g e t p h a s e ( ) == GamePhase .TWO e l s
# add t h i s f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += g a m e p h a s e e n c o d e d

# − talon size ( int )

t a l o n s i z e = perspective . g e t t a l o n s i z e ()
# add t h i s f e a t u r e s t o t h e f e a t u r e s e t
18 Anonymous Submission

s t a t e f e a t u r e l i s t += [ t a l o n s i z e ]

# − i f t h i s p l a y e r i s l e a d e r (1− h o t e n c o d i n g )
i am leader = [0 , 1] if p e r s p e c t i v e . am i leader () else [1 , 0]
# add t h i s f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += i a m l e a d e r

# g a t h e r a l l known d e c k i n f o r m a t i o n
hand cards = p e r s p e c t i v e . get hand ( ) . cards
trump card = p e r s p e c t i v e . get trump card ()
won cards = p e r s p e c t i v e . get won cards ( ) . g e t c a r d s ()
opponent won cards = p e r s p e c t i v e . get opponent won cards ( ) . g e t c a r d s ()
opponent known cards = p e r s p e c t i v e . get known cards of opponent hand ( ) . g e t c
# e a c h c a r d can e i t h e r b e i ) on p l a y e r ’ s hand , i i ) on p l a y e r ’ s won c a r d s ,
# v ) b e t h e trump c a r d or v i ) i n an unknown p o s i t i o n −> e i t h e r on t h e t a l o n
# There a r e a l l d i f f e r e n t c a s e s r e g a r d i n g c a r d ’ s k n o w l e d g e , and we r e p r e s e n

d e c k k n o w l e d g e i n c o n s e c u t i v e o n e h o t e n c o d i n g s : l i s t [ int ] = [ ]

for c a r d in S c h n a p s e n D e c k G e n e r a t o r ( ) . g e t i n i t i a l d e c k ( ) :
c a r d k n o w l e d g e i n o n e h o t e n c o d i n g : l i s t [ int ]
# i ) on p l a y e r ’ s hand
i f c a r d in h a n d c a r d s :
card knowledge in one hot encoding = [0 , 0 , 0 , 0 , 0 , 1]
# i i ) on p l a y e r ’ s won c a r d s
e l i f c a r d in w o n c a r d s :
card knowledge in one hot encoding = [0 , 0 , 0 , 0 , 1 , 0]
# i i i ) on o p p o n e n t ’ s hand
e l i f c a r d in o p p o n e n t k n o w n c a r d s :
card knowledge in one hot encoding = [0 , 0 , 0 , 1 , 0 , 0]
# i v ) on o p p o n e n t ’ s won c a r d s
e l i f c a r d in o p p o n e n t w o n c a r d s :
card knowledge in one hot encoding = [0 , 0 , 1 , 0 , 0 , 0]
# v ) b e t h e trump c a r d
e l i f c a r d == t r u m p c a r d :
card knowledge in one hot encoding = [0 , 1 , 0 , 0 , 0 , 0]
# v i ) i n an unknown p o s i t i o n a s i t i s i n v i s i b l e t o t h i s p l a y e r . Thus ,
else :
card knowledge in one hot encoding = [1 , 0 , 0 , 0 , 0 , 0]
# T h i s l i s t e v e n t u a l l y d e v e l o p s t o one l o n g 1− d i m e n s i o n a l numpy a r r a y
d e c k k n o w l e d g e i n c o n s e c u t i v e o n e h o t e n c o d i n g s += (
card knowledge in one hot encoding
)
# d e c k k n o w l e d g e f l a t t e n e d : np . n d a r r a y = np . c o n c a t e n a t e ( t u p l e ( d e c k k n o w l e d
 Title Suppressed Due to Excessive Length 19

# add t h i s f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += d e c k k n o w l e d g e i n c o n s e c u t i v e o n e h o t e n c o d i n g s

return s t a t e f e a t u r e l i s t

def c r e a t e r e p l a y m e m o r y d a t a s e t ( b o t 1 : Bot , b o t 2 : Bot ) −> None :

””” C r e a t e o f f l i n e d a t a s e t f o r t r a i n i n g a ML b o t .

Args :
b o t 1 , b o t 2 : t h e b o t o f y o ur c h o i c e .

”””
# d e f i n e r e p l a y memory d a t a b a s e c r e a t i o n p a r a m e t e r s
num of games : i n t = 10000
replay memory dir : str = ” ML replay memories ”
r e p l a y m e m o r y f i l e n a m e : s t r = ” random random 10k games . t x t ”
r e p l a y m e m o r y l o c a t i o n = p a t h l i b . Path ( r e p l a y m e m o r y d i r ) / r e p l a y m e m o r y f i

d e l e t e e x i s t i n g o l d e r d a t a s e t = False

# c h e c k i f n e e d e d t o d e l e t e any o l d e r v e r s i o n s o f t h e d a t a s e t
i f d e l e t e e x i s t i n g o l d e r d a t a s e t and r e p l a y m e m o r y l o c a t i o n . e x i s t s ( ) :
print (
f ”An e x i s t i n g d a t a s e t was f o u n d a t l o c a t i o n ’ { r e p l a y m e m o r y l o c a t i o
)
replay memory location . unlink ()

# i n any c a s e make s u r e t h e d i r e c t o r y e x i s t s
r e p l a y m e m o r y l o c a t i o n . p a r e n t . mkdir ( p a r e n t s=True , e x i s t o k =True )

# c r e a t e new r e p l a y memory d a t a s e t , a c c o r d i n g t o t h e b e h a v i o u r o f t h e p r o v
e n g i n e = SchnapsenGamePlayEngine ( )
r e p l a y m e m o r y r e c o r d i n g b o t 1 = MLDataBot (
b o t 1 , r e p l a y m e m o r y l o c a t i o n=r e p l a y m e m o r y l o c a t i o n
)
r e p l a y m e m o r y r e c o r d i n g b o t 2 = MLDataBot (
b o t 2 , r e p l a y m e m o r y l o c a t i o n=r e p l a y m e m o r y l o c a t i o n
)
for i in range ( 1 , num of games + 1 ) :
i f i % 500 == 0 :
print ( f ” P r o g r e s s : { i }/{ num of games } ” )
engine . play game (
replay memory recording bot 1 ,
replay memory recording bot 2 ,
20 Anonymous Submission

random . Random ( i ) ,
)
print (
f ” R e p l a y memory d a t a s e t r e c o r d e r f o r { num of games } games . \ n D a t a s e t i s
)

def t r a i n m o d e l ( m o d e l t y p e : s t r ) −> None :

””” Train model f o r ML b o t .

Args :
m o d e l t y p e : e i t h e r ’NN ’ f o r a n e u r a l network , or ’LR ’ f o r a l o g i s t i c r

”””
# d i r e c t o r y where t h e r e p l a y memory i s s a v e d
r e p l a y m e m o r y f i l e n a m e : s t r = ” random random 10k games . t x t ”
# f i l e n a m e o f r e p l a y memory w i t h i n t h a t d i r e c t o r y
r e p l a y m e m o r i e s d i r e c t o r y : str = ” ML replay memories ”
# Whether t o t r a i n a c o m p l i c a t e d N e u r a l Network model or a s i m p l e one .
# T i p s : a n e u r a l n e t w o r k u s u a l l y r e q u i r e s b i g g e r d a t a s e t s t o b e t r a i n e d on
# F e e l f r e e t o p l a y w i t h t h e h y p e r p a r a m e t e r s o f t h e model i n f i l e ’ m l b o t .
# under t h e c o d e o f body o f t h e i f s t a t e m e n t ’ i f u s e n e u r a l n e t w o r k : ’
replay memory location = (
p a t h l i b . Path ( r e p l a y m e m o r i e s d i r e c t o r y ) / r e p l a y m e m o r y f i l e n a m e
)
model name : s t r = ” s i m p l e m o d e l ”
m o d e l d i r : s t r = ” ML models ”
m o d e l l o c a t i o n = p a t h l i b . Path ( m o d e l d i r ) / model name
o v e r w r i t e : bool = True

i f o v e r w r i t e and m o d e l l o c a t i o n . e x i s t s ( ) :
print (
f ” Model a t { m o d e l l o c a t i o n } e x i s t s a l r e a d y and w i l l b e o v e r w r i t t e n
)
model location . unlink ()

train ML model (
r e p l a y m e m o r y l o c a t i o n=r e p l a y m e m o r y l o c a t i o n ,
m o d e l l o c a t i o n=m o d e l l o c a t i o n ,
m o d e l c l a s s=m o d e l t y p e ,
)

def p l a y g a m e s a n d r e t u r n s t a t s (
e n g i n e : GamePlayEngine , b o t 1 : Bot , b o t 2 : Bot , n u m b e r o f g a m e s : i n t
) −> i n t :
 Title Suppressed Due to Excessive Length 21

”””
P l a y n u m b e r o f g a m e s games b e t w e e n b o t 1 and b o t 2 , u s i n g t h e SchnapsenGameP
Prints progress .
”””
bot1 wins : int = 0
lead , f o l l o w e r = bot1 , bot2
for i in range ( 1 , n u m b e r o f g a m e s + 1 ) :
i f i % 2 == 0 :
# swap b o t s s o b o t h s t a r t t h e same number o f t i m e s
lead , f o l l o w e r = follower , lead
winner , , = e n g i n e . p l a y g a m e ( l e a d , f o l l o w e r , random . Random ( i ) )
i f w i n n e r == b o t 1 :
b o t 1 w i n s += 1
i f i % 500 == 0 :
print ( f ” P r o g r e s s : { i }/{ n u m b e r o f g a m e s } ” )
return b o t 1 w i n s

7.2 Appendix B

This is the code used for the modified ML Bot:

Listing 1.2. Modified ML Bot code Python-code

from t y p i n g import O p t i o n a l , c a s t , L i t e r a l
import random
import c l i c k
import t i m e
import p a t h l i b

import j o b l i b
from s k l e a r n . n e u r a l n e t w o r k import M L P C l a s s i f i e r
from s k l e a r n . l i n e a r m o d e l import L o g i s t i c R e g r e s s i o n

from s c h n a p s e n . d e c k import S u i t , Rank

from s c h n a p s e n . game import (
Bot ,
PlayerPerspective ,
SchnapsenDeckGenerator ,
Move ,
Trick ,
GamePhase ,
GamePlayEngine ,
SchnapsenGamePlayEngine ,
22 Anonymous Submission

TrumpExchange ,
)
from s c h n a p s e n . a l t e r n a t i v e e n g i n e s . a c e o n e e n g i n e import AceOneGamePlayEngine
from s c h n a p s e n . b o t s import RandBot , RdeepBot , MiniMaxBot , AlphaBetaBot
from s c h n a p s e n . b o t s . e x a m p l e b o t import ExampleBot
from s c h n a p s e n . a l t e r n a t i v e e n g i n e s . t w e n t y f o u r c a r d s c h n a p s e n import (
TwentyFourSchnapsenGamePlayEngine ,
)

c l a s s MLPlayingBot ( Bot ) :
”””
T h i s c l a s s l o a d s a t r a i n e d ML model and u s e s i t t o p l a y
”””
def i n i t ( s e l f , m o d e l l o c a t i o n : p a t h l i b . Path , name : O p t i o n a l [ s t r ] = Non
super ( ) . i n i t ( name )
model location = model location
a s s e r t m o d e l l o c a t i o n . e x i s t s ( ) , f ” Model c o u l d n o t b e f o u n d a t : { m o d e l
s e l f . model = j o b l i b . load ( model location )

def g e t m o v e ( s e l f , p e r s p e c t i v e : P l a y e r P e r s p e c t i v e , l e a d e r m o v e : O p t i o n a l [ M
state representation = get state feature vector ( perspective )
leader move representation = get move feature vector ( leader move )
my valid moves = p e r s p e c t i v e . valid moves ()
my move representations : l i s t [ l i s t [ int ] ] = [ ]
for my move in m y v a l i d m o v e s :
m y m o v e r e p r e s e n t a t i o n s . append ( g e t m o v e f e a t u r e v e c t o r ( my move ) )

a c t i o n s t a t e r e p r e s e n t a t i o n s : l i s t [ l i s t [ int ] ] = [ ]

if perspective . am i leader ():

f o l l o w e r m o v e r e p r e s e n t a t i o n = g e t m o v e f e a t u r e v e c t o r ( None )
for m y m o v e r e p r e s e n t a t i o n in m y m o v e r e p r e s e n t a t i o n s :
a c t i o n s t a t e r e p r e s e n t a t i o n s . append (
s t a t e r e p r e s e n t a t i o n + my move representation + follower m
else :
for m y m o v e r e p r e s e n t a t i o n in m y m o v e r e p r e s e n t a t i o n s :
a c t i o n s t a t e r e p r e s e n t a t i o n s . append (
s t a t e r e p r e s e n t a t i o n + leader move representation + my mo

model output = s e l f . model . predict proba ( a c t i o n s t a t e r e p r e s e n t a t i o n s

w i n n i n g p r o b a b i l i t i e s o f m o v e s = [ o u t c o m e p r o b [ 1 ] for o u t c o m e p r o b in
h i g h e s t v a l u e : f l o a t = −1
b e s t m o v e : Move
for i n d e x , v a l u e in enumerate ( w i n n i n g p r o b a b i l i t i e s o f m o v e s ) :
 Title Suppressed Due to Excessive Length 23

if value > highest value :

highest value = value
best move = my valid moves [ index ]
a s s e r t b e s t m o v e i s not None
return b e s t m o v e

c l a s s MLDataBot ( Bot ) :
”””
This c l a s s i s d e f i n e d to a l l o w t h e c r e a t i o n o f a t r a i n i n g schnapsen b o t d a
”””
def i n i t ( s e l f , b o t : Bot , r e p l a y m e m o r y l o c a t i o n : p a t h l i b . Path ) −> None
s e l f . b o t : Bot = b o t
s e l f . r e p l a y m e m o r y f i l e p a t h : p a t h l i b . Path = r e p l a y m e m o r y l o c a t i o n

def g e t m o v e ( s e l f , p e r s p e c t i v e : P l a y e r P e r s p e c t i v e , l e a d e r m o v e : O p t i o n a l [ M
return s e l f . b o t . g e t m o v e ( p e r s p e c t i v e=p e r s p e c t i v e , l e a d e r m o v e=l e a d e r m o

def n o t i f y g a m e e n d ( s e l f , won : bool , p e r s p e c t i v e : P l a y e r P e r s p e c t i v e ) −> No

game history : l i s t [ tuple [ P l a y e r P e r s p e c t i v e , Trick ] ] = c a s t ( l i s t [ tuple [
w o n l a b e l = won

for r o u n d p l a y e r p e r s p e c t i v e , r o u n d t r i c k in g a m e h i s t o r y :
if round trick . is trump exchange () :
leader move = r o u n d t r i c k . exchange
f o l l o w e r m o v e = None
else :
leader move = round trick . leader move
follower move = round trick . follower move

if round player perspective . am i leader ():

f o l l o w e r m o v e = None

state actions representation = create state and actions vector repr

p e r s p e c t i v e=r o u n d p l a y e r p e r s p e c t i v e , l e a d e r m o v e=l e a d e r m o v e ,

w i t h open ( f i l e= s e l f . r e p l a y m e m o r y f i l e p a t h , mode=” a ” ) a s r e p l a y m
r e p l a y m e m o r y f i l e . w r i t e ( f ”{ s t r ( s t a t e a c t i o n s r e p r e s e n t a t i o n )[1

def t r a i n M L m o d e l ( r e p l a y m e m o r y l o c a t i o n : O p t i o n a l [ p a t h l i b . Path ] ,
m o d e l l o c a t i o n : O p t i o n a l [ p a t h l i b . Path ] ,
m o d e l c l a s s : L i t e r a l [ ”NN” , ”LR” ] = ”LR”
) −> None :
i f r e p l a y m e m o r y l o c a t i o n i s None :
r e p l a y m e m o r y l o c a t i o n = p a t h l i b . Path ( ’ M L r e p l a y m e m o r i e s ’ ) / ’ t e s t r e p
i f m o d e l l o c a t i o n i s None :
24 Anonymous Submission

m o d e l l o c a t i o n = p a t h l i b . Path ( ” ML models ” ) / ’ t e s t m o d e l ’
a s s e r t m o d e l c l a s s == ’NN ’ or m o d e l c l a s s == ’LR ’ , ”Unknown model c l a s s ”

i f not r e p l a y m e m o r y l o c a t i o n . e x i s t s ( ) :
r a i se V a l u e E r r o r ( f ” D a t a s e t was n o t f o u n d a t : { r e p l a y m e m o r y l o c a t i o n }

if model location . exists ():

r a i se V a l u e E r r o r (
f ” Model a t { m o d e l l o c a t i o n } e x i s t s a l r e a d y and o v e r w r i t e i s s e t t o

m o d e l l o c a t i o n . p a r e n t . mkdir ( p a r e n t s=True , e x i s t o k =True )

data : l i s t [ l i s t [ int ] ] = [ ]
t a r g e t s : l i s t [ int ] = [ ]
w i t h open ( f i l e=r e p l a y m e m o r y l o c a t i o n , mode=” r ” ) a s r e p l a y m e m o r y f i l e :
for l i n e in r e p l a y m e m o r y f i l e :
feature string , won label str = line . s p l i t (” | | ”)
f e a t u r e l i s t s t r i n g s : l i s t [ str ] = f e a t u r e s t r i n g . s p l i t ( ” , ” )
f e a t u r e l i s t = [ i n t ( f e a t u r e ) for f e a t u r e in f e a t u r e l i s t s t r i n g s ]
w o n l a b e l = int ( w o n l a b e l s t r )
d a t a . append ( f e a t u r e l i s t )
t a r g e t s . append ( w o n l a b e l )

print ( ” D a t a s e t S t a t i s t i c s : ” )
s a m p l e s o f w i n s = sum( t a r g e t s )
s a m p l e s o f l o s s e s = len ( t a r g e t s ) − s a m p l e s o f w i n s
print ( ” Samples o f w i n s : ” , s a m p l e s o f w i n s )
print ( ” Samples o f l o s s e s : ” , s a m p l e s o f l o s s e s )

i f m o d e l c l a s s == ’NN ’ :
h i d d e n l a y e r s i z e s = (30)
l e a r n i n g r a t e = 0.0001
r e g u l a r i z a t i o n s t r e n g t h = 0.0001
l e a r n e r = M L P C l a s s i f i e r ( h i d d e n l a y e r s i z e s=h i d d e n l a y e r s i z e s , l e a r n i n
a l p h a=r e g u l a r i z a t i o n s t r e n g t h , v e r b o s e=True , e
a c t i v a t i o n= ’ t a n h ’ )
e l i f m o d e l c l a s s == ’LR ’ :
l e a r n e r = L o g i s t i c R e g r e s s i o n ( m a x i t e r =1000)
else :
r a i se A s s e r t i o n E r r o r ( ”Unknown model c l a s s ” )

s t a r t = time . time ()
print ( ” S t a r t i n g t r a i n i n g p h a s e . . . ” )

model = l e a r n e r . f i t ( data , t a r g e t s )
 Title Suppressed Due to Excessive Length 25

j o b l i b . dump ( model , m o d e l l o c a t i o n )
end = t i m e . t i m e ( )
print ( ’ The model was t r a i n e d i n ’ , ( end − s t a r t ) / 6 0 , ’ m i n u t e s . ’ )

def c r e a t e s t a t e a n d a c t i o n s v e c t o r r e p r e s e n t a t i o n ( p e r s p e c t i v e : P l a y e r P e r s p e c t i
f o l l o w e r m o v e : O p t i o n a l [ Mov
”””
T h i s f u n c t i o n t a k e s a s i n p u t a P l a y e r P e r s p e c t i v e v a r i a b l e , and t h e two mov
and r e t u r n s a l i s t o f c o m p l e t e f e a t u r e r e p r e s e n t a t i o n t h a t c o n t a i n s a l l i n f
”””
player game state representation = get state feature vector ( perspective )
leader move representation = get move feature vector ( leader move )
follower move representation = get move feature vector ( follower move )

return p l a y e r g a m e s t a t e r e p r e s e n t a t i o n + l e a d e r m o v e r e p r e s e n t a t i o n + f o l l

def g e t o n e h o t e n c o d i n g o f c a r d s u i t ( c a r d s u i t : S u i t ) −> l i s t [ i n t ] :
”””
T r a n s l a t i n g t h e s u i t o f a c a r d i n t o one h o t v e c t o r e n c o d i n g o f s i z e 4 .
”””
c a r d s u i t o n e h o t : l i s t [ int ]
i f c a r d s u i t == S u i t .HEARTS:
card suit one hot = [0 , 0 , 0 , 1]
e l i f c a r d s u i t == S u i t . CLUBS :
card suit one hot = [0 , 0 , 1 , 0]
e l i f c a r d s u i t == S u i t . SPADES :
card suit one hot = [0 , 1 , 0 , 0]
e l i f c a r d s u i t == S u i t .DIAMONDS:
card suit one hot = [1 , 0 , 0 , 0]
else :
r a i se V a l u e E r r o r ( ” S u i t o f c a r d was n o t f o u n d ! ” )

return c a r d s u i t o n e h o t

def g e t o n e h o t e n c o d i n g o f c a r d r a n k ( c a r d r a n k : Rank ) −> l i s t [ i n t ] :

”””
T r a n s l a t i n g t h e r an k o f a c a r d i n t o one h o t v e c t o r e n c o d i n g o f s i z e 1 3 .
”””
card rank one hot : l i s t [ int ]
i f c a r d r a n k == Rank .ACE:
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1]
e l i f c a r d r a n k == Rank .TWO:
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 0]
26 Anonymous Submission

e l i f c a r d r a n k == Rank .THREE:
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 0 , 0]
e l i f c a r d r a n k == Rank .FOUR:
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank . FIVE :
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank . SIX :
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank .SEVEN:
card rank one hot = [0 , 0 , 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank . EIGHT :
card rank one hot = [0 , 0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank . NINE :
card rank one hot = [0 , 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank .TEN:
card rank one hot = [0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank .JACK:
card rank one hot = [0 , 0 , 1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank .QUEEN:
card rank one hot = [0 , 1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
e l i f c a r d r a n k == Rank . KING :
card rank one hot = [1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0]
else :
r a i se A s s e r t i o n E r r o r ( ” P r o v i d e d c a r d Rank d o e s n o t e x i s t ! ” )
return c a r d r a n k o n e h o t

def g e t m o v e f e a t u r e v e c t o r ( move : O p t i o n a l [ Move ] ) −> l i s t [ i n t ] :

”””
In c a s e t h e r e i s n ’ t any move p r o v i d e d move t o encode , we s t i l l need t o
f i l l e d w i t h 0 s , s i n c e t h e ML m o d e l s need t o r e c e i v e i n p u t o f t h e same
O t h e r w i s e , we c r e a t e a l l t h e i n f o r m a t i o n o f t h e move i ) move t y p e , i i )
t r a n s l a t e t h i s i n f o r m a t i o n i n t o one−h o t v e c t o r s r e s p e c t i v e l y , and c o n c
”””

i f move i s None :
move type one hot encoding numpy array = [0 , 0 , 0]
card rank one hot encoding numpy array = [0 , 0 , 0 , 0]
card suit one hot encoding numpy array = [0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0

else :
move type one hot encoding : l i s t [ int ]
# i n c a s e t h e move i s a m a r r i a g e move
i f move . i s m a r r i a g e ( ) :
move type one hot encoding = [0 , 0 , 1]
 Title Suppressed Due to Excessive Length 27

c a r d = move . q u e e n c a r d
# i n c a s e t h e move i s a trump e x c h a n g e move
e l i f move . i s t r u m p e x c h a n g e ( ) :
move type one hot encoding = [0 , 1 , 0]
c a r d = move . j a c k
# i n c a s e i t i s a r e g u l a r move
else :
move type one hot encoding = [1 , 0 , 0]
c a r d = move . c a r d
move type one hot encoding numpy array = move type one hot encoding
card rank one hot encoding numpy array = get one hot encoding of card r
card suit one hot encoding numpy array = get one hot encoding of card

return m o v e t y p e o n e h o t e n c o d i n g n u m p y a r r a y + c a r d r a n k o n e h o t e n c o d i n g

def g e t s t a t e f e a t u r e v e c t o r ( p e r s p e c t i v e : P l a y e r P e r s p e c t i v e ) −> l i s t [ i n t ] :
”””
This f u n c t i o n g a t h e r s a l l s u b j e c t i v e i n f o r m a t i o n t h a t t h i s b o t has a c c
− points of t h i s player ( int )
− p o i n t s of the opponent ( i n t )
− pending p oi n t s of t h i s player ( i n t )
− pending p o i n t s of opponent ( i n t )
− t h e trump s u i t (1− h o t e n c o d i n g )
− p h a s e o f game (1− hoy e n c o d i n g )
− talon size ( int )
− i f t h i s p l a y e r i s l e a d e r (1− h o t e n c o d i n g )
− What i s t h e s t a t u s o f e a c h c a r d o f t h e d e c k ( where i t i s , or i f i t s

I m p o r t a n t : T h i s f u n c t i o n s h o u l d n o t i n c l u d e t h e move o f t h i s a g e n t .
I t s h o u l d o n l y i n c l u d e any e a r l i e r a c t i o n s o f o t h e r a g e n t s ( s o t h e a c t i
”””
# a l i s t o f a l l t h e f e a t u r e s t h a t c o n s i s t t h e s t a t e f e a t u r e s e t , o f t y p e np
s t a t e f e a t u r e l i s t : l i s t [ int ] = [ ]

player score = perspective . get my score ()

# − points of t h i s player ( int )
player points = player score . direct points
# − pending p oi n t s of t h i s player ( i n t )
player pending points = player score . pending points

# add t h e f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += [ p l a y e r p o i n t s ]
s t a t e f e a t u r e l i s t += [ p l a y e r p e n d i n g p o i n t s ]
28 Anonymous Submission

opponents score = perspective . get opponent score ()

# − p o i n t s of the opponent ( i n t )
opponents points = opponents score . direct points
# − pending p o i n t s of opponent ( i n t )
opponents pending points = opponents score . pending points

# add t h e f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += [ o p p o n e n t s p o i n t s ]
s t a t e f e a t u r e l i s t += [ o p p o n e n t s p e n d i n g p o i n t s ]

# − t h e trump s u i t (1− h o t e n c o d i n g )
trump suit = perspective . get trump suit ()
trump suit one hot = get one hot encoding of card suit ( trump suit )
# add t h i s f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += t r u m p s u i t o n e h o t

# − p h a s e o f game (1− h o t e n c o d i n g )
g a m e p h a s e e n c o d e d = [ 1 , 0 ] i f p e r s p e c t i v e . g e t p h a s e ( ) == GamePhase .TWO e l s
# add t h i s f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += g a m e p h a s e e n c o d e d

# − talon size ( int )

t a l o n s i z e = perspective . g e t t a l o n s i z e ()
# add t h i s f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += [ t a l o n s i z e ]

# − i f t h i s p l a y e r i s l e a d e r (1− h o t e n c o d i n g )
i am leader = [0 , 1] if p e r s p e c t i v e . am i leader () else [1 , 0]
# add t h i s f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += i a m l e a d e r

# g a t h e r a l l known d e c k i n f o r m a t i o n
hand cards = p e r s p e c t i v e . get hand ( ) . cards
trump card = p e r s p e c t i v e . get trump card ()
won cards = p e r s p e c t i v e . get won cards ( ) . g e t c a r d s ()
opponent won cards = p e r s p e c t i v e . get opponent won cards ( ) . g e t c a r d s ()
opponent known cards = p e r s p e c t i v e . get known cards of opponent hand ( ) . g e t c
# e a c h c a r d can e i t h e r b e i ) on p l a y e r ’ s hand , i i ) on p l a y e r ’ s won c a r d s ,
# v ) b e t h e trump c a r d or v i ) i n an unknown p o s i t i o n −> e i t h e r on t h e t a l o n
# There a r e a l l d i f f e r e n t c a s e s r e g a r d i n g c a r d ’ s k n o w l e d g e , and we r e p r e s e n

d e c k k n o w l e d g e i n c o n s e c u t i v e o n e h o t e n c o d i n g s : l i s t [ int ] = [ ]

for c a r d in S c h n a p s e n D e c k G e n e r a t o r ( ) . g e t i n i t i a l d e c k ( ) :
c a r d k n o w l e d g e i n o n e h o t e n c o d i n g : l i s t [ int ]
 Title Suppressed Due to Excessive Length 29

# i ) on p l a y e r ’ s hand
i f c a r d in h a n d c a r d s :
card knowledge in one hot encoding = [0 , 0 , 0 , 0 , 0 , 1]
# i i ) on p l a y e r ’ s won c a r d s
e l i f c a r d in w o n c a r d s :
card knowledge in one hot encoding = [0 , 0 , 0 , 0 , 1 , 0]
# i i i ) on o p p o n e n t ’ s hand
e l i f c a r d in o p p o n e n t k n o w n c a r d s :
card knowledge in one hot encoding = [0 , 0 , 0 , 1 , 0 , 0]
# i v ) on o p p o n e n t ’ s won c a r d s
e l i f c a r d in o p p o n e n t w o n c a r d s :
card knowledge in one hot encoding = [0 , 0 , 1 , 0 , 0 , 0]
# v ) b e t h e trump c a r d
e l i f c a r d == t r u m p c a r d :
card knowledge in one hot encoding = [0 , 1 , 0 , 0 , 0 , 0]
# v i ) i n an unknown p o s i t i o n a s i t i s i n v i s i b l e t o t h i s p l a y e r . Thus ,
else :
card knowledge in one hot encoding = [1 , 0 , 0 , 0 , 0 , 0]
# T h i s l i s t e v e n t u a l l y d e v e l o p s t o one l o n g 1− d i m e n s i o n a l numpy a r r a y
d e c k k n o w l e d g e i n c o n s e c u t i v e o n e h o t e n c o d i n g s += c a r d k n o w l e d g e i n o n
# d e c k k n o w l e d g e f l a t t e n e d : np . n d a r r a y = np . c o n c a t e n a t e ( t u p l e ( d e c k k n o w l e d

# add t h i s f e a t u r e s t o t h e f e a t u r e s e t
s t a t e f e a t u r e l i s t += d e c k k n o w l e d g e i n c o n s e c u t i v e o n e h o t e n c o d i n g s

return s t a t e f e a t u r e l i s t

def c r e a t e r e p l a y m e m o r y d a t a s e t ( b o t 1 : Bot , b o t 2 : Bot ) −> None :

””” C r e a t e o f f l i n e d a t a s e t f o r t r a i n i n g a ML b o t .

Args :
b o t 1 , b o t 2 : t h e b o t o f y o ur c h o i c e .

”””
# d e f i n e r e p l a y memory d a t a b a s e c r e a t i o n p a r a m e t e r s
num of games : i n t = 10000
replay memory dir : str = ” ML replay memories ”
r e p l a y m e m o r y f i l e n a m e : s t r = ” random random 10k games . t x t ”
r e p l a y m e m o r y l o c a t i o n = p a t h l i b . Path ( r e p l a y m e m o r y d i r ) / r e p l a y m e m o r y f i

d e l e t e e x i s t i n g o l d e r d a t a s e t = False

# c h e c k i f n e e d e d t o d e l e t e any o l d e r v e r s i o n s o f t h e d a t a s e t
i f d e l e t e e x i s t i n g o l d e r d a t a s e t and r e p l a y m e m o r y l o c a t i o n . e x i s t s ( ) :
30 Anonymous Submission

print (
f ”An e x i s t i n g d a t a s e t was f o u n d a t l o c a t i o n ’ { r e p l a y m e m o r y l o c a t i o
)
replay memory location . unlink ()

# i n any c a s e make s u r e t h e d i r e c t o r y e x i s t s
r e p l a y m e m o r y l o c a t i o n . p a r e n t . mkdir ( p a r e n t s=True , e x i s t o k =True )

# c r e a t e new r e p l a y memory d a t a s e t , a c c o r d i n g t o t h e b e h a v i o u r o f t h e p r o v
e n g i n e = SchnapsenGamePlayEngine ( )
r e p l a y m e m o r y r e c o r d i n g b o t 1 = MLDataBot (
b o t 1 , r e p l a y m e m o r y l o c a t i o n=r e p l a y m e m o r y l o c a t i o n
)
r e p l a y m e m o r y r e c o r d i n g b o t 2 = MLDataBot (
b o t 2 , r e p l a y m e m o r y l o c a t i o n=r e p l a y m e m o r y l o c a t i o n
)
for i in range ( 1 , num of games + 1 ) :
i f i % 500 == 0 :
print ( f ” P r o g r e s s : { i }/{ num of games } ” )
engine . play game (
replay memory recording bot 1 ,
replay memory recording bot 2 ,
random . Random ( i ) ,
)
print (
f ” R e p l a y memory d a t a s e t r e c o r d e r f o r { num of games } games . \ n D a t a s e t i s
)

def t r a i n m o d e l ( m o d e l t y p e : s t r ) −> None :

””” Train model f o r ML b o t .

Args :
m o d e l t y p e : e i t h e r ’NN ’ f o r a n e u r a l network , or ’LR ’ f o r a l o g i s t i c r

”””
# d i r e c t o r y where t h e r e p l a y memory i s s a v e d
r e p l a y m e m o r y f i l e n a m e : s t r = ” random random 10k games . t x t ”
# f i l e n a m e o f r e p l a y memory w i t h i n t h a t d i r e c t o r y
r e p l a y m e m o r i e s d i r e c t o r y : str = ” ML replay memories ”
# Whether t o t r a i n a c o m p l i c a t e d N e u r a l Network model or a s i m p l e one .
# T i p s : a n e u r a l n e t w o r k u s u a l l y r e q u i r e s b i g g e r d a t a s e t s t o b e t r a i n e d on
# F e e l f r e e t o p l a y w i t h t h e h y p e r p a r a m e t e r s o f t h e model i n f i l e ’ m l b o t .
# under t h e c o d e o f body o f t h e i f s t a t e m e n t ’ i f u s e n e u r a l n e t w o r k : ’
replay memory location = (
 Title Suppressed Due to Excessive Length 31

p a t h l i b . Path ( r e p l a y m e m o r i e s d i r e c t o r y ) / r e p l a y m e m o r y f i l e n a m e
)
model name : s t r = ” s i m p l e m o d e l ”
m o d e l d i r : s t r = ” ML models ”
m o d e l l o c a t i o n = p a t h l i b . Path ( m o d e l d i r ) / model name
o v e r w r i t e : bool = True

i f o v e r w r i t e and m o d e l l o c a t i o n . e x i s t s ( ) :
print (
f ” Model a t { m o d e l l o c a t i o n } e x i s t s a l r e a d y and w i l l b e o v e r w r i t t e n
)
model location . unlink ()

train ML model (
r e p l a y m e m o r y l o c a t i o n=r e p l a y m e m o r y l o c a t i o n ,
m o d e l l o c a t i o n=m o d e l l o c a t i o n ,
m o d e l c l a s s=m o d e l t y p e ,
)

7.3 Appendix C

This is the code used for the Randbot:

Listing 1.3. Randbot code Python-code

import random
from t y p i n g import O p t i o n a l
from s c h n a p s e n . game import Bot , P l a y e r P e r s p e c t i v e , Move

c l a s s RandBot ( Bot ) :
””” T h i s b o t p l a y s random moves , d e t e r m i n i s t i c a l l y u s i n g t h e random number

Args :
rand ( random . Random ) : The random number g e n e r a t o r u s e d t o make t h e rand
name ( O p t i o n a l [ s t r ] ) : The o p t i o n a l name o f t h i s b o t
”””
def i n i t ( s e l f , rand : random . Random , name : O p t i o n a l [ s t r ] = None ) −> Non
super ( ) . i n i t ( name )
s e l f . rng = rand

def g e t m o v e (
self ,
perspective : PlayerPerspective ,
32 Anonymous Submission

l e a d e r m o v e : O p t i o n a l [ Move ] ,
) −> Move :
moves : l i s t [ Move ] = p e r s p e c t i v e . v a l i d m o v e s ( )
move = s e l f . rng . c h o i c e ( moves )
return move

7.4 Appendix D

This is the code used to conduct the experiment:

Listing 1.4. Voorbeeld Python-code

%%cmd
rd / s / q s c h n a p s e n
g i t c l o n e h t t p s : / / g i t h u b . com/ i n t e l l i g e n t −s y s t e m s −c o u r s e / s c h n a p s e n
cd s c h n a p s e n
p i p i n s t a l l −e .
cd . .

”””
Initialising

F i r s t , we h a v e t o i n s t a l l t h e s c h n a p s e n p y t h o n p a c k a g e . Run t h e b e l o w c o d e c e l
A f t e r r u n n i n g t h e c e l l , you h a v e t h e s c h n a p s e n G i t h u b r e p o s i t o r y c l o n e d i n yo u
c u r r e n t d i r e c t o r y . You can f i n d t h e new d i r e c t o r y c r e a t e d w i t h t h e name s c h n a p
The d e t a i l e d i n s t a l l a t i o n i n s t r u c t i o n s can b e f o u n d i n t h e README. md o f t h e r e

”””

import random
from s c h n a p s e n . b o t s import RandBot
from m l b o t import c r e a t e r e p l a y m e m o r y d a t a s e t

r a n d o m g e n e r a t o r = random . Random ( )

b o t 1 = RandBot ( rand=r a n d o m g e n e r a t o r )
b o t 2 = RandBot ( rand=r a n d o m g e n e r a t o r )

c r e a t e r e p l a y m e m o r y d a t a s e t ( b o t 1=b o t 1 , b o t 2=b o t 2 )

”””
c r e a t e r e p l a y memory d a t a s e t f o r ML Bo ts
”””
 Title Suppressed Due to Excessive Length 33

from m l b o t import t r a i n m o d e l

t r a i n m o d e l ( m o d e l t y p e= ’NN ’ )

”””
Train ML Bo ts
”””

import p a t h l i b
import random
from s c h n a p s e n . game import SchnapsenGamePlayEngine
from s c h n a p s e n . b o t s import RandBot
from m l b o t import MLPlayingBot

m y r e p e a t s = 10000 #p l a y i n g 10000 games

m o d e l d i r = ” ML models ”
model name = ” s i m p l e m o d e l ”
m o d e l l o c a t i o n = p a t h l i b . Path ( m o d e l d i r ) / model name

b o t 1 = RandBot ( rand=random . Random ( random . r a n d i n t ( 1 , 1 0 0 ) ) , name=” r a n d b o t ” )

b o t 3 = MLPlayingBot ( m o d e l l o c a t i o n=m o d e l l o c a t i o n , name= ’ MLBot ’ )
# Update t h e b o t c l a s s

b o t s = [ bot1 , bot3 ]
n = len ( b o t s )
w i n s = { s t r ( b o t ) : 0 for b o t in b o t s }

total games = my repeats # Only p l a y 10000 matches b e t w e e n b o t 1 and b o t 3

played games = 0

print ( ” P l a y i n g {} games : ” . format ( i n t ( t o t a l g a m e s ) ) )

for r in range ( m y r e p e a t s ) :
# C r e a t i n g a new i n s t a n c e o f SchnapsenGamePlayEngine f o r e a c h game
e n g i n e = SchnapsenGamePlayEngine ( )

i f random . c h o i c e ( [ True , F a l s e ] ) :
p = [ 0 , 1 ] # I n d e x 0 c o r r e s p o n d s t o b o t 1 , and i n d e x 1 c o r r e s p o n d s t o
else :
p = [1 , 0]

winner id , game points , scores = engine . play game ( bots [ p [ 0 ] ] , bots [ p [ 1 ] ] , r

w i n n e r b o t = b o t s [ p [ 0 ] i f w i n n e r i d == 0 e l s e p [ 1 ] ]
34 Anonymous Submission

# Updating t h i s l i n e to convert the winner bot to a s t r i n g

w i n s [ s t r ( w i n n e r b o t ) ] += 1

p l a y e d g a m e s += 1
print (
” P l a y e d {} o u t o f { : . 0 f } games ( { : . 0 f }%): {} \ r ” . format (
played games , t o t a l g a m e s , played games / f l o a t ( t o t a l g a m e s ) ∗ 100
)
)
”””
The c o d e t o run t h e 10000 matches
”””