PXPDatabooks

Reference Manual
 www.flsmidth.com/automation
csc@flsmidth.com

Version Pxp V8.5

Last modified 2022-06-13

Disclaimer:

The information in this document is subject to change without notice and should not be construed
as a commitment by FLSmidth A/S. FLSmidth assumes no responsibility for any errors that may
appear in this document.

In no event shall FLSmidth be liable for direct, indirect, special, incidental, or consequential
damages of any nature or kind arising from the use of this document, nor shall FLSmidth be liable
for incidental or consequential damages arising from use of any software or hardware described in
this document.

Options specified and described in the FLSmidth documentation as part of the general description -
but initially neither ordered by the customer nor confirmed by the seller - will not commit the
supplier to any further and future supply and/or installation.

The software described in this document is furnished under a license and may be used, copied, or
disclosed only in accordance with the terms of such license.

Copyright © FLSmidth A/S. All rights reserved.

PXPDatabooks Pxp V8.5 2

Table of Contents

1. Introduction 4
1.1 Purpose 4
1.2 Intended audience 4
1.3 How to use this manual 4

2. Concept and terms 5

2.1 PXP Databooks 5
2.2 Integrated development environment (IDE) 5
2.3 Jupyter Notebook 5
2.4 Default Validation 5

3. Using PXP Databooks 6

3.1 Creating new PXP DataBook 7
3.2 Configuring PXP Databook 8
3.3 Accessing Input signals in a Python program 9
3.4 Writing Output signals from Python program 10
3.5 Writing Events and Logs 11
3.6 Installing Python packages 12
3.7 General Recommendations 12

4. Related Manuals 14
4.1 Related Manuals 14

5. Annexure 15
5.1 Annexure I 15

PXPDatabooks Pxp V8.5 3

 1. Introduction

1.1 Purpose
The purpose of this manual is to give deeper details of how to configure and use
the PXP Databooks feature.

1.2 Intended audience

This manual is relevant for the engineer/administrator of the system. The focus of
the manual is therefore on configuration and usage of the PXP Databooks feature.

1.3 How to use this manual

The first part is a brief description of the concepts and terms involved in the
system. The understanding of these concepts and terms is crucial for the efficient
use of the following sections.
Subsequently, a number of sections describe the procedure to be followed by the
user to perform a specific task e.g. (“find configuration of….”). These descriptions
are set up in a step-by-step structure.

PXPDatabooks Pxp V8.5 4

 2. Concept and terms

2.1 PXP Databooks

A collaborative framework oriented to skilled process specialists, engineers and
data scientists. It will enable them to build together new solutions using Python
and deploy into the control strategy in real time.

Furthermore, it will facilitate inference of machine learning or deep learning

models already trained offline in other environments.

2.2 Integrated development environment (IDE)

An integrated development environment (IDE) is a software application that
provides comprehensive facilities to computer programmers for software
development. An IDE normally consists of at least a source code editor, build
automation tools and a debugger.

2.3 Jupyter Notebook

The Jupyter Notebook is an open-source web application that allows you to create
and share documents that contain live code, equations, visualizations and
narrative text. Uses include: data cleaning and transformation, numerical
simulation, statistical modeling, data visualization, machine learning, and much
more.

2.4 Default Validation

This describes the general validation that is allied to all input signals and code in
the Databook. The validation will start at the top right corner and if no errors
exist propagate to the 'normal' state. When errors are encountered in the code,
then the default validator propagates to Error state. Input problems can be
missing links, one of the Input is in Invalid state or inconsistent configuration.

Default Validator
Bugs in Code?

Input Problem? Error (0)

Normal (2) Input Problem (1)

PXPDatabooks Pxp V8.5 5

3. Using PXP Databooks
PXP Databooks is a collaborative framework oriented to skilled process specialists,
engineers and data scientists. It will enable them to build together new solutions
using trendy programming languages (starting with Python in v8.5) and deploy
into the control strategy in real time.
Furthermore, it will facilitate inference of machine learning or deep learning
models already trained offline in other environments.

The Jupyter notebook IDE is integrated into the PXP Databooks, where it can be
used for Creating, Editing or Debugging the Python program. Users will be able to
use the complete features of the Jupyter notebook in the PXP Databooks.

More information on the Jupyter notebook can be found here.

It is to be noted that the Jupyter notebook in the PXP Databooks is only used as
an IDE, the execution of the Databooks at runtime (i.e., during PXP Application
execution) the PXP Databooks are executed on Python 3.8.6 runtime
environment.

Most of the 3rd party packages used for building the AI/ML solutions are included
in the PXP Databooks by default. Moreover, users will be able to install their own
packages or 3rd packages on the environment.

PXPDatabooks Pxp V8.5 6

 3.1 Creating new PXP DataBook
The PXP Databooks are designed in a similar way as softsensors (Calculator),
where the Databooks can be created from Custom components in a PXP
Application.
To Create new Databook, navigate to the Custom components section, use
symbol and provide the relevant naming to the Databook. Special characters or
Spaces are not allowed in the naming of the Databooks.

To delete a Databook, select the Databook that needs to be deleted and click on
the symbol. An existing Databook can also disabled by unchecking on the
check box. Once the Databook is disabled, it will not be executed anymore, and it
can be re-enabled later at anytime.

Once the Databook is created, you can navigate to the Databook present in the

custom component section and you will be able to find the options to Edit,

Export and Import a Databook.

When you export a Databook, the Python program is exported along with the
Input and Output variables section to an XML. You will be able to import back the
XML into the Databook again. While exporting/importing the Input variable links
are not included. So, users need to configure the input variable links once they
import an exported Databook.

When a new Databook is created, it will be created with a default Jupyter

notebook template. The default template contains the example code defining how
to Read and Write data between the Python and PXP layer.

There is a configurable parameter “Max execution time” on the top menu bar of
the Databook. It is defined in seconds with a default value as 5. When the

PXPDatabooks Pxp V8.5 7

 Databook takes more time to execute than this parameter, then the Default
Validator of the state changes to Error.

If a Databook takes either more time to execute than the Max execution time or
fails to execute due an error in the python program for 3 consecutive executions,
then the Databook is blocked for execution. All the Blocked Databooks are auto
unblocked every 30 min. However, if the users make any changes and saves the
application, then the all the blocked Databooks are unblocked.

3.2 Configuring PXP Databook

Similar to Softsensors, the Input and Output variables can be defined. Users will
be able to link the outputs of PXP Components layer as Inputs in the Databooks.
The outputs from the Databooks can be consumed either in Group Interlocks or
Decision tree or Controller or in any of the Core features (Eg: Mimics, Trend etc.).

The Databook and Application needs to be saved first to be able to retrieve values
in the python program, whenever a new Input/Output variables are created or
existing ones are modified.

Currently, the following Data types are supported for the Input variables

Input Variables Output Variables

Spot Values Historical Values Spot Values
Data Real, Boolean, RealTS, BooleanTS Real, Boolean,
Types Integer and Time and IntegerTS Integer and Time
Note: TS stands for Timeseries.

PXPDatabooks Pxp V8.5 8

 The Historical (Timeseries) values are retrieved from the Core value database.
So, in order to use a PXP signal as a Timeseries signal in PXP Databooks, the
“Write to Core” has to be enabled for the signal.

The PXPCommon Python package’s notebook class facilitates for Input & output
operations between PXP and Python layer.
The notebook class in the PXPCommon has the following methods
Read()
Read_Timeseries()
Write()
Write_Trace()
Trigger_InfoEvent()
Trigger_WarningEvent()
Trigger_ErrorEvent()

To use any of the above methods, the notebook class must be inherited while
passing the Databook name as argument to the class.

Obj = PXPCommon.notebook(‘KL1.Custom.PXPDataBook’)

Please Note: When the PXP Application is cloned with different application name,
the Databook name in the above Databook code is not changed to the newer
application name. So, users must correct the Databook name in all the PXP
Databooks present in the application after importing the new cloned application.

3.3 Accessing Input signals in a Python program

The PXP Databooks currently supports Reading of Spot values and Historical
values (Timeseries).
Accessing Spot Values
The Read() method present in the notebook class can be used to access the Input
variables of Real, Boolean, Integer and Time types. Users need to pass the Input
variables Designation name in the form of a list, as arguments to the Read()
method.

input_tags = ["InputReal", "InputBoolean"]

response=obj.Read(input_tags)

The Response object is a Python Named Tuple which contains the retrieved values
in the same variables passed as arguments to the Read method. The values from
the response object can be accessed by response.<variable name>

InputReal = response.InputReal

Please note: All the values of variables of type will be in string format. You can
convert it to Float, Int or Boolean by using the inbuilt Python functions. The Time
type will give the value in Seconds.

Accessing Historical Values

The Read_Timeseries() method in the notebook class can be used to access the
input variables of RealTS, BooleanTS and IntegerTS types. Currently, PXP
Databooks supports retrieval for 60 historical values for a variable.

PXPDatabooks Pxp V8.5 9

 The below arguments needs to be passed as arguments to Read_Timeseries().
Tags : A list of Input variable names in Python list format.
EndTime : The end time for the data retrieval period in the UTC DataTime format.
The Timeseries data will be retrieved until the timestamp defined. The utcnow()
method is datetime module can be used for getting the current time.
IntervalLength : The Interval Length is used to calculate the Start Time for the
Data retrieval period. The IntervalLength has to be defined in Minutes.
AggInterval : The Values retrieved are aggregated (Averaged) based on the
Aggregation Interval specified. It must be defined in seconds.

Tags= ["InputRealTimeSeries"]
EndTime=datetime.utcnow()
IntervalLength = 120 # in minutes
AggInterval = 60 # in seconds
responseTS=obj.Read_TimeSeries(Tags, IntervalLength, AggInterval,EndTime)

The above example code retrieves the data for the last 120 minutes with 60
seconds (1 min) average for the “InputRealTimeSeries” variable. However, since
only 60 values can be retrieved, the above code retrieved the last 60 minutes of
data.
The Response object is a Python Named Tuple. For each Timeseries variable, the
response has 4 properties - Values, Times, States and Length.
Where, Values property has the Values of the variable; Times has the
Timestamps in DateTime format; States has the State values of the Variable;
Length property provides the Length of Series retrieved. All the retrieved data of
timeseries variable is in list format.

The values from the response object can be accessed by response.<variable

name>_<Property>

InputRealTimeSeriesLength = responseTS.InputRealTimeSeries_Length
InputRealTimeSeriesValues = responseTS.InputRealTimeSeries_Values
InputRealTimeSeriesStates = responseTS.InputRealTimeSeries_States
InputRealTimeSeriesTimes = responseTS.InputRealTimeSeries_Times

Please note: If there is no historical data for a variable of timeseries type, then a
None type object is return instead of a list.

3.4 Writing Output signals from Python program

The Write() method present in the notebook class can be used to Write the
Values of Output variables of Real, Boolean, Integer and Time types. Users need
to pass Python Dictionary as argument to the Write() method, where the Keys of
dictionary as the Output variable Designations and the corresponding variable as
value pair to the Key.

output_tags = {"OutputReal": localOutputReal, "OutputBoolean": True}

response=obj.Write(output_tags)

Please remember, the Python variables type should match with the Type defined
in the Output variables section for the Databook. However, in case of Time type,
either the value can be in Real/integer with seconds or in time format.

PXPDatabooks Pxp V8.5 10

 3.5 Writing Events and Logs
Events
Apart from reading and writing data, the Databooks also supports the feature to
log events to the Core Event System. Currently, Databooks supports
INFORMATION, WARNING and ERROR level events.
The Trigger_InfoEvent() method in the notebook class can be used for writing
Information level event.
The Trigger_WarningEvent() method in the notebook class can be used for writing
Warning level event.
Whereas, the Trigger_ErrorEvent() method in the notebook class can be used for
writing Information level event. When an event is triggered with Error level
Users can pass the message to be displayed for the event as an argument to the
above methods. The message has to be in String format.

obj.Trigger_InfoEvent('Test Event from PXPDatabooks with INFO Level')

obj.Trigger_ WarningEvent ('Test Event from PXPDatabooks with Warning level')
obj.Trigger_ ErrorEvent ('Test Event from PXPDatabooks' with error level)

The event will be displayed in <Databookname>:<Message> format and with a

Category of ‘Pxp Databook’.
Logs
All the Errors that occur during the Execution of the PXP Databooks(Python part)
are captured in an log file. A separate log file for each application is available in
C:\Users\Public\Documents\FLSmidth\Logging files\Informations Directory. It will
be named in “Fls.Pxp.Databooks.<Appname>.log” format.

The below text is an log written to the log file when there was an SyntaxError in
the PXP Databook thk1.custom.punderflowpump1.

[2021-10-04 14:50:56,874][60009][17528] ERROR from ScriptService : An Exception occurred while

executing script <thk1.custom.punderflowpump1.ipynb>
Traceback (most recent call last):
File "C:\ProgramData\FLSmidth\Fls.Pxp\PyService\pxp.script.service.py", line 161, in create_task
exec(request.json['file'])

PXPDatabooks Pxp V8.5 11

 File "<string>", line 149
outputs={'UnderflowPumpSP':UnderflowPumpSP_Int)
^
SyntaxError: closing parenthesis ')' does not match opening parenthesis '{'

Users will also be able to place logs into this file by using the Write_Trace()
method of the notebook class. This functionality will be helpful in writing
debugging statements to capture any Errors that are occurring only during the
execution of the PXP Databook but not occurring during the debugging though
Jupyter Notebook.

obj.Write_Trace('Debug statement from PXPDataBook')

[2021-10-12 04:25:42,851][60004][21872] INFO from ScriptService : From PXPDataBook :

Debug statement from PXPDataBook

Please note: When using Trigger_xxxxEvent method while debugging in Jupyter

notebooks, these methods won’t do any thing they are skipped. This functionality
is made in order to avoid flooding the ECS event system with events while
debugging.

3.6 Installing Python packages

The Python distribution used in the PXP uses PIP as Package management
system. Users will be able to use the PIP commands to manage the packages on
the PXP server. More information on the PIP commands can be found here.
PIP will normally look for the packages on the Online Repository
https://pypi.org/simple. However, here a local directory (C:\Program
Files\FLSmidthPXPDataBooks\PIPOfflineRepository) on the PXP server is
configured as an offline repository for PIP. All the packages that are by default
included are first transferred to this repository and then installation of the
packages is initiated during the installation of PXPDatabooks Easyinstaller.
The Python packages that are installed by default along with their versions are
listed in the Annexure I.
If users want to install any of their own developed or 3rd party packages then
they can place the package along with their dependency packages in this offline
repository and initiate the PIP install/upgrade command to install the packages.
Please note: While developing solutions or own packages, users should consider
using the same versions of the packages that are installed on the PXP server. If
users use a different version of any of the default installed packages and then
upgrading/downgrading the packages may result in un-compatibility with other
solutions or packages being developed.

3.7 General Recommendations

 Users should not install/upgrade/downgrade to an alternate
version/distribution of Python other that the one supplied by FLS.

 Users should never install/upgrade the Jupyter Core python package with an
alternate version other that the one supplied by FLS.

 It is not recommended to use the Print statements during the execution of

Databooks. However, users can use print statements while debugging the
code with Jupyter notebook.

PXPDatabooks Pxp V8.5 12

  It is not recommended to use any plotting during the execution of
Databooks. Users can use plots within the Jupyter notebook however, while
saving and exiting from the Databook editor, the plots should be disabled.

 Never use functions like time.sleep(), which will pause the execution of the
Databook at runtime.

PXPDatabooks Pxp V8.5 13

 4. Related Manuals

4.1 Related Manuals

Process Expert – Editor – Reference Manual

PXPDatabooks Pxp V8.5 14

 5. Annexure

5.1 Annexure I
The list of Python packages that are installed on the PXP server by Default can be
found below.

Package Name Package Version

absl-py 0.11.0
argon2-cffi 20.1.0
astunparse 1.6.3
async-generator 1.10
attrs 20.3.0
Automat 20.2.0
backcall 0.2.0
bleach 3.2.1
cachetools 4.1.1
certifi 2020.11.8
cffi 1.14.4
chardet 3.0.4
cheroot 8.5.1
CherryPy 18.6.0
click 7.1.2
colorama 0.4.4
constantly 15.1.0
cryptography 3.2.1
cssselect 1.1.0
cycler 0.10.0
dataclasses 0.6
decorator 4.4.2
defusedxml 0.6.0
entrypoints 0.3
Flask 1.1.2
future 0.18.2
gast 0.3.3
google-auth 1.23.0
google-auth-oauthlib 0.4.2
google-pasta 0.2.0

PXPDatabooks Pxp V8.5 15

 grpcio 1.33.2
h5py 2.10.0
hyperlink 20.0.1
idna 2.10
incremental 17.5.0
ipykernel 5.3.4
ipython 7.19.0
ipython-genutils 0.2.0
itemadapter 0.2.0
itemloaders 1.0.4
itsdangerous 1.1.0
jaraco.classes 3.1.0
jaraco.collections 3.0.0
jaraco.functools 3.0.1
jaraco.text 3.2.0
jedi 0.17.2
Jinja2 2.11.2
jmespath 0.10.0
joblib 0.17.0
jsonschema 3.2.0
jupyter-client 6.1.7
jupyter-core 4.7.0
jupyterlab-pygments 0.1.2
Keras 2.4.3
Keras-Preprocessing 1.1.2
kiwisolver 1.3.1
lxml 4.6.1
Markdown 3.3.3
MarkupSafe 1.1.1
matplotlib 3.3.3
mistune 0.8.4
more-itertools 8.6.0
mpmath 1.1.0
nbclient 0.5.1
nbconvert 6.0.7
nbformat 5.0.8
nest-asyncio 1.4.3

PXPDatabooks Pxp V8.5 16

 notebook 6.1.5
networkx 2.6.3
numpy 1.18.5
oauthlib 3.1.0
opt-einsum 3.3.0
packaging 20.4
pandas 1.1.4
pandocfilters 1.4.3
parsel 1.6.0
parso 0.7.1
pickleshare 0.7.5
Pillow 8.0.1
plotly 4.13.0
portend 2.7.0
prometheus-client 0.9.0
prompt-toolkit 3.0.8
Protego 0.1.16
protobuf 3.14.0
PXPCommon 1.1.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycparser 2.20
PyDispatcher 2.0.5
Pygments 2.7.2
PyHamcrest 2.0.2
pyodbc 4.0.30
pyOpenSSL 19.1.0
pyparsing 2.4.7
pyrsistent 0.17.3
python-dateutil 2.8.1
pytz 2020.4
pywin32 300
pywinpty 0.5.7
PyYAML 5.3.1
pyzmq 20.0.0
queuelib 1.5.0
requests 2.25.0

PXPDatabooks Pxp V8.5 17

 requests-oauthlib 1.3.0
retrying 1.3.3
rsa 4.6
scikit-fuzzy 0.4.2
scikit-learn 0.23.2
scipy 1.5.4
Scrapy 2.4.1
seaborn 0.11.0
Send2Trash 1.5.0
service-identity 18.1.0
six 1.15.0
sympy 1.6.2
tempora 4.0.1
tensorboard 2.4.0
tensorboard-plugin-
wit 1.7.0
tensorflow 2.3.1
tensorflow-estimator 2.3.0
termcolor 1.1.0
terminado 0.9.1
testpath 0.4.4
threadpoolctl 2.1.0
timeloop 1.0.2
torch 1.7.0+cpu
tornado 6.1
traitlets 5.0.5
Twisted 20.3.0
typing-extensions 3.7.4.3
urllib3 1.26.2
w3lib 1.22.0
wcwidth 0.2.5
webencodings 0.5.1
Werkzeug 1.0.1
wrapt 1.12.1
zc.lockfile 2.0
zope.interface 5.2.0

PXPDatabooks Pxp V8.5 18