Control-M Architechture
Control-M Architechture
Control-M Architechture
Components
Control-M is well known for its three-tier architecture. By utilizing networking
technology, Control-M components among the three tiers can communicate with each
other freely, therefore work together to provide cross platform job submission and
tracking, and at the same time allow batch workload to be monitored and managed
from a centralized location.
Control-M/Enterprise Manager sits at the top layer. It provides the backend of
graphical user interface and administration facilities. The middle tierControlM/Server is the schedule engine that performs the actual job submission and tracking.
And the bottom tier is Control-M/Agent that runs on different machines to handle job
submission requests from Control-M/Server. Control Modules and Agentless hosts are
also part of the bottom layer, but just under Control-M/Agent. This is because Control
Modules are only to be installed as add-on components of Control-M/Agents and
Agentless hosts are managed by Control-M/Server through selected ControlM/Agent(s).
The combination of the three tiers is considered as a complete Control-M environment.
Each environment can have exactly one Control-M/Enterprise Manager connecting
with one or many Control-M/Servers, and each of the Control-M/Server can connect
with hundreds or thousands of Control-M/Agents as well as optionally some CMs
attached or Agentless remote hosts.
With this highly scalable architecture, users can choose to run their batch environment
a single scheduling engine or divide the workload onto multiple scheduling engines,
but still have the ability to monitor and manage the jobs within a single GUI as long
as every Control-M/Server configured to communicate with the same ControlM/Enterprise Manager.
Control-M/Enterprise Manager
There are two sub-layers within the Control-M/Enterprise Manager layer Control-M/Enterprise
Manager GUI Client components which have already been introduced and Control-M/Enterprise
Manager Server components. Unlike the GUI client components which can only be installed and
running on Windows based machines, Control-M/Enterprise Manager Server components can be
installed and run on Windows, Linux, or Unix based machines.
6.4.01, BMC shipped Control-M with Oracle and Sybase databases (database licenses inclusive).
The user can choose to use the BMC supplied database or their own Oracle or Sybase. For ControlM/Enterprise Manager running on Windows machines, users also have the option to install it with
their own MS SQL database. Since version 6.4.01, BMC added support for BMC supplied
PostgreSQL database and excluded Sybase from its software supply, but Control-M/Enterprise
Manager was still allowed to install with user owned Sybase. In version 7, only PostgreSQL is
supplied, which means for users who wants to run Control-M/Enterprise Manager on an Oracle or
Sybase database, they have to provide th eir own.
Control-M/Enterprise Manager Server components are a set of processes that play different roles
at the back end. These processes are:
Naming Services
GUI Server
Naming Service
The Naming Service provides the foundation for the majority of communications between ControlM/Enterprise Manager components. It is in fact an embedded TAO (The Ace Orb) naming service
an open-source C++ implementation of CORBA (Common Object Request Broker Architecture)
Note
From Wikipedia: The Common Object Request Broker Architecture (CORBA) is a standard defined
by the Object Management Group (OMG) that enables software components written in multiple
computer languages and running on multiple computers to work together (that is, it supports
multiple platforms). CORBA is useful because it enables separate pieces of software written in
different languages and running on different computers to work together like a single application or
set of services. More specifically, CORBA is a mechanism in software for normalizing the methodcall semantics between application objects residing either in the same address space (application)
or remote address space (same host, or remote host on a network).
Control-M Configuration Server
Control-M Configuration Server component was introduced in Control-M/Enterprise Manager
6.3.01 as a background service to support CCM GUI front-end and perform backend administration
tasks like cleaning hysterical data from the database on regular intervals.
Control-M/Enterprise Manager Configuration Agent
The role of Control-M/Enterprise Manager Configuration Agent hasn't changed much since the
earlier versions. It manages the states of a number of Control-M/Enterprise Manager Server
components. Those managed components are: GUI Server, Gateway processes, Global Condition
server, and Global Alert Server. Control-M/Enterprise Manager Configuration Agent does the startup, shutdown, and recycle for these components according to the process Desire State flag that is
set by the user in CCM.
Unlike other Control-M/Enterprise Manager Server components, GUI Server has a lot more
responsibilities. For GUI clients, it does most of the heavy lifting part of displaying jobs (reduce the
workload on each GUI clients) as well as handles user requests that come from each active GUI
client session. For Control-M Desktop, GUI Server handles user actions on job definitions, such as
loading job definitions from EM database and writing job definition modifications to EM database.
It also handles other tasks such as EM security (GUI client authentication), EM command line utility
requests, and collecting GUI client auditing data.
Gateway process (GTW)
As we already know, Control-M/Enterprise Manager does not handle the actual job scheduling, the
actual scheduling engine that submits jobs and tracks job execution statues is Control-M/Server.
Gateway process enables the communication between the two layers so jobs can be displayed on
GUI clients in real time and user actions can be immediately passed to Control-M/Server.
Dedicated gateway process is required to be defined for each Control-M/Server to handle the
communication. Within the Control-M/Enterprise Manager layer, each gateway process has direct
access to the EM database. On the remote side, a connection is permanently established to the
target Control-M/Server.
Global Alert Server (GAS)
The Control-M/Enterprise Manager GUI Client provides an alert window for the user to track
exceptions in the batch environment. Such exception can be a job's failure, a job has been running
too long, or Control-M/Agent has gone unavailabe. The alert information gets transmitted from
Control-M/Server to Control-M/Enterprise Manager by the gateway process. Global Alert Server is
in charge for processing and presenting these alert notifications.
Global Condition Server (GCS)
In a multi Control-M/Server environment, jobs resided in different Control-M/Servers might be
related and to be executed in a pre-defined logical order. the Global Condition Server handles the
job dependencies between jobs running on different datacenters (Control-M/Servers).
In Control-M, a job condition is a string defined as part of the job definition, the Global conditions
are distinguished from normal conditions by global condition prefix. User needs to predefine the
condition prefix, as well as its source datacenter and destination. During runtime, the Global
Condition Server recognizes conditions that are newly generated by jobs in the source datacenter
(Control-M/Server) and distributes them to the destination datacenters (Control-M/Servers)
according to the condition prefix definition.
Control-M Web Server
Control-M Web Launch (that is, web deployment of Control-M/Enterprise Manager GUI) and
Business Impact Manager web interface. During late 2011, BMC released a new Control-M add-on
product - Control-M Self Service. It is a web-based GUI that presents the batch workload in
services view for the business level users. Those users can focus on business critical batch at
service level without requiring a Control-M/EM GUI installation.
The web-based GUI has also been optimised to be displayed on mobile devices such as iPhone or
iPad.
Control-M/Server
The Control-M/Server is the heart of Control-M Workload Automation. In the background, it
submits jobs onto job execution hosts according to their dependencies and priorities and tracks
their status until the execution completes. Just the same as Control-M/Enterprise Manager,
Control-M/Server for distributed systems stores information in its own database. (The reason we
specifically stated distributed systems here, is because Control-M for mainframe stores
information in data files rather than a database.) The stored information includes static job
definitions, active jobs (AJF), job execution statistics, job logs, and so on.
Control-M/Server for distributed systems can be installed on Windows, Unix, or Linux machines.
Control-M/Enterprise Manager and Control-M/Server can share the same machine and same
database, but users normally separate the two for production environment to get better
performance and in case of increase in load in the future.
Control-M/Server processes
Control-M/Server is a collection of 9 core background processes, plus 1 configuration agent
process (introduced since version 6.3.01). Each process has its own role, such as being in charge
of internal communication, communication with Control-M/Enterprise Manager, communication
with Control-M/Agents, or handling job submission, tracking, and logging. Each process also
generates its own log file for error diagnosis and monitoring. These processes are:
SU: Supervisor
LG: Logger
SU: Supervisor
As we can see within Control-M/Server there are another 8 core processes (excluding SU and CA)
and each of these processes are required for Control-M/Server's normal functioning. Instead of
letting users manually start or stop these processes one by one and monitor their running, ControlM/Server provides the supervisor SU process to manage these processes. During a ControlM/Server start-up, the SU process gets started first and it brings up the other 9 processes. While
the Control-M/Server is running if any process is exited for any reason, the SU process will detect
the absence of the process and try to bring it up again. There is a max retry parameter to determine
how many times the SU will re-try to bring up the exited processes if the processes keep on failing.
Once the max retry is reached, SU will stop trying and bring all other processes down. It will shut
down the entire Control-M/Server (SU process certainly knows the true meaning of Insanity: doi ng
the same thing over and over again and expecting different results). During a normal shut down of
Control-M/Server, the shut down request gets sent to the SU process, and the SU process will then
stop all other active core processes and follow by exiting itself.
SL: Job Selector
SL process is one of the most critical processes in Control-M/Server (in my opinion). As we can tell
from its name, SL process selects the right job for execution. It allocates resource for jobs that are
eligible to run and releases the resource after the job is completed. It also handles part of the post
processing work for each job. Since Control-M/Server 6.2.01, SL process became a multi-threaded
process for performance improvement. As you can imagine, if SL process fails, all jobs that are
waiting to be scheduled in Control-M will be in a hung state even if all prerequisite conditions for
the job are met.
There is a lot more to talk about the SL process. We will take up the whole chapter if we go into
every detail. Luckily, the internal algorithm of SL process is well designed, therefore, ControlM/Server can handle thousands of job submissions concurrently without us worrying about how it
works internally.
TR: Job Tracker
The TR process is also a very important process. It tracks the states of all submitted jobs. Without
TR process, we simply don't know what happened to a job after it got submitted. TR process gets
job status update from Control-M/Agent in real time and updates the job's status in Control-
M/Server database. By doing so, the SL process can perform post processing for that finished job
and continue submitting new jobs as the down flow jobs' prerequisites are met.
Apart from relying on Control-M/Agents to provide job updates in real time, TR process also
performs a Track All periodically to all running jobs. This feature is to prevent in events of job
status updates were not received due to connection with Control-M/Agent was temporarily lost or
TR process (or Control-M/Server) was down just at the time when the original job status updates
were sent from Control-M/Agent.
NS: Agent Communication Process
Again, this is another critical process that is related to job submission and status tracking. In
saying that, the NS process doesn't have either of these two roles, in fact it handles all
communications between Control-M/Server processes and Control-M/Agent. It also maintains the
communication with Control-M/Agents check if they are available or not, and update the status
into Control-M/Server's job execution node availability record.
CE: New Day and EM Communication Process
The CE process is newly introduced for version 7, in fact it is the combination of 2 processes (CD
and CO) from the previous versions.
Secondly, the CE process is in charge of Control-M/Server's daily refresh New Day procedure
(NDP). We will introduce NDP in detail when reviewing life cycle of a job during Chapter 4, Create
and Manage Batch Flows with Control-M GUI.
CS: Server Process
The CS process handles GUI requests, such as holding a job, killing a job, and reruning a job. Up to
10 CS processes are allowed for handling multiple requests at the same time. The number of active
CS processes can be hardcoded by the user or left to Control-M/Server to manage. From the user's
point of view, they may feel a better response time during peak usage when multiple CS processes
are presented. We will look at how to change the value of this parameter in Chapter
5, Administrating the Control-M Infrastructure.
LG: Logger Process
The LG process does event logging for the Control-M/Server. In additional, it handles utility
requests from Control-M/Agent there are a number of Control-M/Server command line utilities
that are allowed to be invoked from Control-M/Agent. We can imagine those Control-M/Server
processes on Control-M/Agent as soft links rather than performing the real task. LG process
handles these request by triggering the actual process residing on the Control-M server and sends
the execution result back to Control-M/Agent through the NS process.
WD: Watchdog Process
The WD process can do many different things, but these things are all optional.
Firstly, the WD process monitors Control-M processes and raises an alert when there's a problem.
This feature can be turned on or turned off through configuration parameters.
Thirdly, the WD process performs Control-M/Server User Exists. We will introduce "User Exists"
in Chapter 6,Advanced Batch Scheduling and Management.
RT: Internal Communication Router
The RT process is also one of the most important processes within Control-M/Server however
hardly requres our attention. It acts like a network router to reduce the number of communication
channels between Control-M/Server processes, therefore reducing the complicity of ControlM/Server inter-process communication. It listens on an Internal Process Communication (IPC) port
that allows all the Control-M/Server processes we talked about to connect with each other.
Control-M/Agent
Control-M/Agent is a collection of processes that are constantly running on the job execution host
to act on job submission requests from Control-M/Server. Control-M/Agent submits the actual
executable program to operating system for execution, monitors the program execution, and
provides the execution result back to Control-M/Server. Control-M/Agent is also the base of
Control Modules, as well as handles the communication between Control-M/Server and Agentless
remote hosts.
Users can choose from two connection options for the communication between Control-M/Agent
and Control-M/Server transient and persistent. With transient connection method, a new
connection is created each time when Control-M/Server needs to talk to Control-M/Agent or vice
versa. With persistent connection mode, a permanent connection is maintained between ControlM/Server and Agent. Persistent connection option was provided since version 6.2.01 to allow
Control-M/Server and Agent connecting over a firewall that only allows communication from one
side. It turns out that persistent connection also offers better performance when Control-M/Server
and Agent are communicating in SSL mode.
Before version 6.2.01, Control-M/Agent on the Unix/Linux systems relies on operating system's
INETD daemon for listening on server-to-agent communication. For each request that comes from
Control-M/Server, the INETD process will create a new instance of the AG process to handle it.
The TR process also performs post processing related to job's output file, by scanning the file and
searching for required strings that Control-M/Server will later trigger post processing events for.
As we mentioned while introducing Control-M/Server's LG process, there are a number of ControlM/Servercommand line utilities that are allowed to be invoked from Control-M/Agent. UT process
is in charge for processing and submitting such utility requests from Control-M/Agent to ControlM/Server.
Agentless Technology
Job Submission to a remote host without Control-M/Agent installation is achieved by utilizing
existing
technologies
that
come
with
operating
systems
Secure
Remote
Shell
(SSH) and Windows Management Instrumentation (WMI). Control-M/Server submits jobs to remote
hosts through appointed Control-M/Agents, then the Control-M/Agent opens the SSH or WMI
Connection with the remote host and forwards the job submission requests. Multiple ControlM/Agents can be defined for submitting jobs to the sameAgentless Remote Host for the purpose of
workload sharing and high availability.
From the user's point of view, defining and monitoring jobs running on an Agentless Remote
Host has very little difference as compared to working with Control-M/Agent hosts. From a
management point of view, the connection status of Agentless Remote Host can be monitored from
CCM. Control-M also allows users to convert a Control-M/Agent host into an Agentless Remote
Host or vice versa.
Control-M/Control Modules
We briefly introduced optional Control-M/Control Modules from the functionality aspects. There are
two types of Control Modules (CM) OS CM and Application CM.
In fact, all jobs submitted are handled by CMs. Normal jobs such as executing a command or
running an executable are handled by the OS CM, which is came as a part of Control-M/Agent. The
Control Modules we discussed earlier are the application CMs CM for SAP, CM for Oracle EBusiness Suite, CM for PeopleSoft, and so on. Application CMs are extensions of Control-M/Agent,
therefore has to be installed on top of Control-M/Agent. In other words, Agentless remote hosts
cannot handle application CM jobs.
Application Owners: Keep track of their batch job running at service level.
failed
jobs,
have
to
work
with
the Developers and Schedulers for RCA (root cause analysis), fix the problem, and
provide summary to Incident Management and the Application owners.
In a smaller environment, the job roles of scheduler, operator, and administrator could
come down to one person or a small group of people in the operations team.