wlmdurdc(1M)wlmdurdc(1M)NAMEwlmdurdc - Manage duration of processes controlled by HP-UX Workload
Manager
SYNOPSIS
discovery_command_and_arguments
DESCRIPTION
======================> NOTE ON DEPRECATION <======================
The Duration Management Toolkit (DMTK) and its command are being depre‐
cated in a future release.
===================================================================
manages the duration of a process by stretching out its walltime.
This stretching can be desirable when the results of a process are not
needed for eight hours, but the process could easily finish in two
hours. By delaying the finish time, the spike in system resources for
those two hours is smoothed to cover eight hours. Without the spike,
other applications can be launched and run with more predictable per‐
formance.
This stretching of walltime can be particularly beneficial when running
a collection of related processes with vastly different durations.
After determining the process with the longest duration, you can con‐
figure all its related processes to finish within the same approximate
timeframe. Any unrelated processes on the system then have a consistent
level of performance because there is no sudden increase in system
resources when processes from the collection finish at various times.
is part of the WLM Duration Management Toolkit, or DMTK. It requires
three values on standard out in the following order from the user-sup‐
plied discovery command:
· PID of the process needing duration management
· Profile value
This value is the CPU time in integer seconds needed by the
process to finish when it has unlimited CPU resources avail‐
able.
· Desired duration of the managed process in whole seconds
(integer seconds)
You can develop the discovery command in any fashion you desire as long
as it provides the values above on standard out. For an example discov‐
ery command, see the EXAMPLES section below. For example discovery com‐
mands that are specific to SAS(R) software, see the directory
/opt/wlm/toolkits/sas/examples/.
uses these values to determine how many CPU cycles the process is cur‐
rently using, how many cycles it needs to finish, and the desired dura‐
tion to determine by when the process should get all those cycles.
Based on these values, sends a request to WLM for more CPU for the
process's workload group if the process needs more resources to finish
within the timeframe. Similarly, a request for fewer CPU resources is
sent to WLM if the process is finishing too quickly.
For optimal duration management, place each application for which
you'll be managing its duration in its own workload group. This ensures
that the resources being allocated by WLM for the application's manage‐
ment are used only by that application.
OPTIONS
Enables Profile mode. In this mode,
only uses the PID from the discovery command output. Use this
option when you want to verify:
· Applications are going into the proper workload groups
· The discovery command is picking up the correct PIDs
In this mode, WLM does not adjust the CPU allocation for any work‐
load groups with SLOs comparing metrics against 0. If you compare
a metric against a value other than 0 though, WLM may adjust the
CPU allocation for the SLO's associated workload group.
Displays version information and exits. This option overrides all other
options. Use this option only from the command line--not from
inside a WLM configuration file.
discovery_command_and_arguments
Any script or binary program that provides the PID, profile
value, and desired duration of a process on standard out for use
by
HOW TO USE DMTK (wlmdurdc)
The following steps show how to use to manage the duration of a
process:
1. Write a discovery command
The discovery commands identify and correlate each target applica‐
tion with its profile value and desired duration. These commands
must print to standard out the target application's:
· PID
· Profile value (CPU time needed by the process to finish when
it has unlimited CPU resources) in integer seconds
· Desired duration in whole seconds (integer seconds)
Many applications' discovery commands can be simple shell scripts.
Example discovery commands are included with DMTK and SASTK.
If your target application is started by a script, be sure that your
discovery command looks for the executable rather than the script
that starts it.
When first setting up a discovery command, just focus on getting the
target application's PID to standard out. For now, you can enter a
value of 0 (zero) for both the profile value and for the desired
duration:
PID 0 0
will display the profile value for the application in syslog
(/var/adm/syslog/syslog.log). You can then have your discovery com‐
mand report the target application's actual profile value and your
desired duration for the application.
For an example discovery command showing one way to get the PID, see
the EXAMPLES section below. For example discovery commands that are
specific to SAS software, see the directory /opt/wlm/tool‐
kits/sas/examples/.
2. Define WLM workload groups in which to place your target applica‐
tions
In a WLM configuration file, say configfile, define a WLM workload
group for each target application. For example, if we have two
groups GroupA and GroupB for our target applications app1 and app2,
the group definitions would be:
prm {
groups = GroupA : 1, GroupB : 2;
}
3. Define SLOs for each group
Moving to the structure in the WLM configuration file, we define
goals for each of the target applications. With duration management,
the goal is always to have the metric from be greater than 0. First,
we set up an SLO for application1:
slo app1_slo {
pri = 1;
mincpu = 5;
maxcpu = 75;
entity = PRM group GroupA;
goal = metric app1_met > 0;
}
We follow that with application2's SLO:
slo app2_slo {
pri = 1;
mincpu = 10;
maxcpu = 60;
entity = PRM group GroupB;
goal = metric app2_met > 0;
}
4. Set up wlmdurdc invocations
With the structures in place, we need to get values for the metrics
app1_met and app2_met. These values come from which bases its values
on the data from the discovery commands. We get the metric values
through invocations in structures (in the WLM configuration file)
named for the metrics:
tune app1_met {
coll_argv = wlmrcvdc
/opt/wlm/toolkits/duration/bin/wlmdurdc
discovery_command_and_args;
}
tune app2_met {
coll_argv = wlmrcvdc
/opt/wlm/toolkits/duration/bin/wlmdurdc
discovery_command_and_args;
}
The discovery commands are launched using
To see all the configuration file fragments from the previous steps
together, see the EXAMPLES section below.
5. Check the syntax of the configuration file
configfile
and fix any errors found in the syntax check
6. Start WLM
configfile
7. Run only one application that will have its duration managed
Run each application by itself so that it has unlimited CPU access.
8. Let the application complete
Once the application finishes, writes its profile value to syslog.
9. Repeat Steps 7 and 8 for each application that will have
its duration managed
10. Use the profile values from syslog
Modify your discovery command to use the profile values. Also, now
that you have the profile values, you can specify reasonable
desired durations.
You can generate "percent complete" reports for your applications
to feed to at various points during the application's duration.
would then use this information to better tune the CPU allocation
for the application's workload group. For SAS jobs, you can instru‐
ment the jobs with the macro to report "percent complete". For more
information on this macro, see hp_wlmtk_goals_report(1M).
For other applications, implement the reporting represented by the
following pseudo-code, either inside the application or as an
external utility:
PID=process_id # Get the application's PID
# wlmdurdc creates this file and checks it for data
file=/var/opt/wlm/tk_config/wlmdurdc.met.PID
point_in_dur=0 # Number points in the application's duration
# at which "percent complete" data is reported
percent=pct_complete # Indicate the percent completed
if(file exists){ # Write the data to the file
write(file, "%d %d %d\n", PID, point_in_dur, percent)
}
automatically creates the file wlmdurdc.met.PID and uses any data
present in it.
For an example of how to set up your discovery command, see the
EXAMPLES section below.
11. Start the applications in workload groups
You must start your applications in the correct group; otherwise,
WLM controls a group's CPU resources based on duration management
data for an application not in the group.
Next time you need to run the applications with duration management
in effect, start each of them with the command. To place applica‐
tions in the GroupA workload group, use the command:
% prmrun -g GroupA app1 <app1_arguments>
Similarly, to place applications in the GroupB workload group, use
the command:
% prmrun -g GroupB app2 <app2_arguments>
Now WLM and are properly configured to manage the duration of your tar‐
get applications.
NOTES
SIGTERM to discovery commands
sends a SIGTERM to its discovery command if that command is running
when exits.
Application exceeds desired duration
If an application continues running beyond its desired duration, keeps
requesting the amount of CPU for the associated workload group as it
did immediately before the desired duration elapsed.
wlmdurdc operation
works as described below:
1. Launches its discovery command
launches the discovery command specified on its command line using
2. Gets discovery data
waits for the discovery command to output data (a PID, a profile
value, and a desired duration).
3. Locates target application
locates the process matching the PID from the discovery command out‐
put. If no such PID is found, returns to
Step 1.
4. Calculates and sends request for CPU shares to WLM
uses the application's process metrics, the profile value, and the
desired duration to calculate proper requests for CPU shares. makes
these requests once per WLM interval. WLM then weighs the request
with all the requests being made for the other SLOs and grants CPU
based on priority and available CPU. continuously performs this
calculation and makes CPU requests until the target application has
completed.
5. Repeats
After the target application has completed, restarts the duration
management sequence at Step 1.
EXAMPLES
This section has two example WLM configuration files showing how to set
up the workload groups, SLOs, and statement. Following that is the
example discovery command used by the second example WLM configuration.
WLM configuration from steps above
This first configuration pulls all the configuration file fragments
from the section "HOW TO USE DMTK (wlmdurdc)" into a single file.
prm {
groups = GroupA : 2, GroupB : 3;
}
slo app1_slo {
pri = 1;
mincpu = 5;
maxcpu = 75;
entity = PRM group GroupA;
goal = metric app1_met > 0;
}
slo app2_slo {
pri = 1;
mincpu = 10;
maxcpu = 60;
entity = PRM group GroupB;
goal = metric app2_met > 0;
}
tune app1_met {
coll_argv = wlmrcvdc
/opt/wlm/toolkits/duration/bin/wlmdurdc
discovery_command_and_args;
}
tune app2_met {
coll_argv = wlmrcvdc
/opt/wlm/toolkits/duration/bin/wlmdurdc
discovery_command_and_args;
}
WLM configuration from /opt/wlm/toolkits/duration/config/
For an actual example of a WLM configuration see /opt/wlm/tool‐
kits/duration/config/duration.wlm. It shows how to use the example dis‐
covery command /opt/wlm/toolkits/duration/examples/duration.sh.
Discovery command from /opt/wlm/toolkits/duration/examples/duration.sh
The discovery command invoked in duration.wlm is available at
/opt/wlm/toolkits/duration/examples/duration.sh. Note that duration.sh
is merely a starting point; there are more sophisticated ways to col‐
lect this information.
In essence, this script:
1. Takes a workload group name on its command line
2. References a file where each line specifies:
· An executable name
· That executable's profile value
· That executable's desired duration
3. Makes a list from the file's executable names
4. Makes a list of active processes for the workload group
5. Checks the lists against each other:
Checks each active process in the workload group to see whether
there is data for it in the file from Step 2
6. Determines the process's PID if there is a match in Step 5
If Step 5 does not produce a match, the script returns to Step 3
7. Prints the process's PID, profile value, and desired duration
8. Exits
RETURN VALUE
returns an exit status of if an invalid command line is specified.
When executed as a data collector in a WLM configuration file, it exits
at wlmd shutdown/reconfiguration.
AUTHOR
was developed by HP.
FEEDBACK
If you would like to comment on the current WLM Toolkits functionality
or make suggestions for future releases, please send email to:
wlmfeedback@rsn.hp.com
SEE ALSOhp_wlmtk_goals_report(1M), syslogd(1M), wlmd(1M), wlmconf(4), wlm(5),
wlmtk(5)
HP-UX Workload Manager User's Guide (/opt/wlm/share/doc/WLMug.pdf)
HP-UX Workload Manager Toolkits User's Guide (/opt/wlm/tool‐
kits/doc/WLMTKug.pdf)
Example WLM configuration files
/opt/wlm/toolkits/duration/config/
/opt/wlm/toolkits/sas/config/
Example discovery commands
/opt/wlm/toolkits/duration/examples/
/opt/wlm/toolkits/sas/examples/
http://www.hp.com/go/wlm
WLMTK updates and information
wlmdurdc(1M)
