Skip to main content

EasySMF:JE Installation

This document serves as an outline for the installation procedures. Full installation documentation can be found in the Installation and Users Guide.

Prerequisite: Java 8

Java 8 added greatly improved time classes in the java.time package. EasySMF:JE uses these extensively so Java 8 is required.

Prerequisite: JZOS Toolkit

Java SMF uses JZOS functions to read and convert data.

The JZOS Toolkit allows you to run your Java program as a batch job using DD statements to specify the SMF datasets to be read.

If the JZOS Toolkit is not already set up, install it according to the instructions in the Installation and User’s Guide. This involves copying the load module from <JAVA_HOME>/mvstools into a PDS/E, and copying and customizing the PROC and JCL from <JAVA_HOME>/mvstools/samples/jcl.

JCL to run Java programs is distributed with the JZOS Toolkit in <JAVA_HOME>/mvstools/samples/jcl/JVMJCL80.

Java SMF also uses functions from ibmjzos.jar. When developing or running Java SMF programs on platforms other than z/OS, you will need to copy this jar (ibmjzos-n.n.n.jar, where n.n.n is the version) from your z/OS Java installation to the build/class path.

EasySMF:JE

EasySMF:JE is distributed as a zip file containing the EasySMF jar, EasySMF Javadoc and EasySMF samples or as a TSO TRANSMIT format file with the same items plus a JCL library with installation JCL and sample PROCs.

On z/OS

Upload the TSO TRANSMIT file in binary format to a FB 80 dataset on the mainframe.

Use TSO RECEIVE to restore the installation dataset.

The #INSTALL member contains one job which will restore the jar files and samples to HFS files. Set the target location for the files, update the source to point to the installation dataset and run the job.

There are also several IVPs that can be run to verify the installation, and JCL procedures that can be customized to simplify the use from batch JCL.

Other platforms

Unzip the file and add the EasySMF jar to the Java class path or build path.

The class path and build path also need to contain the JZOS jar. This should be downloaded from your z/OS system (observe any IBM licensing restrictions).

Running .class files

When running .class files, add the jars to the CLASSPATH.

Using the JVMJCL80 sample JCL:

# Customize your CLASSPATH here APP_HOME=/u/ibmuser/java CLASSPATH=$APP_HOME:"${JAVA_HOME}"/lib:"${JAVA_HOME}"/lib/ext

CLASSPATH="$CLASSPATH":/<directory>/easysmf-je-n.n.n.jar

Add Application required jars to end of CLASSPATH

for i in "${APP_HOME}"/*.jar; do     CLASSPATH="$CLASSPATH":"$i"     done export CLASSPATH="$CLASSPATH":

where <directory> is the HFS directory containing the jar files.

If they are in the APP_HOME directory the JVMJCL80 script will add them to the CLASSPATH automatically.

Running .jar files

Any classpath specified by environment variables or Java arguments is ignored when running a jar file. The classpath for dependencies can be specified in the jar file manifest.

The sample program jar files specify lib/ as the classpath. The EasySMF:JE  jar needs to be in the lib subdirectory, relative to the location of the sample jar.

Running the Sample Programs

To run the sample programs, copy the jar file (or class file if you compiled from source) to a HFS directory. Copy the EasySMF:JE  jar to the lib subdirectory.

Make the following changes to JVMJCL80:

  • Change JAVACLS on the EXEC statement, where <directory> is the location of the jar file to run

//JAVA EXEC PROC=JVMPRC80,                                 // JAVACLS='-jar /<directory>/recordcount.jar'

where recordcount.jar is the sample you want to run.

  • Add the INPUT DD statement to the step

//INPUT     DD DISP=SHR,DSN=smf.dataset      

Submit the JCL.

Java SMF Sample 1 : SMF Records by type and subtype

This sample program reads all the SMF records from a file or dataset, and for each record type and subtype lists the number of records, total bytes and percentage of the total.

These examples are not intended to suggest any particular coding style or structure for Java programs. They are intended to show what is possible and illustrate how to access various features of the Java SMF API. In fact I have deliberately ignored some Java conventions in these samples (e.g. encapsulation in the inner classes) to cut the total amount of code and concentrate on illustrating the SMF processing facilities.

To start with, I’ll skip the most of the Java stuff and concentrate on the SMF processing.

Reading SMF data

The SmfRecordReader class provides the facility to read SMF records. It uses a JZOS RecordReader internally to read from a DDNAME or reads directly from a stream.

A stream can be any type of InputStream. Typically it would be a file but it could also be some sort of network stream, or you could chain streams together to read compressed data etc. The stream must contain the record descriptor words (RDWs) so the record lengths can be determined.

SmfRecordReader implements AutoCloseable which means that it can be used in a try-with-resources block to automatically close the reader when exiting the block.

To open a DD for reading:

try (SmfRecordReader reader = SmfRecordReader.fromDD("INPUT")) {
...
}

To read from a file:

try (SmfRecordReader reader = SmfRecordReader
.fromStream(new FileInputStream(
"C:\\Users\\Andrew\\Documents\\SMF Data\\weekly.smf"))) {
...
}

The samples handle both cases by accepting the filename as an argument to the program. If no filename is provided they open the INPUT DD:

try (SmfRecordReader reader =
args.length == 0 ?
SmfRecordReader.fromDD("INPUT") :
SmfRecordReader.fromStream(new FileInputStream(args[0])))
{
...
}

This makes it simple to develop and test on the workstation, then move the program to z/OS to run as a batch job.

Processing the data

for (SmfRecord record : reader)
{        
// do stuff with record...
};

SmfRecordReader implements the Iterable<SmfRecord> interface. The loop above will read and process each record until the reader indicates that no more records are available. Each iteration provides the next SmfRecord in the record variable.

Accessing information from the record

Record information can be accessed using record methods:

Integer key = (record.recordType() << 16)
+ (record.hasSubtypes() ? record.subType() : 0);
int length = record.recordLength();

That’s all the SMF processing this program does.

The Java Stuff

You could probably store record type and subtype information in a very sparse 2 dimensional array, but for reduced memory usage this program uses a HashMap with the type and subtype combined into an Integer key, and an object containing the statistics as the value.

Map<Integer, RecordStats> statsMap =
new HashMap<Integer, RecordStats>();

RecordStats is the class that keeps the counters for a type/subtype combination.

The processing is simple:

For each record:
1. Get the type/subtype key
2. Try to get an existing statistics entry for the key
3. If there is no existing entry, create a new RecordStats entry

Integer key = (record.recordType() << 16)
+ (record.hasSubtypes() ? record.subType() : 0);

RecordStats stats = statsMap.get(key);
if (stats == null)
{
statsMap.put(key, new RecordStats(record));
}
else
{
stats.add(record);
}

SMF record type is a 1 byte field and subtype is 2 bytes, so they can both be combined into a 4 byte Integer to form the key.

Output

The writeReport method produces the output.

The statistics entries are sorted by the largest to smallest number of bytes. First the map entries need to be put into a List for sorting:

List<Entry<Integer, RecordStats>> results =
new ArrayList<Entry<Integer, RecordStats>>(statsMap.entrySet());

Then the Collections.sort method is used to sort the list. We can provide a Comparator in the call to sort to sort by anything we like. In this case we compare the bytes in the RecordStats object.

Collections.sort(results,
new Comparator<Entry<Integer, RecordStats>>()
    {
     public int compare(
         Entry<Integer, RecordStats> s1,
            Entry<Integer, RecordStats> s2)
            {
             return Long.compare( // reversed to sort descending
                 s2.getValue().bytes,
                    s1.getValue().bytes);
            }
    });

Then System.out.format is used to write headings and data. The output:

Type  Subtype     Records          MB     Pct       Min       Max       Avg
   74        5       60384        1330    23.5      4668     32484     23107
   30        4      857136        1204    21.3      1137     32740      1473
   30        3      853873        1195    21.1       626     32740      1468
   74        1       37536        1128    19.9     17668     32632     31517
   30        5       97123         236     4.2      1137     32740      2558
...

The complete Java program:

import java.io.FileInputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;

import com.blackhillsoftware.smf.SmfRecord;
import com.blackhillsoftware.smf.SmfRecordReader;

public class recordcount
{
public static void main( String[] args ) throws IOException
{
// use a HashMap to keep the statistics
Map<Integer, RecordStats> statsMap =
new HashMap<Integer, RecordStats>();

// Read from a DD or file
try (SmfRecordReader reader =
args.length == 0 ?
SmfRecordReader.fromDD("INPUT") :
SmfRecordReader.fromStream(new FileInputStream(args[0])))
{

// SmfRecordReader implements Iterable<SmfRecord>, so we can
// simply iterate to read SmfRecords
for (SmfRecord record : reader)
{
Integer key = (record.recordType() << 16)
+ (record.hasSubtypes() ? record.subType() : 0);

// try to retrieve an existing entry
RecordStats stats = statsMap.get(key);
if (stats == null) // none -> create new
{
statsMap.put(key, new RecordStats(record));
}
else // otherwise add this record to existing
{
stats.add(record);
}
};
}
writeReport(statsMap);
}

private static void writeReport(Map<Integer, RecordStats> statsMap)
{
// Turn the HashMap into a list so it can be sorted
List<Entry<Integer, RecordStats>> results =
new ArrayList<Entry<Integer, RecordStats>>(statsMap.entrySet());

// get the total bytes from all record types
long totalbytes = 0;
for (Entry<Integer, RecordStats> entry : results)
{
totalbytes += entry.getValue().bytes;
}

// sort by total bytes descending
Collections.sort(results,
new Comparator<Entry<Integer, RecordStats>>()
{
public int compare(
Entry<Integer, RecordStats> s1,
Entry<Integer, RecordStats> s2)
{
return Long.compare( // reversed to sort descending
s2.getValue().bytes,
s1.getValue().bytes);
}
}
);

// write heading
System.out.format("%5s %8s %11s %11s %7s %9s %9s %9s%n",
"Type",
"Subtype",
"Records",
"MB",
"Pct",
"Min",
"Max",
"Avg");

// write data
for (Entry<Integer, RecordStats> entry : results)
{
System.out.format("%5d %8d %11d %11d %7.1f %9d% 9d% 9d%n",
entry.getKey().intValue() >> 16,
entry.getKey().intValue() & 0xFFFF,
entry.getValue().count,
entry.getValue().bytes / (1024*1024),
(float)(entry.getValue().bytes) / totalbytes * 100,
entry.getValue().min,
entry.getValue().max,
entry.getValue().bytes / entry.getValue().count);
}
}

/**
* Statistics for a type/subtype combination
*/
private static class RecordStats
{
public RecordStats(SmfRecord record)
{
count = 1;
bytes = min = max = record.recordLength();
}

int count;
long bytes;
int max;
int min;

public void add(SmfRecord record)
{
count++;
int length = record.recordLength();
bytes += length;
if (length < min)
min = length;
if (length > max)
max = length;
}
}
}

Java SMF Sample 2 : Prime Shift Top CPU Consumers

This second sample illustrates the use of java.time functions to report based on time and the day of the week. It reports the top 10 CPU consuming jobs by jobname between 8:30 am and 5:30 pm for each weekday.

Opening the file and reading the records work the same way as the previous sample. However in this case the processing is slightly more complicated.

Program Setup

We use a HashMap<String, JobData> to map job names to the cumulative job information. We want to report the days of the week separately, so we create a List containing a Map for each day of the week. The days of the week are numbered 1-7. We create a list of 8 entries so we can use indexes 1-7, leaving entry 0 unused.

List<HashMap<String, JobData>> jobsByDay =
        new ArrayList<HashMap<String, JobData>>();
        
for (int i=0; i <= 7; i++)
{
     jobsByDay.add(new HashMap<String, JobData>());
}

Java Time LocalTime variables are used to define the times of day we are interested in:

LocalTime primeStart = new LocalTime(8, 30);
LocalTime primeEnd = new LocalTime(17, 30);

Processing the records

Firstly we check whether the record is one we are interested in: type 30 subtype 2 and 3, written during the prime shift we defined.

if (record.recordType() == 30
&& (record.subType() == 2 || record.subType() == 3))
{               
DayOfWeek recordDayOfWeek = record.smfDate().getDayOfWeek();
    LocalTime recordTime = record.smfTime();
                
    if (recordDayOfWeek.getValue() >= DayOfWeek.MONDAY.getValue()
     && recordDayOfWeek.getValue() <= DayOfWeek.FRIDAY.getValue()
        && recordTime.isAfter(primeStart)
        && recordTime.isBefore(primeEnd))
    {
     // process this record...
    }
}

For each record that matches we need to:

  1. Create a Smf30Record from the SmfRecord, to give access to the type 30 information.
  2. Get the collection of jobs for this day of the week.
  3. Extract the job name from the Identification Section, and look for existing statistics for the job name.
  4. Update or add job statistics with information from the Processor Accounting Section.

Each type 30 record has an Identification Section, so the r30.identificationSection() returns that section directly. However, records may or may not have a Processor Accounting Section. The method r30.processorAccountingSections() returns a List which contains 0 or 1 entry. This means that we can iterate over the list and it will happily process 0 sections from records without a processor accounting section. An explicit check for the number of sections isn’t required, and the method of processing can be consistent for all section types where the number of sections is variable.

Smf30Record r30 = new Smf30Record(record);
              
for (ProcessorAccountingSection proc : r30.processorAccountingSections())
{
Map<String, JobData> day = jobsByDay.get(recordDayOfWeek.getValue());
                  
    String jobname = r30.identificationSection().smf30jbn();
    JobData jobentry = day.get(jobname);
    if (jobentry == null)
    {
     day.put(jobname, new JobData(proc));
    }
    else
    {
     jobentry.add(proc);
    }
}

Collecting the CPU times

We collect each of CP, zIIP and zAAP time from the processor accounting section.

private static class JobData {
JobData(ProcessorAccountingSection proc)
    {
     add(proc);
    }
        
    public void add(ProcessorAccountingSection proc)
    {
     cpTime = cpTime.plus(proc.smf30cpt())
            .plus(proc.smf30cps())
            .minus(proc.smf30TimeOnIfa())
            .minus(proc.smf30TimeOnZiip());
        zaapTime = zaapTime.plus(proc.smf30TimeOnIfa());
        ziipTime ziipTime.plus(proc.smf30TimeOnZiip());
    }
        
    Duration cpTime = Duration.ZERO;
    Duration ziipTime = Duration.ZERO;
    Duration zaapTime = Duration.ZERO;
}

Producing the Output

The writeReport() method produces the output. We produce a report for each day of the week where we collected job information. For each day we sort the jobs by CP time using the same method as Sample 1, then output the top 10 jobs.

Accumulated CPU time is written in the format hhh:mm:ss. We create a helper method to format the java.time.duration as hhmmss.

private static String hhhmmss(Duration dur)
{
long hours = dur.toHours();
long minutes = dur.minus(Duration.ofHours(hours)).toMinutes();
long seconds = dur.minus(Duration.ofHours(hours))
.minus(Duration.ofMinutes(minutes)).toMillis() / 1000;
return String.format("%d:%02d:%02d", hours, minutes, seconds);
}

EasySMF:JE provides a helper method to divide one duration by another, e.g. when calculating CPU time as a percentage of elapsed time or one job CPU time as a percentage of total CPU time.
Utils.divideDurations(jobinfo.cpTime, totalCp) * 100
Output looks like:

Tuesday
Name             CPU  CPU%        zIIP        zAAP
CICSP        1:31:32    5%       00:08       00:00
DB2PDIST     1:30:34    5%     4:55:55       00:00
DB2PDBM1     1:02:36    4%       00:00       00:00
...

The Complete Java Program

package com.blackhillsoftware.samples;

import java.io.FileInputStream;
import java.io.IOException;
import java.time.*;
import java.util.*;
import java.util.Map.*;

import com.blackhillsoftware.smf.SmfRecord;
import com.blackhillsoftware.smf.SmfRecordReader;
import com.blackhillsoftware.smf.Utils;
import com.blackhillsoftware.smf.smf30.ProcessorAccountingSection;
import com.blackhillsoftware.smf.smf30.Smf30Record;

public class PrimeShiftTopJobs
{
public static void main(String[] args) throws IOException
{

LocalTime primeStart = LocalTime.of(8, 30);
LocalTime primeEnd = LocalTime.of(17, 30);

// We need a Jobname->JobData map for each day of the week.
// So we will create a List (array) of HashMap<String, JobData>.

List<HashMap<String, JobData>> jobsByDay =
new ArrayList<HashMap<String, JobData>>();

// Populate the list for each day of the week
// We need entries for days 1-7. Entry 0 is unused
for (int i = 0; i <= 7; i++)
{
jobsByDay.add(new HashMap<String, JobData>());
}

// Read and process the data

try (SmfRecordReader reader =
args.length == 0 ?
SmfRecordReader.fromDD("INPUT") :
SmfRecordReader.fromStream(new FileInputStream(args[0])))
{

for (SmfRecord record : reader)
{
// We are only interested in type 30 subtype 2 or 3
if (record.recordType() == 30
&& (record.subType() == 2 || record.subType() == 3))
{

DayOfWeek recordDayOfWeek = record.smfDate().getDayOfWeek();
LocalTime recordTime = record.smfTime();

// check if SMF record was created during prime shift
// Monday-Friday
if (recordDayOfWeek.getValue() >= DayOfWeek.MONDAY.getValue()
&& recordDayOfWeek.getValue() <= DayOfWeek.FRIDAY.getValue()
&& recordTime.isAfter(primeStart)
&& recordTime.isBefore(primeEnd))
{

// Create a type 30 record from the original record
Smf30Record r30 = new Smf30Record(record);

// Iterate over the processor accounting sections
// Since there is either 0 or 1 section in a record, this is
// just a convenient way to skip records without the section
for (ProcessorAccountingSection proc : r30
.processorAccountingSections())
{
// get collection of jobs for this day
Map<String, JobData> day = jobsByDay
.get(recordDayOfWeek.getValue());

String jobname = r30.identificationSection().smf30jbn();
JobData jobentry = day.get(jobname);
if (jobentry == null) // nothing for this jobname,
// create new
{
day.put(jobname, new JobData(proc));
}
else
{
jobentry.add(proc);
}
}
}
}
};
}

writeReport(jobsByDay);
}

private static void writeReport(List<HashMap<String, JobData>> dailyJobs)
{

for (DayOfWeek day : DayOfWeek.values())
{
// get the jobs for the day as a List so we can sort them
List<Entry<String, JobData>> jobs = new ArrayList<Entry<String, JobData>>(
dailyJobs.get(day.getValue()).entrySet());

if (jobs.size() > 0) // if we have data for this day
{
// sort list by CPU time descending
Collections.sort(jobs, new Comparator<Entry<String, JobData>>()
{
public int compare(Entry<String, JobData> s1,
Entry<String, JobData> s2)
{
return s2.getValue().cpTime.compareTo(s1.getValue().cpTime);
}
});

// calculate total CPU for the day
Duration totalCp = Duration.ZERO;
for (Entry<String, JobData> entry : jobs)
{
totalCp = totalCp.plus(entry.getValue().cpTime);
}

// write output for top 10 job s

// Headings
System.out.format("%n%s%n", day.toString());
System.out.format("%-8s %11s %5s %11s %11s %n", "Name", "CPU",
"CPU%", "zIIP", "zAAP");

int count = 0;
for (Entry<String, JobData> entry : jobs)
{
JobData jobinfo = entry.getValue();

// write detail line

System.out.format(
"%-8s %11s %4.0f%% %11s %11s %n",
entry.getKey(), // jobname
hhhmmss(jobinfo.cpTime),
Utils.divideDurations(jobinfo.cpTime, totalCp) * 100,
hhhmmss(jobinfo.ziipTime),
hhhmmss(jobinfo.zaapTime));

// limit to top 10 jobs
if (++count >= 10)
break;
}
}
}
}

private static String hhhmmss(Duration dur)
{
long hours = dur.toHours();
long minutes = dur.minus(Duration.ofHours(hours)).toMinutes();
long seconds = dur.minus(Duration.ofHours(hours))
.minus(Duration.ofMinutes(minutes)).toMillis() / 1000;
return String.format("%d:%02d:%02d", hours, minutes, seconds);
}

private static class JobData
{
JobData(ProcessorAccountingSection proc)
{
add(proc);
}

public void add(ProcessorAccountingSection proc)
{
cpTime = cpTime.plus(proc.smf30cpt()).plus(proc.smf30cps())
.minus(proc.smf30TimeOnIfa()).minus(proc.smf30TimeOnZiip());
zaapTime = zaapTime.plus(proc.smf30TimeOnIfa());
ziipTime = ziipTime.plus(proc.smf30TimeOnZiip());
}

Duration cpTime = Duration.ZERO;
Duration ziipTime = Duration.ZERO;
Duration zaapTime = Duration.ZERO;
}

}

Java SMF Sample 3 : Performance Index

This sample illustrates the use of functions provided in the type 72 classes to calculate the WLM performance index and work with WLM Importances.

Calculation of the performance index can be complex, because it depends on the type of goal: velocity, average response time or percentile response time. Methods are provided in the ServiceReportClassPeriodDataSection class so you don’t have to do it yourself. The performance index can be calculated from single or multiple sections with the same goal. Methods are also provided to calculate velocity.

Processing

The sample program reads the SMF data and builds a list of service class intervals which have a performance index greater than 2. The list is sorted and grouped by system, time and importance, then listed by performance index and service class name.

Type 72 records can have multiple Service/Report Class Period Data sections. This is handled with nested for… loops. To get the sections we are interested in:

  1. Read the record.
  2. Check the type and subtype.
  3. Create the type 72 record.
  4. Skip report classes – we will only report service classes. workloadManagerControlSection().r723mrcl() tells us whether this record relates to a report class.
  5. Loop through the list of sections.
for (SmfRecord record : reader)
{
if (record.recordType() == 72 && record.subType() == 3)
    {
     Smf72Record r72 = new Smf72Record(record);
        if (!r72.workloadManagerControlSection().r723mrcl())
        {
         for (ServiceReportClassPeriodDataSection section :
             r72.serviceReportClassPeriodDataSections())
            {
             // process section...
            }
        }
    }
};

The ServiceClassPeriod class is used to hold information about the entries to be included in the report. The time value comes from the time the SMF record was written. SMF records from a single interval may not have exactly the same time, so we add 30 seconds and truncate the value to a minute value. This ensures that all entries have the same times and the grouping by time works correctly (assuming that all records are written within 30 seconds of the minute boundary).

private static class ServiceClassPeriod
{
public ServiceClassPeriod(Smf72Record record,
ServiceReportClassPeriodDataSection section)
{
system = record.system();
// round to nearest minute
time = record.smfDateTime().plusSeconds(30)
.truncatedTo(ChronoUnit.MINUTES);
name = record.workloadManagerControlSection().r723mcnm();
period = section.r723cper();
importance = section.importance();
perfIndex = section.performanceIndex();
}

String system;
LocalDateTime time;
String name;
int period;
Importance importance;
double perfIndex;
}

For each section we check the performance index, and add it to the list if the performance index is greater than 2.

For velocity goals we also check the amount of activity. Sometimes velocity goals show a high performance index simply because they are doing very little work and have very few using samples. In this case, if the using + delay sample count for a velocity goal is less than 5% of the number of times WLM ran its sampling, we ignore it as a low activity class. These numbers are very arbitrary – you might choose to change them, or not use them at all.

// if performance index > 2 and:
if (section.performanceIndex() > 2 &&
// either not a velocity goal or
   //  using + delay is > 5% of total samples
    (!section.r723cvel() ||
    section.r723ctot() + section.r723ctou()  
     > r72.workloadManagerControlSection().r723mtvNum() / 20)
)
{
highPI.add(new ServiceClassPeriod(r72, section));
}

We then sort the entries by the various criteria that we want to group by, then write group headers and detail lines.

The ServiceReportClassPeriodDataSection class returns the importance as an Importance enum. When comparing Importance enums Importance.SYSTEM is greater than Importance.DISCRETIONARY, which means that Importances 1-5 sort opposite to their numeric order i.e. Importance.I1 is greater than Importance.I5.

Importance.toString() returns the importance name for reporting, e.g. “1”, “Discretionary”.

Output

The output looks like the following:

SYSA

   1/10/14 12:45 AM
      Importance: 1
         TSO      Period 1 4.0
      Importance: 4
         DB2T     Period 1 2.6

   1/10/14 1:00 AM
      Importance: 4
         DB2T     Period 1 4.6</pre>

The Complete Program

import java.io.FileInputStream;
import java.io.IOException;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;
import java.util.*;
import com.blackhillsoftware.smf.Importance;
import com.blackhillsoftware.smf.SmfRecord;
import com.blackhillsoftware.smf.SmfRecordReader;
import com.blackhillsoftware.smf.smf72.Smf72Record;
import com.blackhillsoftware.smf.smf72.subtype3.ServiceReportClassPeriodDataSection;

public class PerformanceIndex
{
public static void main(String[] args) throws IOException
{
ArrayList<ServiceClassPeriod> highPI = new ArrayList<ServiceClassPeriod>();
try (SmfRecordReader reader =
args.length == 0 ?
SmfRecordReader.fromDD("INPUT") :
SmfRecordReader.fromStream(new FileInputStream(args[0])))
{
for (SmfRecord record : reader)
{
if (record.recordType() == 72 && record.subType() == 3)
{
Smf72Record r72 = new Smf72Record(record);

// skip report classes
if (!r72.workloadManagerControlSection().r723mrcl())
{
for (ServiceReportClassPeriodDataSection section : r72
.serviceReportClassPeriodDataSections())
{
// if performance index > 2 and:
if (section.performanceIndex() > 2
&&
// either not a velocity goal or
(!section.r723cvel() ||
// using + delay is > 5% of total samples
section.r723ctot() + section.r723ctou() >
r72.workloadManagerControlSection().r723mtvNum() / 20))
{
highPI.add(new ServiceClassPeriod(r72, section));
}
}
}
}
}
}
writeReport(highPI);
}

private static void writeReport(ArrayList<ServiceClassPeriod> highPI)
{
// Use custom comparator for complex sort order
Collections.sort(highPI, new ServiceClassPeriodComparator());

DateTimeFormatter timef = DateTimeFormatter.ISO_LOCAL_TIME;
DateTimeFormatter datef = DateTimeFormatter.ISO_LOCAL_DATE;

LocalDateTime currentTime = null;
String currentSystem = null;
Importance currentImportance = null;

for (ServiceClassPeriod scInfo : highPI)
{

// group by System
if (!scInfo.system.equals(currentSystem))
{
System.out.format("%n%s%n", scInfo.system);
currentSystem = scInfo.system;
currentTime = null;
currentImportance = null;
}
if (!scInfo.time.equals(currentTime))
{
System.out.format("%n %s %s%n", scInfo.time.format(datef),
scInfo.time.format(timef));
currentTime = scInfo.time;
currentImportance = null;
}
// Then by importance
if (scInfo.importance != currentImportance)
{
System.out.format(" Importance: %s%n", scInfo.importance);
currentImportance = scInfo.importance;
}
// detail line
System.out.format(" %-8s Period %s %3.1f%n", scInfo.name,
scInfo.period, scInfo.perfIndex);
}
}

/**
*
* Class to keep information about a service class period
*
*/
private static class ServiceClassPeriod
{
public ServiceClassPeriod(Smf72Record record,
ServiceReportClassPeriodDataSection section)
{
system = record.system();
// round to nearest minute
time = record.smfDateTime().plusSeconds(30)
.truncatedTo(ChronoUnit.MINUTES);
name = record.workloadManagerControlSection().r723mcnm();
period = section.r723cper();
importance = section.importance();
perfIndex = section.performanceIndex();
}

String system;
LocalDateTime time;
String name;
int period;
Importance importance;
double perfIndex;
}

/**
*
* Comparator to implement custom sort order for report
*/
private static class ServiceClassPeriodComparator implements
Comparator<ServiceClassPeriod>
{
// sort by system,
// then by time,
// then by importance
// then by performance index descending
// finally by name and period
public int compare(ServiceClassPeriod s1, ServiceClassPeriod s2)
{
int result = s1.system.compareTo(s2.system);
if (result != 0)
return result;
result = s1.time.compareTo(s2.time);
if (result != 0)
return result;
result = s2.importance.compareTo(s1.importance);
if (result != 0)
return result;
// reversed to sort descending
result = Double.compare(s2.perfIndex, s1.perfIndex);
if (result != 0)
return result;
result = s1.name.compareTo(s2.name);
if (result != 0)
return result;
result = Integer.compare(s1.period, s2.period);
return result;
}
}
}

Processing z/OS SMF data using Java

Black Hill Software is currently developing an API for processing SMF data using Java.

The API builds on experience gained developing EasySMF to create a powerful and easy to use API for working with SMF data.

The API will run on z/OS and other platforms such as Linux and Windows. The API is 100% Java which makes it zAAP eligible on z/OS.

The design at the time of this post used Joda Time for times and durations. This was changed prior to the initial release to use the java.time package introduced with Java 8. Java.time has the same advantages as Joda Time but it is a core part of Java so the removal of an external dependency was worth the cost of requiring Java 8 as a minimum level. Java.time is well suited to working with the dates and times in SMF, with specific classes to distinguish between e.g. local date/times, UTC date/times and durations. The java.time resolution of 1 nanosecond is accurate enough for most SMF purposes.

Data Conversions

The API aims to provide a consistent interface across different record types and sections.

  • Times representing a duration e.g. CPU time, connect time are converted to floating point values expressed in seconds. This allows easy manipulation and comparison, without dealing with the multitude of time units used in SMF. The raw unconverted value is also available if required.
  • Dates and times of day are expressed using Joda-Time classes. Joda-Time provides simple functions for most date and time calculations, so you can easily work with days of the week etc. It also provides time zone calculations, so you can e.g. apply a time zone to the times from the SMF records and match times between systems in different time zones. Joda-Time includes the Time Zone Database, so if you apply a time zone it can apply the correct rules to account for daylight savings.
  • STCK fields are available as a DateTime value (with only millisecond precision) and as a BigInteger with the complete value for use as a timestamp, for sorting etc.
  • 1, 2 and 3 byte integer values are converted to the Java int datatype.
  • 4 byte/32 bit unsigned integer values are converted to the Java long (64 bit integer) datatype. Java does not have an unsigned datatype, so a 4 byte int can’t hold the complete range of 32 bit unsigned values.
  • 8 byte/64 bit unsigned integer values are available as long or BigInteger values. The long datatype may provide better performance if the value will not exceed the maximum 64 bit signed value. The API will throw an exception if the value is too large for a long. If this is possible, use the BigInteger value.
  • Integers greater than 8 bytes are converted to BigIntegers.
  • Floating point values are converted to the Java double datatype.
  • SMF record classes provide methods to return the SMF subsections. Subsections are returned as a List<T> if the number of sections can vary, or individually if there is always a single section.

Current Status

The API is currently undergoing pre-release testing. I am looking for feedback on usability, performance and naming conventions. You can download the current version from the links on this page. Feedback is welcome in the comments area, or to info@blackhillsoftware.com.

Record types 30, 70, 71, 72, 73, 74, 75, 77 and 78 are currently mapped. More record types are planned, but these record types should make enough interesting data available for some realistic testing and real world feedback on the design of the API.

At this point the API design is changeable, so program changes may be required for new versions. One of the objectives of the testing period is to get enough feedback to identify problems now, and avoid breaking changes in the future.

Feedback

Some of the areas where I would particularly like feedback are:

  • Naming conventions: Many Java APIs use the get…() naming convention for method names. Almost all the SMF methods are get methods. Prefixing everything with get didn’t seem to add much value and made the resulting code more difficult to read, so the methods currently use bare field names. Are there advantages to using the get prefix e.g. for use with code generation/reporting tools?
  • More naming conventions: Many of the names of classes representing section types are long. However, they are named to be consistent with the SMF documentation. I believe this makes the programs easier to write and understand, and makes the Eclipse auto complete easier to use.
  • Use of the Joda Time classes: Joda Time is useful for date and time calculations. However, it does add a dependency on an external library. Would it be preferable to use the standard date time classes and eliminate the dependency? Joda Time could still be used at a site’s own discretion, writing their own conversions from standard times as required.
  • General comments on the data types used.

Documentation and Downloads

The current Javadoc is available here.

For more information, including code samples see:

https://www.blackhillsoftware.com/news/category/java-smf/

Installation Instructions

Sample Programs

Sample 1 : SMF Records by type and subtype

Sample 2 : Prime Shift Top CPU Consumers

Sample 3 : Performance Index

Download the current version here:

Samples are available on Github: https://github.com/BlackHillSoftware/easysmf-samples

EasySMF News: April 2014

In this issue:

  1. Compressing data for transfer
  2. Loading data from a z/OS batch job
  3. Read-only Repository
  4. Extended wild card support
  5. New Time Selection Controls
  6. Performance Improvement
  7. Request for data

It has been quite some time since the last EasySMF News, and there have been some significant updates in that time. Version 2.0.4 in March included a large number of new features and bug fixes. Version 2.0.5 includes improved support for loading compressed data and sample JCL to load data using a z/OS batch job.

Feedback on the various JCL samples included with version 2.0.5 would be welcome. Every z/OS site is slightly different, so sometimes samples need a bit of tweaking to get them working. If there was something you had to change that would be worth noting in the documentation (or just errors!) please let me know.

Compressing Data for Transfer

EasySMF now supports loading compressed data in Gzip format using the built in FTP function. SMF data is very compressible – typical compression is about 10:1 – so this could significantly reduce the time and network traffic to load the data.

Sample jobs are supplied to compress data using Gzip from the IBM Ported Tools with Dovetailed Technologies Co:Z Toolkit, and using a Java program with the IBM JZOS Batch Toolkit. Any other software that produces Gzip format output should also work – as long as the compressed data includes the RDW with the SMF data. Including the RDW proved to be the difficult bit when testing various tools.

Zip format is also supported if the data is loaded from a file on the PC e.g. using EasySMFLoad. That means that Zip is one of the supported formats for loading from a z/OS batch job…

Loading data from a z/OS batch job

Another sample job demonstrates loading data into EasySMF using a z/OS batch job. The process uses the Dovetailed Technologies Hybrid Batch products (Co:Z Launcher and Co:Z Dataset Pipes). The messages and return code from the EasySMFLoad command line program appear in the z/OS batch job output.

EasySMFLoad is invoked using SSH. The data can be transferred over SSH and optionally compressed in transit.

Read only repository

Read only access to the SMF data repository is now supported. This could be useful if you share the repository between multiple people.

Read-write access is still required for some functions, e.g. managing the data and if repository changes are required for a new version of EasySMF.

If repository changes are required for a new version of EasySMF they are normally made the first time the repository is opened with the new version. (You will receive a warning if the changes are not compatible with older versions.) If the repository is read only the new version of EasySMF will not be able to use it until it is opened by someone with write access.

Extended wildcard support

Wildcard and regular expressions are now supported for all the text based report parameters. This is particularly useful for service and report class reports, where you can now use regular expressions to exclude particular service or report classes. For example, the regular expression:

-/TEST/

will exclude anything matching *TEST*. To exclude only TEST but not TESTA, MYTEST etc. you need to add anchors to the start and end of the string:

-/^TEST$/

New time selection controls

The time selection controls have been rewritten. A few bugs have been fixed and some new features added:

  • A button to zoom out to a wider view of the data. This is particularly useful if you have used the mouse to zoom in to a chart, and want to return to a wider view, but not the whole time range you zoomed in from. The button extends the time range to 3 times the current range.
  • Additional predefined times. In addition to Today, This Week, This Month you can now select Yesterday, Last Week, Last Month.

Performance improvement

There was a bug prior to version 2.0.4 that resulted in unnecessary data being read when creating reports. This particularly affected reports where the reporting period was small compared with the period in the SMF dataset e.g. if you had monthly SMF data but ran a report for a day, or if you ran a report for a short running job.

Version 2.0.4 fixes that bug. Updating from 2.0.3 or earlier requires write access to the repository the first time EasySMF runs, and will scan the data to update information in the repository catalog. If you installed EasySMFLoad on another PC to load data it also needs to be updated.

Request for data

I’m looking for some type 30 data that includes the new Counter section. That is from z/OS 2.1, with Hardware Instrumentation Services active and SMF30COUNT specified in SMFPRMxx. Some z/OS 2.1 type 113 records would also be useful for some new reports under development. At this stage I don’t need a lot of data – a few hours from a test LPAR would be very helpful.

EasySMF Evaluations

EasySMF has a 30 day evaluation period. You can download it and start using it immediately.

Trial Extensions

New releases of EasySMF automatically provide a 7 day “re-evaluation” period if you have already used the 30 day evaluation so you can try out new features. Trial extensions can also be arranged – please contact info@blackhillsoftware.com if required.

Get the current version of EasySMF

EasySMF News: September 2013

In this issue:

  1. EasySMF Version 2 Released
  2. Report Spotlight: Job Status During Interval

EasySMF Version 2

EasySMF version 2 has recently been released.

There are minimal outward changes, but there have been significant changes under the covers. The focus with Version 2 was to fix areas where performance or memory usage was a problem. There is a new internal architecture, and more than 20 of the largest reports were rewritten to reduce the memory required. Testing has shown that in many cases memory usage has been reduced by as much as 90%.

The chart layout code was also rewritten, so that charts will cope better with different screen sizes and resolutions and larger amounts of data.

Some of the other significant changes in version 2:

  • Centralized license key management. You can put the key on your web server, and set the URL in EasySMF or as a parameter to the installation. The key will then be retrieved via HTTP, so it does not need to be installed on each computer for new installations or renewals.
  • Optional filtering by regular expression. Regular expressions allow more complex filtering, including finding values that are not equal to an expression.
  • Paging and storage reports. Storage information has been added to the Job Status During Interval report.
  • Context sensitive help for reports. Pressing F1 when viewing a report will show the help page for that report.
  • Various bug fixes.

Another notable performance problem was fixed in version 1.1.9. Deleting data was very slow. Elapsed time for deleting data was reduced by 99% in testing after the fix.
(How do you reduce the elapsed time for a piece of code by 99%? I am sure many people are wondering. It’s simple –
Step 1: Write some very slow code…
More seriously: As the data was being deleted, entries were removed from the repository catalog. Each of those database updates was not only writing to disk, but waiting for the physical write to complete. Reorganizing the updates and wrapping a transaction around them avoided most of the waits, as the database only waits for the physical write at the end of a transaction.)

EasySMF version 2 requires version 3.5 SP1 of the Microsoft .NET Framework. If it is not installed the installer will download and install it for you (Internet access and administrator privileges are required). Alternatively, .NET 3.5 SP1 can be installed using Windows Update. Dot NET 3.5 was released in 2007, included with Windows 7 and has been a high priority update for Windows XP for a long time so hopefully this will not inconvenience too many people.

Licensed users can upgrade to version 2 at no cost, and the price of licenses remains the same.

Report Spotlight: Job Status During Interval

This somewhat awkwardly named report is actually one of the key reports in EasySMF, because it can show you what was actually running in the system during a time period. It uses all the type 30 subtypes to show the activity of each job during that time (CPU time, I/O etc). If you are looking at one of the other reports, and you are wondering what was running at the time, this report will show you.

  • If you see an jump in CPU busy, and want to know which jobs started at the time the CPU usage increased, check this report. Click and drag to draw a box around the CPU spike to zoom into the time in question. Switch to Job Status During Interval, select “Started” as the status, and “Complete” if the job(s) in question might have ended. You can sort by CPU time or any other column to find the most active jobs.
  • If you see a jump in paging or page dataset usage this report can show you the address space storage high water marks and how much they changed during the interval, so you can see if a task has increased its storage consumption.
  • If you want to know which jobs were running in a service class at a particular time, this report can tell you, along with which jobs used the most CPU etc.

There are 4 different statuses that can be shown: Started, Ended, Running and Complete.
Started: the job started, but did not end during the time period.
Ended: the job was running at the start of the period, and ended during the period.
Running: the job was running for the whole period i.e. there were interval/step end records, but no job start or end records.
Complete: the complete job was contained within the time period.
The information in the report includes:

  • Service
  • CPU time
  • I/O activity
  • Paging
  • Changes to storage high water marks
  • The number of SMF records produced by the job

for the time period selected.

This report is one of the reports where memory usage has been dramatically reduced in EasySMF version 2. If you are reporting long time periods or large amounts of data you will find version 2 is a great improvement.

Obviously, accurate data requires all type 30 subtypes. CPU time etc. is calculated as the difference between the first and last values, so more than one SMF interval is required for jobs that are not Complete. The more SMF intervals are included the more accurate the results will be.

An example

This Service by Service Class report shows a lot of service in STCMED between about 2 and 7am.

Dragging on the chart allows you to zoom in on the time in question:

Clicking on the STCMED area switches to the Job Status During Interval report for that service class:

You can sort the report by Service, and quickly see that DFHSM was consuming the resources – which not surprising for that time of day.

You can use a similar sequence to find out what was running at the time of any other report. Some reports, like Service by Service Class, will open the Job Status during interval report directly if you click on a data item. For others, you need to select it from the report menu.

EasySMF News April 2013

EasySMF Price Update

As mentioned in the last newsletter, the price of EasySMF is increasing.

New Prices

User Licenses
User licenses are priced per user of the product. There is no restriction of the number of z/OS systems they may be used for.

Single User
1 year: $795
2 years: $1,350

5 User pack
1 year: $2,995
2 years: $4,995

10 User pack
1 year: $4,995
2 years: $8,395

Site License
A site license allows any number of users within your organization, and is priced by the total number of z/OS systems from which you will be processing data.

1 year: From $995 (1 system)
2 years: From $1,690 (1 system)
Volume discounts apply to site licenses, e.g. a 1 year site license for 5 systems will cost $3,325.

Due to the delay in announcing the prices, the new prices will come into effect on the 1st May 2013 instead of the 15th April as previously indicated.
Until the 1st May, new licenses and renewals can be purchased at the old price. Existing licenses can be extended by up to 2 years, even if they are not yet due for renewal.

Buy licenses online at https://www.blackhillsoftware.com/easysmf-order-form/ or contact info@blackhillsoftware.com.

EasySMF News March 2013

In this issue:

Loading Data from the Command Line

The latest release of EasySMF includes a function that many people have been wanting: the ability to load data from the Windows command line. This allows you to create automated scripts to load data into EasySMF.
The syntax is:

EasySMFLoad “repository-location” “smf-file-name”

There are several samples included to help you get started:

  • loadfile.bat – A batch file to load data from a PC file into EasySMF
  • loadftp.bat – A batch file to download data from z/OS using Windows command line FTP and load it into EasySMF
  • loadftp.txt – The FTP script used by loadftp.bat
  • ftpjcl.txt – JCL to transfer data using a z/OS batch job.

To find the samples, go to Start -> All Programs -> EasySMF -> Samples.

Price Update

We are currently reviewing the price of EasySMF. Due to the continued weakness of the US Dollar relative to the Australian Dollar (a decline of almost 40% since pricing was announced and 20% since EasySMF first went on sale!) a price rise is unavoidable.

To give fair warning, new prices will come into effect from 15th April. Licenses and renewals before that date will be at the current prices.

If your license expires soon after 15th April or if you just want to save a few dollars you can renew early to get the current prices. Existing licenses can be extended even if they are not due for renewal. On the EasySMF Purchase page select “Renew License” and enter your current license key. The remaining time on your current license will be added to the new key.

Storage Management Reports

SMF data is not just for the capacity planning and performance management groups. One of the objectives of EasySMF is to make SMF information available to groups who traditionally may not have had tools to use it. Here are some reports that the Storage Management group might be interested in:

Non-VSAM SMS Information Report
This report shows SMS information for newly allocated non-VSAM datasets. Dataset attributes and SMS information are shown. You can filter the list to show datasets created with a particular storage class, data class, management class or storage group. Storage group information isn’t included in the type 14/15 records, so EasySMF uses type 74 subtype 1 records to map volumes to storage groups.

Non-VSAM Space Report
This report shows uses several record types to track non-VSAM datasets being created and deleted over time. It gives an indication of the peak storage demand in your storage groups, and how much space is used. Not all allocations are included (e.g. VSAM datasets, datasets allocated but never opened and additional extents for existing datasets are not included) and some assumptions need to be made about the time the datasets are created, so it can only provide an indication of the space required. Despite that the information is interesting, and can show patterns in creating and deleting datasets. It could be useful to work out the best time to run utilities to monitor free space. Or it might show storage groups with completely different allocation patterns where the free space could be combined.

This report also provides a good example of how you can zoom and drill through to other reports for more information. In this report you can:

  • Click and drag on a chart to zoom in on the peaks.
  • Click on the chart to drill through to the Non-VSAM SMS Information report, which will show the datasets allocated in that storage group during that interval. Then scroll across and click the header for the Tracks column to sort the datasets by size.
  • From that report, click a job name to drill through to the Job Detail report or the dataset name to view the Dataset Activity report.

Allocation Failures Report
This report lists jobs which suffered B37, D37 and E37 abends, and volume selection failures when allocating a dataset. A high number of these events might indicate insufficient free space in DASD pools.

VSAM Statistics and Non-VSAM Statistics Reports
These reports show various statistics from the type 64 records for VSAM and type 14 and 15 records for non-VSAM. Information about the jobs using a dataset, number of EXCPs and number of extents might be interesting.

EasySMF News: November 2012

In this issue:

  1. New report: Service Class Overview
  2. Improved WLM reporting
  3. Copy and paste to other applications
  4. Request for data: z/OS 1.13 Type 72
  5. Report Spotlight: Service Class Overview

New report: Service Class Overview

The latest release of EasySMF adds a new report: Workload Manager – Service Class Overview. This report gives information about all the service classes in the sysplex in a single report. It shows goals, activity and delays for each service class period. The report is particularly useful if you are not familiar with the WLM policy on a system, because you can see at a glance the goals and importances as well as the amount of work in each service class period.

For a more detailed look at the report, see the Report Spotlight in this issue.

Improved WLM reporting

Many of the changes in the new release were concentrated on improving Workload Manager and Sysplex reporting. The WLM reports can now be viewed by sysplex or individual system. The Workload Manager – Service Class Detail report now shows the performance index for the sysplex and each system in the sysplex. A new chart shows the response time for the service class period.

The type 30 Job Completions reports and Job Status During Interval report now support filtering by sysplex and service class, to complement the Workload Manager reports. From the Service Class Detail report you can select the Job Status During Interval report to see the work running in that service class during the interval, or select one of the Job Completions reports to see the jobs that ended in the service class.

Copy and paste to other applications

Copying to the clipboard is now available in the reports, allowing Copy and Paste to other applications e.g. Microsoft Word.

Individual charts can be copied by right clicking on the chart and selecting “Copy to Clipboard”. The menu option Edit->Copy or Control-C will copy all the charts in the current report to the clipboard as a single image.

Data from the tabular reports can also be copied and pasted – either the entire report or if rows are selected, specific rows.

Request for data

z/OS 1.13 added response time distribution information to type 72 records for service classes with velocity goals. Charting the response time distribution for velocity goals is not as simple as response time goals, because the velocity response time “buckets” are not fixed values. A new chart design is needed for the velocity goals.

Type 72 data from some z/OS 1.13 systems with real workloads would be very useful for developing these reports. If you can help, please contact info@blackhillsoftware.com or reply to this email.

Report Spotlight: Service Class Overview

The Workload Manager – Service Class Overview report shows you information about all the service classes in a single report.

The report helps you understand the WLM policy in use by showing not only the service classes, goals and importances, but also the amount of work and the delays for each service class period.

Selecting “Active Service Classes Only” is a quick way to see the number of active service class periods in the system.

Information in the report includes:

  • Service class description
  • Duration of service class periods
  • Goal
  • Importance
  • Performance Index (calculated over the selected time interval)
  • CPU critical and storage critical attributes
  • Number of transactions
  • Average response and standard deviation
  • Amount of service consumed
  • Delays
  • Resource group information

Each column is sortable, so you can click on the column header to sort by Importance, Performance Index, Velocity, Service etc.

The report serves as an index for the Service Class Detail report. Clicking on an entry for a service class will take you to the Service Class Detail report for that service class, where you can view more information about the service class during the time interval.