Skip to main content

Java SMF Sample 3 : Performance Index

This sample illustrates the use of functions provided in the type 72 classes to calculate the WLM performance index and work with WLM Importances.

Calculation of the performance index can be complex, because it depends on the type of goal: velocity, average response time or percentile response time. Methods are provided in the ServiceReportClassPeriodDataSection class so you don’t have to do it yourself. The performance index can be calculated from single or multiple sections with the same goal. Methods are also provided to calculate velocity.

Processing​

The sample program reads the SMF data and builds a list of service class intervals which have a performance index greater than 2. The list is sorted and grouped by system, time and importance, then listed by performance index and service class name.

Type 72 records can have multiple Service/Report Class Period Data sections. This is handled with nested for… loops. To get the sections we are interested in:

  1. Read the record.
  2. Check the type and subtype.
  3. Create the type 72 record.
  4. Skip report classes – we will only report service classes. workloadManagerControlSection().r723mrcl() tells us whether this record relates to a report class.
  5. Loop through the list of sections.
for (SmfRecord record : reader)
{
if (record.recordType() == 72 && record.subType() == 3)
    {
     Smf72Record r72 = new Smf72Record(record);
        if (!r72.workloadManagerControlSection().r723mrcl())
        {
         for (ServiceReportClassPeriodDataSection section :
             r72.serviceReportClassPeriodDataSections())
            {
             // process section...
            }
        }
    }
};

The ServiceClassPeriod class is used to hold information about the entries to be included in the report. The time value comes from the time the SMF record was written. SMF records from a single interval may not have exactly the same time, so we add 30 seconds and truncate the value to a minute value. This ensures that all entries have the same times and the grouping by time works correctly (assuming that all records are written within 30 seconds of the minute boundary).

private static class ServiceClassPeriod
{
public ServiceClassPeriod(Smf72Record record,
ServiceReportClassPeriodDataSection section)
{
system = record.system();
// round to nearest minute
time = record.smfDateTime().plusSeconds(30)
.truncatedTo(ChronoUnit.MINUTES);
name = record.workloadManagerControlSection().r723mcnm();
period = section.r723cper();
importance = section.importance();
perfIndex = section.performanceIndex();
}

String system;
LocalDateTime time;
String name;
int period;
Importance importance;
double perfIndex;
}

For each section we check the performance index, and add it to the list if the performance index is greater than 2.

For velocity goals we also check the amount of activity. Sometimes velocity goals show a high performance index simply because they are doing very little work and have very few using samples. In this case, if the using + delay sample count for a velocity goal is less than 5% of the number of times WLM ran its sampling, we ignore it as a low activity class. These numbers are very arbitrary – you might choose to change them, or not use them at all.

// if performance index > 2 and:
if (section.performanceIndex() > 2 &&
// either not a velocity goal or
   //  using + delay is > 5% of total samples
    (!section.r723cvel() ||
    section.r723ctot() + section.r723ctou()  
     > r72.workloadManagerControlSection().r723mtvNum() / 20)
)
{
highPI.add(new ServiceClassPeriod(r72, section));
}

We then sort the entries by the various criteria that we want to group by, then write group headers and detail lines.

The ServiceReportClassPeriodDataSection class returns the importance as an Importance enum. When comparing Importance enums Importance.SYSTEM is greater than Importance.DISCRETIONARY, which means that Importances 1-5 sort opposite to their numeric order i.e. Importance.I1 is greater than Importance.I5.

Importance.toString() returns the importance name for reporting, e.g. “1”, “Discretionary”.

Output​

The output looks like the following:

SYSA

   1/10/14 12:45 AM
      Importance: 1
         TSO      Period 1 4.0
      Importance: 4
         DB2T     Period 1 2.6

   1/10/14 1:00 AM
      Importance: 4
         DB2T     Period 1 4.6</pre>

The Complete Program​

import java.io.FileInputStream;
import java.io.IOException;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;
import java.util.*;
import com.blackhillsoftware.smf.Importance;
import com.blackhillsoftware.smf.SmfRecord;
import com.blackhillsoftware.smf.SmfRecordReader;
import com.blackhillsoftware.smf.smf72.Smf72Record;
import com.blackhillsoftware.smf.smf72.subtype3.ServiceReportClassPeriodDataSection;

public class PerformanceIndex
{
public static void main(String[] args) throws IOException
{
ArrayList<ServiceClassPeriod> highPI = new ArrayList<ServiceClassPeriod>();
try (SmfRecordReader reader =
args.length == 0 ?
SmfRecordReader.fromDD("INPUT") :
SmfRecordReader.fromStream(new FileInputStream(args[0])))
{
for (SmfRecord record : reader)
{
if (record.recordType() == 72 && record.subType() == 3)
{
Smf72Record r72 = new Smf72Record(record);

// skip report classes
if (!r72.workloadManagerControlSection().r723mrcl())
{
for (ServiceReportClassPeriodDataSection section : r72
.serviceReportClassPeriodDataSections())
{
// if performance index > 2 and:
if (section.performanceIndex() > 2
&&
// either not a velocity goal or
(!section.r723cvel() ||
// using + delay is > 5% of total samples
section.r723ctot() + section.r723ctou() >
r72.workloadManagerControlSection().r723mtvNum() / 20))
{
highPI.add(new ServiceClassPeriod(r72, section));
}
}
}
}
}
}
writeReport(highPI);
}

private static void writeReport(ArrayList<ServiceClassPeriod> highPI)
{
// Use custom comparator for complex sort order
Collections.sort(highPI, new ServiceClassPeriodComparator());

DateTimeFormatter timef = DateTimeFormatter.ISO_LOCAL_TIME;
DateTimeFormatter datef = DateTimeFormatter.ISO_LOCAL_DATE;

LocalDateTime currentTime = null;
String currentSystem = null;
Importance currentImportance = null;

for (ServiceClassPeriod scInfo : highPI)
{

// group by System
if (!scInfo.system.equals(currentSystem))
{
System.out.format("%n%s%n", scInfo.system);
currentSystem = scInfo.system;
currentTime = null;
currentImportance = null;
}
if (!scInfo.time.equals(currentTime))
{
System.out.format("%n %s %s%n", scInfo.time.format(datef),
scInfo.time.format(timef));
currentTime = scInfo.time;
currentImportance = null;
}
// Then by importance
if (scInfo.importance != currentImportance)
{
System.out.format(" Importance: %s%n", scInfo.importance);
currentImportance = scInfo.importance;
}
// detail line
System.out.format(" %-8s Period %s %3.1f%n", scInfo.name,
scInfo.period, scInfo.perfIndex);
}
}

/**
*
* Class to keep information about a service class period
*
*/
private static class ServiceClassPeriod
{
public ServiceClassPeriod(Smf72Record record,
ServiceReportClassPeriodDataSection section)
{
system = record.system();
// round to nearest minute
time = record.smfDateTime().plusSeconds(30)
.truncatedTo(ChronoUnit.MINUTES);
name = record.workloadManagerControlSection().r723mcnm();
period = section.r723cper();
importance = section.importance();
perfIndex = section.performanceIndex();
}

String system;
LocalDateTime time;
String name;
int period;
Importance importance;
double perfIndex;
}

/**
*
* Comparator to implement custom sort order for report
*/
private static class ServiceClassPeriodComparator implements
Comparator<ServiceClassPeriod>
{
// sort by system,
// then by time,
// then by importance
// then by performance index descending
// finally by name and period
public int compare(ServiceClassPeriod s1, ServiceClassPeriod s2)
{
int result = s1.system.compareTo(s2.system);
if (result != 0)
return result;
result = s1.time.compareTo(s2.time);
if (result != 0)
return result;
result = s2.importance.compareTo(s1.importance);
if (result != 0)
return result;
// reversed to sort descending
result = Double.compare(s2.perfIndex, s1.perfIndex);
if (result != 0)
return result;
result = s1.name.compareTo(s2.name);
if (result != 0)
return result;
result = Integer.compare(s1.period, s2.period);
return result;
}
}
}

Processing z/OS SMF data using Java

Black Hill Software is currently developing an API for processing SMF data using Java.

The API builds on experience gained developing EasySMF to create a powerful and easy to use API for working with SMF data.

The API will run on z/OS and other platforms such as Linux and Windows. The API is 100% Java which makes it zAAP eligible on z/OS.

The design at the time of this post used Joda Time for times and durations. This was changed prior to the initial release to use the java.time package introduced with Java 8. Java.time has the same advantages as Joda Time but it is a core part of Java so the removal of an external dependency was worth the cost of requiring Java 8 as a minimum level. Java.time is well suited to working with the dates and times in SMF, with specific classes to distinguish between e.g. local date/times, UTC date/times and durations. The java.time resolution of 1 nanosecond is accurate enough for most SMF purposes.

Data Conversions​

The API aims to provide a consistent interface across different record types and sections.

  • Times representing a duration e.g. CPU time, connect time are converted to floating point values expressed in seconds. This allows easy manipulation and comparison, without dealing with the multitude of time units used in SMF. The raw unconverted value is also available if required.
  • Dates and times of day are expressed using Joda-Time classes. Joda-Time provides simple functions for most date and time calculations, so you can easily work with days of the week etc. It also provides time zone calculations, so you can e.g. apply a time zone to the times from the SMF records and match times between systems in different time zones. Joda-Time includes the Time Zone Database, so if you apply a time zone it can apply the correct rules to account for daylight savings.
  • STCK fields are available as a DateTime value (with only millisecond precision) and as a BigInteger with the complete value for use as a timestamp, for sorting etc.
  • 1, 2 and 3 byte integer values are converted to the Java int datatype.
  • 4 byte/32 bit unsigned integer values are converted to the Java long (64 bit integer) datatype. Java does not have an unsigned datatype, so a 4 byte int can’t hold the complete range of 32 bit unsigned values.
  • 8 byte/64 bit unsigned integer values are available as long or BigInteger values. The long datatype may provide better performance if the value will not exceed the maximum 64 bit signed value. The API will throw an exception if the value is too large for a long. If this is possible, use the BigInteger value.
  • Integers greater than 8 bytes are converted to BigIntegers.
  • Floating point values are converted to the Java double datatype.
  • SMF record classes provide methods to return the SMF subsections. Subsections are returned as a List<T> if the number of sections can vary, or individually if there is always a single section.

Current Status​

The API is currently undergoing pre-release testing. I am looking for feedback on usability, performance and naming conventions. You can download the current version from the links on this page. Feedback is welcome in the comments area, or to info@blackhillsoftware.com.

Record types 30, 70, 71, 72, 73, 74, 75, 77 and 78 are currently mapped. More record types are planned, but these record types should make enough interesting data available for some realistic testing and real world feedback on the design of the API.

At this point the API design is changeable, so program changes may be required for new versions. One of the objectives of the testing period is to get enough feedback to identify problems now, and avoid breaking changes in the future.

Feedback​

Some of the areas where I would particularly like feedback are:

  • Naming conventions: Many Java APIs use the get…() naming convention for method names. Almost all the SMF methods are get methods. Prefixing everything with get didn’t seem to add much value and made the resulting code more difficult to read, so the methods currently use bare field names. Are there advantages to using the get prefix e.g. for use with code generation/reporting tools?
  • More naming conventions: Many of the names of classes representing section types are long. However, they are named to be consistent with the SMF documentation. I believe this makes the programs easier to write and understand, and makes the Eclipse auto complete easier to use.
  • Use of the Joda Time classes: Joda Time is useful for date and time calculations. However, it does add a dependency on an external library. Would it be preferable to use the standard date time classes and eliminate the dependency? Joda Time could still be used at a site’s own discretion, writing their own conversions from standard times as required.
  • General comments on the data types used.

Documentation and Downloads​

The current Javadoc is available here.

For more information, including code samples see:

https://www.blackhillsoftware.com/news/category/java-smf/

Sample Programs​

Sample 1 : SMF Records by type and subtype

Sample 2 : Prime Shift Top CPU Consumers

Sample 3 : Performance Index

Download the current version here:

Samples are available on Github: https://github.com/BlackHillSoftware/easysmf-samples