This sample program reads all the SMF records from a file or dataset, and for each record type and subtype lists the number of records, total bytes and percentage of the total.
These examples are not intended to suggest any particular coding style or structure for Java programs. They are intended to show what is possible and illustrate how to access various features of the Java SMF API. In fact I have deliberately ignored some Java conventions in these samples (e.g. encapsulation in the inner classes) to cut the total amount of code and concentrate on illustrating the SMF processing facilities.
To start with, I’ll skip the most of the Java stuff and concentrate on the SMF processing.
Reading SMF data
The SmfRecordReader class provides the facility to read SMF records. It uses a JZOS RecordReader internally to read from a DDNAME or reads directly from a stream.
A stream can be any type of InputStream. Typically it would be a file but it could also be some sort of network stream, or you could chain streams together to read compressed data etc. The stream must contain the record descriptor words (RDWs) so the record lengths can be determined.
SmfRecordReader implements AutoCloseable which means that it can be used in a try-with-resources block to automatically close the reader when exiting the block.
To open a DD for reading:
try (SmfRecordReader reader = SmfRecordReader.fromDD("INPUT")) {
...
}
To read from a file:
try (SmfRecordReader reader = SmfRecordReader
.fromStream(new FileInputStream(
"C:\\Users\\Andrew\\Documents\\SMF Data\\weekly.smf"))) {
...
}
The samples handle both cases by accepting the filename as an argument to the program. If no filename is provided they open the INPUT DD:
try (SmfRecordReader reader =
args.length == 0 ?
SmfRecordReader.fromDD("INPUT") :
SmfRecordReader.fromStream(new FileInputStream(args[0])))
{
...
}
This makes it simple to develop and test on the workstation, then move the program to z/OS to run as a batch job.
Processing the data
for (SmfRecord record : reader)
{
// do stuff with record...
};
SmfRecordReader implements the Iterable<SmfRecord> interface. The loop above will read and process each record until the reader indicates that no more records are available. Each iteration provides the next SmfRecord in the record variable.
Accessing information from the record
Record information can be accessed using record methods:
Integer key = (record.recordType() << 16)
+ (record.hasSubtypes() ? record.subType() : 0);
int length = record.recordLength();
That’s all the SMF processing this program does.
The Java Stuff
You could probably store record type and subtype information in a very sparse 2 dimensional array, but for reduced memory usage this program uses a HashMap with the type and subtype combined into an Integer key, and an object containing the statistics as the value.
Map<Integer, RecordStats> statsMap =
new HashMap<Integer, RecordStats>();
RecordStats is the class that keeps the counters for a type/subtype combination.
The processing is simple:
For each record:
1. Get the type/subtype key
2. Try to get an existing statistics entry for the key
3. If there is no existing entry, create a new RecordStats entry
Integer key = (record.recordType() << 16)
+ (record.hasSubtypes() ? record.subType() : 0);
RecordStats stats = statsMap.get(key);
if (stats == null)
{
statsMap.put(key, new RecordStats(record));
}
else
{
stats.add(record);
}
SMF record type is a 1 byte field and subtype is 2 bytes, so they can both be combined into a 4 byte Integer to form the key.
Output
The writeReport method produces the output.
The statistics entries are sorted by the largest to smallest number of bytes. First the map entries need to be put into a List for sorting:
List<Entry<Integer, RecordStats>> results =
new ArrayList<Entry<Integer, RecordStats>>(statsMap.entrySet());
Then the Collections.sort method is used to sort the list. We can provide a Comparator in the call to sort to sort by anything we like. In this case we compare the bytes in the RecordStats object.
Collections.sort(results,
new Comparator<Entry<Integer, RecordStats>>()
{
public int compare(
Entry<Integer, RecordStats> s1,
Entry<Integer, RecordStats> s2)
{
return Long.compare( // reversed to sort descending
s2.getValue().bytes,
s1.getValue().bytes);
}
});
Then System.out.format is used to write headings and data. The output:
Type Subtype Records MB Pct Min Max Avg
74 5 60384 1330 23.5 4668 32484 23107
30 4 857136 1204 21.3 1137 32740 1473
30 3 853873 1195 21.1 626 32740 1468
74 1 37536 1128 19.9 17668 32632 31517
30 5 97123 236 4.2 1137 32740 2558
...
The complete Java program:
import java.io.FileInputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import com.blackhillsoftware.smf.SmfRecord;
import com.blackhillsoftware.smf.SmfRecordReader;
public class recordcount
{
public static void main( String[] args ) throws IOException
{
// use a HashMap to keep the statistics
Map<Integer, RecordStats> statsMap =
new HashMap<Integer, RecordStats>();
// Read from a DD or file
try (SmfRecordReader reader =
args.length == 0 ?
SmfRecordReader.fromDD("INPUT") :
SmfRecordReader.fromStream(new FileInputStream(args[0])))
{
// SmfRecordReader implements Iterable<SmfRecord>, so we can
// simply iterate to read SmfRecords
for (SmfRecord record : reader)
{
Integer key = (record.recordType() << 16)
+ (record.hasSubtypes() ? record.subType() : 0);
// try to retrieve an existing entry
RecordStats stats = statsMap.get(key);
if (stats == null) // none -> create new
{
statsMap.put(key, new RecordStats(record));
}
else // otherwise add this record to existing
{
stats.add(record);
}
};
}
writeReport(statsMap);
}
private static void writeReport(Map<Integer, RecordStats> statsMap)
{
// Turn the HashMap into a list so it can be sorted
List<Entry<Integer, RecordStats>> results =
new ArrayList<Entry<Integer, RecordStats>>(statsMap.entrySet());
// get the total bytes from all record types
long totalbytes = 0;
for (Entry<Integer, RecordStats> entry : results)
{
totalbytes += entry.getValue().bytes;
}
// sort by total bytes descending
Collections.sort(results,
new Comparator<Entry<Integer, RecordStats>>()
{
public int compare(
Entry<Integer, RecordStats> s1,
Entry<Integer, RecordStats> s2)
{
return Long.compare( // reversed to sort descending
s2.getValue().bytes,
s1.getValue().bytes);
}
}
);
// write heading
System.out.format("%5s %8s %11s %11s %7s %9s %9s %9s%n",
"Type",
"Subtype",
"Records",
"MB",
"Pct",
"Min",
"Max",
"Avg");
// write data
for (Entry<Integer, RecordStats> entry : results)
{
System.out.format("%5d %8d %11d %11d %7.1f %9d% 9d% 9d%n",
entry.getKey().intValue() >> 16,
entry.getKey().intValue() & 0xFFFF,
entry.getValue().count,
entry.getValue().bytes / (1024*1024),
(float)(entry.getValue().bytes) / totalbytes * 100,
entry.getValue().min,
entry.getValue().max,
entry.getValue().bytes / entry.getValue().count);
}
}
/**
* Statistics for a type/subtype combination
*/
private static class RecordStats
{
public RecordStats(SmfRecord record)
{
count = 1;
bytes = min = max = record.recordLength();
}
int count;
long bytes;
int max;
int min;
public void add(SmfRecord record)
{
count++;
int length = record.recordLength();
bytes += length;
if (length < min)
min = length;
if (length > max)
max = length;
}
}
}