VMS Tech Demo 9 - index-sequential files and JVM script languages

Content:

  1. Introduction
  2. Setup
  3. Native code
  4. Java
  5. JVM script - DB style
  6. JVM script - map style
  7. Comparison code size
  8. Comparison execution speed
  9. Conclusion

Introduction:

Index-sequential files are a part of many VMS applications. It is certainly possible to argue that those applications should switch to a relational database, but such migrations take time.

I have previously written about using Jython and my ISAM library for converting index-sequential files to another format like relational database or ISAM file on Windows/Linux:

But now we will look at permanently working with index-sequential files from JVM script languages.

This is for scenarios like:

The script languages being used will be:

Jython is JVM Python which integrates nicely with Java.

Groovy is a JVM specific language, very Java like but requires way less code and integrated nicely with Java.

Jython 2.7 and Groovy 4.0 run with Java 8 and are therefore available on VMS Itanium and VMS x86-64.

Jython 2.5 run with Java 5 and are therefore available on VMS Alpha as well.

Note that Jython is Python 2.x not 3.x. Jython 3.x was never completed. The logical 3.0 replacement for Jython is GraalPy. GraalPy is Python 3.x and tries to be Jython compatible. Unfortunatetly GraalPy is not available for VMS.

Java is available from VSI.

Jython including VMS COM wrappers is available as vmsscript bundle here.

For Groovy just unpack a standard Groovy ZIP and get VMS COM wrappers here.

Java/Jython/Groovy will all use my ISAM library that is available here.

The layers in the file access are:

Layers in tech stack

Setup:

Let us create a test database to use for the demo.

We will use the following class model:

Class model diagram

Note the two potential compliactions:

The data structure is obviously unrealistic simple, but it should contain enough complexity to illustrate the points. Having 10-50 fields in each type would not help understanding just make all code examples longer.

For each language/style we will create two small programs:

No matter the language we will need to define the data structure. VMS index-sequential files only have meta data for key fields - the rest of a record is basically just a BLOB and the application need to provide the structure.

customer.pas:

const
   POTENTIAL = 1;
   ACTUAL = 2;

type
   customer = record
                 id : [key(0)] integer;
                 name : packed array [1..32] of char;
                 phone : packed array [1..16] of char;
                 status : integer;
                 case integer of
                    POTENTIAL:
                       (
                          source : packed array[1..256] of char;
                       );
                    ACTUAL:
                       (
                          address1 : packed array [1..64] of char;
                          address2 : packed array [1..64] of char;
                          contact : packed array [1..32] of char;
                          discount : integer;
                       );
              end;

var
   customer_file : file of customer;

order.pas:

type
   orderline = record
                  item : packed array [1..32] of char;
                  quantity : integer;
                  price : decimal;
               end;
   order = record
              id : [key(0)] integer;              
              customer : [key(1)]integer;
              status : packed array [1..16] of char;
              nlines : integer;
              line : array [1..10] of orderline;
           end;

var
   order_file : file of order;

Besides the data definition then Pascal also need some extra to support the BCD data type.

util.pas:

type
   pstr = varying [255] of char;
   nibble = 0..15;
   decimal = packed array [1..8] of nibble;

function trim(s : packed array[$u..$l:integer] of char) : pstr;

var
   i : integer;

begin
   i := length(s);
   while (i > 0) and (s[i] = ' ') do i := i - 1;
   trim := substr(s, 1, i);
end;

function fromdecimal(v : decimal) : integer;

begin
   fromdecimal := v[2] * 1000000 + v[1] * 100000 + v[4] * 10000 + v[3] * 1000 + v[6] * 100 + v[5] * 10 + v[8];
end;

function todecimal(v : integer) : decimal;

var
   res : decimal;

begin
   res[2] := v div 1000000;
   res[1] := (v div 100000) mod 10;
   res[4] := (v div 10000) mod 10;
   res[3] := (v div 1000) mod 10;
   res[6] := (v div 100) mod 10;
   res[5] := (v div 10) mod 10;
   res[8] := v mod 10;
   res[7] := 12; (* assume positive/unsigned *)
   todecimal := res;
end;

customer.cob:

fd customer-file.
01 customer-record.
    03 customer-id pic s9(8) comp.
    03 customer-name pic x(32).
    03 customer-phone pic x(16).
    03 customer-status pic s9(8) comp.
    03 customer-potential.
        05 customer-source pic x(256).
    03 customer-actual redefines customer-potential.
        05 customer-address1 pic x(64).
        05 customer-address2 pic x(64).
        05 customer-contact pic x(32).
        05 customer-discount pic s9(8) comp.

order.cob:

fd order-file.
01 order-record.
    03 order-id pic s9(8) comp.
    03 order-customer-id pic s9(8) comp.
    03 order-status pic x(16).
    03 order-nlines pic s9(8) comp.
    03 order-line occurs 10 times.
       05 order-line-item pic x(32).
       05 order-line-quantity pic s9(8) comp.
       05 order-line-price pic 9(5)v9(2) packed-decimal.

Besides the data definition Cobol also need a little utility function to help trim trailing spaces.

trim.cob:

identification division.
program-id.trim.

data division.
working-storage section.
01 actlen pic s9(8) comp.
linkage section.
01 s pic x(256).
01 len pic s9(8) comp.

procedure division using s,len giving actlen.
main-paragraph.
    move len to actlen.
    perform varying actlen from len by -1 until s(actlen:1) not = " " or actlen = 1
        continue
    end-perform.
    end program trim.

Customer.java:

import dk.vajhoej.isam.KeyField;
import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Selector;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;
import dk.vajhoej.record.SubType;

@lombok.Getter @lombok.Setter
@Struct
public class Customer {
    public static final int POTENTIAL = 1;
    public static final int ACTUAL = 2;
    @KeyField(n=0)
    @StructField(n=0, type=FieldType.INT4)
    private int id;
    @StructField(n=1, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
    private String name;
    @StructField(n=2, type=FieldType.FIXSTR, length=16, pad=true, padchar=' ')
    private String phone;
    @StructField(n=3, type=FieldType.INT4)
    @Selector(subtypes= {@SubType(value=POTENTIAL, type=CustomerPotential.class),
                         @SubType(value=ACTUAL, type=CustomerActual.class)}, pad=true)
    private int status;
}

CustomerPotential.java:

import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;

@lombok.Getter @lombok.Setter
@Struct
public class CustomerPotential extends Customer {
    @StructField(n=4, type=FieldType.FIXSTR, length=256, pad=true, padchar=' ')
    private String source;
}

CustomerActual.java:

import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;

@lombok.Getter @lombok.Setter
@Struct
public class CustomerActual extends Customer {
    @StructField(n=4, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
    private String address1;
    @StructField(n=5, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
    private String address2;
    @StructField(n=6, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
    private String contact;
    @StructField(n=7, type=FieldType.INT4)
    private int discount;
}

Order.java:

import dk.vajhoej.isam.KeyField;
import dk.vajhoej.record.ArrayField;
import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;

@lombok.Getter @lombok.Setter
@Struct
public class Order {
    @KeyField(n=0)
    @StructField(n=0, type=FieldType.INT4)
    private int id;
    @KeyField(n=1)
    @StructField(n=1, type=FieldType.INT4)
    private int customer;
    @StructField(n=2, type=FieldType.FIXSTR, length=16, pad=true, padchar=' ')
    private String status;
    @StructField(n=3, type=FieldType.INT4)
    private int nlines;
    @StructField(n=4, type=FieldType.STRUCT)
    @ArrayField(elements=10)
    private OrderLine[] line = new OrderLine[10];
}

OrderLine.java:

import java.math.BigDecimal;

import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;

@lombok.Getter @lombok.Setter
@Struct
public class OrderLine {
    @StructField(n=0, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
    private String item;
    @StructField(n=1, type=FieldType.INT4)
    private int quantity;
    @StructField(n=2, type=FieldType.PACKEDBCD, length=4, decimals=2)
    private BigDecimal price;
}

There are multiple ways to implement data classes in Java:

Here we use the last because getters and setters is the Java way and using Lombok keeps focus on what is important.

Build:

$ javac -classpath .:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar:lombok.jar Customer*.java Order*.java

The data classes can also easily be be defined in Groovy.

customer.groovy:

import dk.vajhoej.isam.KeyField
import dk.vajhoej.record.FieldType
import dk.vajhoej.record.Selector
import dk.vajhoej.record.Struct
import dk.vajhoej.record.StructField
import dk.vajhoej.record.SubType

@Struct
class Customer {
    public static final int POTENTIAL = 1
    public static final int ACTUAL = 2
    @KeyField(n=0)
    @StructField(n=0, type=FieldType.INT4)
    int id
    @StructField(n=1, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
    String name
    @StructField(n=2, type=FieldType.FIXSTR, length=16, pad=true, padchar=' ')
    String phone
    @StructField(n=3, type=FieldType.INT4)
    @Selector(subtypes= [@SubType(value=POTENTIAL, type=CustomerPotential.class),
                         @SubType(value=ACTUAL, type=CustomerActual.class)], pad=true)
    int status
}

@Struct
class CustomerPotential extends Customer {
    @StructField(n=4, type=FieldType.FIXSTR, length=256, pad=true, padchar=' ')
    String source
}

@Struct
class CustomerActual extends Customer {
    @StructField(n=4, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
    String address1
    @StructField(n=5, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
    String address2
    @StructField(n=6, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
    String contact
    @StructField(n=7, type=FieldType.INT4)
    int discount
}

order.groovy:

import dk.vajhoej.isam.KeyField
import dk.vajhoej.record.ArrayField
import dk.vajhoej.record.FieldType
import dk.vajhoej.record.Struct
import dk.vajhoej.record.StructField

@Struct
class Order {
    @KeyField(n=0)
    @StructField(n=0, type=FieldType.INT4)
    int id
    @KeyField(n=1)
    @StructField(n=1, type=FieldType.INT4)
    int customer
    @StructField(n=2, type=FieldType.FIXSTR, length=16, pad=true, padchar=' ')
    String status
    @StructField(n=3, type=FieldType.INT4)
    int nlines
    @StructField(n=4, type=FieldType.STRUCT)
    @ArrayField(elements=10)
    OrderLine[] line = new OrderLine[10]
}

@Struct
class OrderLine {
    @StructField(n=0, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
    String item
    @StructField(n=1, type=FieldType.INT4)
    int quantity
    @StructField(n=2, type=FieldType.PACKEDBCD, length=4, decimals=2)
    BigDecimal price
}

Build:

$ groovy_cp == "/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar"
$ groovyc Customer.groovy Order.groovy

And the Groovy code is sligtly simpler than the Java code (even with Lombok), but defining it in Groovy require the Groovy jars in classpath for all programs using the data classes - not only for Groovy programs but also for Java and Jython programs.

People will obviously consider the data definition in the languages they know best to be the most readable, but the reality is that there is little difference. The Cobol definition is actually the shortest!!

Code to load data:

program load(input,output);

%include 'util.pas'

%include 'customer.pas'
%include 'order.pas'

var
   c : customer;
   o : order;

begin
   open(customer_file, 'customer.isq', new, organization := indexed, access_method := keyed);
   rewrite(customer_file);
   c.id := 1;
   c.name := 'A company';
   c.phone := '1111-1111';
   c.status := POTENTIAL;
   c.source := 'Sleazy information seller';
   customer_file^ := c;
   put(customer_file);
   c.id := 2;
   c.name := 'B company';
   c.phone := '2222-2222';
   c.status := POTENTIAL;
   c.source := 'Sleazy information seller';
   customer_file^ := c;
   put(customer_file);
   c.id := 3;
   c.name := 'C company';
   c.phone := '3333-3333';
   c.status := ACTUAL;
   c.address1 := 'C road 3';
   c.address2 := 'C town';
   c.contact := 'Mr. C';
   c.discount := 10;
   customer_file^ := c;
   put(customer_file);
   c.id := 4;
   c.name := 'D company';
   c.phone := '4444-4444';
   c.status := ACTUAL;
   c.address1 := 'D road 4';
   c.address2 := 'D town';
   c.contact := 'Mr. D';
   c.discount := 20;
   customer_file^ := c;
   put(customer_file);
   c.id := 5;
   c.name := 'E company';
   c.phone := '5555-5555';
   c.status := ACTUAL;
   c.address1 := 'E road 5';
   c.address2 := 'E town';
   c.contact := 'Mr. E';
   c.discount := 10;
   customer_file^ := c;
   put(customer_file);
   close(customer_file);
   open(order_file, 'order.isq', new, organization := indexed, access_method := keyed);
   rewrite(order_file);
   o.id := 1;
   o.customer := 3;
   o.status := 'Delivered';
   o.nlines := 1;
   o.line[1].item := 'X stuff';
   o.line[1].quantity := 1;
   o.line[1].price := todecimal(7500);
   order_file^ := o;
   put(order_file);
   o.id := 2;
   o.customer := 4;
   o.status := 'Delivered';
   o.nlines := 1;
   o.line[1].item := 'Y stuff';
   o.line[1].quantity := 10;
   o.line[1].price := todecimal(4000);
   order_file^ := o;
   put(order_file);
   o.id := 3;
   o.customer := 4;
   o.status := 'Delivered';
   o.nlines := 1;
   o.line[1].item := 'Y stuff';
   o.line[1].quantity := 5;
   o.line[1].price := todecimal(4000);
   order_file^ := o;
   put(order_file);
   o.id := 4;
   o.customer := 4;
   o.status := 'Delivered';
   o.nlines := 1;
   o.line[1].item := 'Y stuff';
   o.line[1].quantity := 20;
   o.line[1].price := todecimal(4000);
   order_file^ := o;
   put(order_file);
   o.id := 5;
   o.customer := 5;
   o.status := 'Delivered';
   o.nlines := 3;
   o.line[1].item := 'X stuff';
   o.line[1].quantity := 1;
   o.line[1].price := todecimal(7200);
   o.line[2].item := 'Y stuff';
   o.line[2].quantity := 1;
   o.line[2].price := todecimal(4500);
   o.line[3].item := 'Z stuff';
   o.line[3].quantity := 1;
   o.line[3].price := todecimal(9000);
   order_file^ := o;
   put(order_file);
   close(order_file);
end.

Build and run:

$ pas load
$ link load
$ run load
identification division.
program-id.load.

environment division.
input-output section.
file-control.
    select optional customer-file assign to "customer.isq"
                                  organization is indexed
                                  access mode is dynamic
                                  record key is customer-id.
    select optional order-file assign to "order.isq"
                               organization is indexed
                               access mode is dynamic
                               record key is order-id
                               alternate record key is order-customer-id with duplicates.

data division.
file section.
copy "customer.cob".
copy "order.cob".
working-storage section.
01  POTENTIAL pic s9(8) comp value 1.
01  ACTUAL pic s9(8) comp value 2.
procedure division.
main-paragraph.
    open i-o customer-file
    move 1 to customer-id
    move "A company" to customer-name
    move "1111-1111" to customer-phone
    move POTENTIAL to customer-status
    move "Sleazy information seller" to customer-source
    perform insert-customer-paragraph
    move 2 to customer-id
    move "B company" to customer-name
    move "2222-2222" to customer-phone
    move POTENTIAL to customer-status
    move "Sleazy information seller" to customer-source
    perform insert-customer-paragraph
    move 3 to customer-id
    move "C company" to customer-name
    move "3333-3333" to customer-phone
    move ACTUAL to customer-status
    move "C road 3" to customer-address1
    move "C town" to customer-address2
    move "Mr. C" to customer-contact
    move 10 to customer-discount
    perform insert-customer-paragraph
    move 4 to customer-id
    move "D company" to customer-name
    move "4444-4444" to customer-phone
    move ACTUAL to customer-status
    move "D road 4" to customer-address1
    move "D town" to customer-address2
    move "Mr. D" to customer-contact
    move 20 to customer-discount
    perform insert-customer-paragraph
    move 5 to customer-id
    move "E company" to customer-name
    move "5555-5555" to customer-phone
    move ACTUAL to customer-status
    move "E road 5" to customer-address1
    move "E town" to customer-address2
    move "Mr. E" to customer-contact
    move 10 to customer-discount
    perform insert-customer-paragraph
    close customer-file
    open i-o order-file
    move 1 to order-id
    move 3 to order-customer-id
    move "Delivered" to order-status
    move 1 to order-nlines
    move "X stuff" to order-line-item(1)
    move 1 to order-line-quantity(1)
    move 72.00 to order-line-price(1)
    perform insert-order-paragraph
    move 2 to order-id
    move 4 to order-customer-id
    move "Delivered" to order-status
    move 1 to order-nlines
    move "Y stuff" to order-line-item(1)
    move 10 to order-line-quantity(1)
    move 40.00 to order-line-price(1)
    perform insert-order-paragraph
    move 3 to order-id
    move 4 to order-customer-id
    move "Delivered" to order-status
    move 1 to order-nlines
    move "Y stuff" to order-line-item(1)
    move 5 to order-line-quantity(1)
    move 40.00 to order-line-price(1)
    perform insert-order-paragraph
    move 4 to order-id
    move 4 to order-customer-id
    move "Delivered" to order-status
    move 1 to order-nlines
    move "Y stuff" to order-line-item(1)
    move 20 to order-line-quantity(1)
    move 40.00 to order-line-price(1)
    perform insert-order-paragraph
    move 5 to order-id
    move 5 to order-customer-id
    move "Delivered" to order-status
    move 3 to order-nlines
    move "X stuff" to order-line-item(1)
    move 1 to order-line-quantity(1)
    move 72.00 to order-line-price(1)
    move "Y stuff" to order-line-item(2)
    move 1 to order-line-quantity(2)
    move 45.00 to order-line-price(2)
    move "Z stuff" to order-line-item(3)
    move 1 to order-line-quantity(3)
    move 90.00 to order-line-price(3)
    perform insert-order-paragraph
    close order-file
    stop run.
insert-customer-paragraph.
    write customer-record
        invalid key display "Error writing customer"
        not invalid key continue
    end-write.
insert-order-paragraph.
    write order-record
        invalid key display "Error writing order"
        not invalid key continue
    end-write.

Build and run:

$ cob load
$ link load
$ run load

Native code:

Pascal and Cobol has builtin support for index-sequential files. The support in Pascal is VMS specific. The support in Cobol is standard.

Add code:

program add(input,output);

%include 'util.pas'

%include 'customer.pas'
%include 'order.pas'

var
   c : customer;
   o : order;

begin
   open(customer_file, 'customer.isq', old, organization := indexed, access_method := keyed);
   reset(customer_file);
   c.id := 6;
   c.name := 'F company';
   c.phone := '6666-6666';
   c.status := ACTUAL;
   c.address1 := 'F road 6';
   c.address2 := 'F town';
   c.contact := 'Mr. F';
   c.discount := 10;
   customer_file^ := c;
   put(customer_file);
   close(customer_file);
   open(order_file, 'order.isq', old, organization := indexed, access_method := keyed);
   reset(order_file);
   o.id := 6;
   o.customer := 6;
   o.status := 'In progress';
   o.nlines := 1;
   o.line[1].item := 'X stuff';
   o.line[1].quantity := 1;
   o.line[1].price := todecimal(7200);
   order_file^ := o;
   put(order_file);
   close(order_file);
end.

Build and run:

$ pas add
$ link add
$ run add
identification division.
program-id.myadd.

environment division.
input-output section.
file-control.
    select optional customer-file assign to "customer.isq"
                                  organization is indexed
                                  access mode is dynamic
                                  record key is customer-id.
    select optional order-file assign to "order.isq"
                               organization is indexed
                               access mode is dynamic
                               record key is order-id
                               alternate record key is order-customer-id with duplicates.

data division.
file section.
copy "customer.cob".
copy "order.cob".
working-storage section.
01  POTENTIAL pic s9(8) comp value 1.
01  ACTUAL pic s9(8) comp value 2.
procedure division.
main-paragraph.
    open i-o customer-file
    move 6 to customer-id
    move "F company" to customer-name
    move "6666-6666" to customer-phone
    move ACTUAL to customer-status
    move "F road 6" to customer-address1
    move "F town" to customer-address2
    move "Mr. F" to customer-contact
    move 10 to customer-discount
    perform insert-customer-paragraph
    close customer-file
    open i-o order-file
    move 6 to order-id
    move 6 to order-customer-id
    move "In progress" to order-status
    move 1 to order-nlines
    move "X stuff" to order-line-item(1)
    move 1 to order-line-quantity(1)
    move 72.00 to order-line-price(1)
    perform insert-order-paragraph
    close order-file
    stop run.
insert-customer-paragraph.
    write customer-record
        invalid key display "Error writing customer"
        not invalid key continue
    end-write.
insert-order-paragraph.
    write order-record
        invalid key display "Error writing order"
        not invalid key continue
    end-write.

Build and run:

$ cob add
$ link add
$ run add

List code:

program list(input,output);

%include 'util.pas'

%include 'customer.pas'
%include 'order.pas'

var
   c : customer;
   o : order;
   sum, i : integer;

begin
   open(customer_file, 'customer.isq', old, organization := indexed, access_method := keyed);
   reset(customer_file);
   open(order_file, 'order.isq', old, organization := indexed, access_method := keyed);
   reset(order_file);
   while not eof(customer_file) do begin
      c := customer_file^;
      if c.status = POTENTIAL then begin
         write('**future customer** ');
         write(trim(c.name));
         write(', ');
         write(trim(c.phone));
         write(', ');
         write(trim(c.source));
         writeln;
      end else if c.status = ACTUAL then begin
         write(c.id:1);
         write(', ');
         write(trim(c.name));
         write(', ');
         write(trim(c.address1));
         write(', ');
         write(trim(c.address2));
         write(', ');
         write(trim(c.phone));
         writeln;
         findk(order_file, 1, c.id);
         while (not eof(order_file)) and (not ufb(order_file)) and (order_file^.customer = c.id) do begin
            o := order_file^;
            sum := 0;
            for i := 1 to o.nlines do begin
               sum := sum + o.line[i].quantity * fromdecimal(o.line[i].price);
            end;
            write('  ');
            write(o.id:1);
            write(', ');
            write(trim(o.status));
            write(', ');
            write((sum/100):1:2);
            writeln;
            get(order_file);
         end;
      end;
      get(customer_file);
   end;
   close(order_file);
   close(customer_file);
end.

Build and run:

$ pas list
$ link list
$ run list
identification division.
program-id.list.

environment division.
input-output section.
file-control.
    select optional customer-file assign to "customer.isq"
                                  organization is indexed
                                  access mode is dynamic
                                  record key is customer-id.
    select optional order-file assign to "order.isq"
                               organization is indexed
                               access mode is dynamic
                               record key is order-id
                               alternate record key is order-customer-id with duplicates.

data division.
file section.
copy "customer.cob".
copy "order.cob".
working-storage section.
01  POTENTIAL pic s9(8) comp value 1.
01  ACTUAL pic s9(8) comp value 2.
01  eof-flag pic x(1).
01  done-flag pic x(1).
01  len pic s9(8) comp.
01  name-len pic s9(8) comp.
01  phone-len pic s9(8) comp.
01  source-len pic s9(8) comp.
01  address1-len pic s9(8) comp.
01  address2-len pic s9(8) comp.
01  status-len pic s9(8) comp.
01  id2 pic s9(8) display.
01  order-sum pic 9(5)v9(2) packed-decimal.
01  order-sum2 pic 9(5)v9(2) display.
01  i pic s9(8) comp.
procedure division.
main-paragraph.
    open i-o customer-file
    open i-o order-file
    move "N" to eof-flag
    perform until eof-flag = "Y"
        read customer-file next
            at end move "Y" to eof-flag
            not at end perform display-customer-paragraph
        end-read
    end-perform
    close customer-file
    close order-file
    stop run.
display-customer-paragraph.
    if customer-status = POTENTIAL
        move 32 to len
        call "trim" using customer-name, len giving name-len
        move 16 to len
        call "trim" using customer-phone, len giving phone-len
        move 256 to len
        call "trim" using customer-source, len giving source-len
        display "**future customer** " customer-name(1:name-len) ", " customer-phone(1:phone-len) ", " customer-source(1:source-len)
    end-if
    if customer-status = ACTUAL
        move customer-id to id2
        move 32 to len
        call "trim" using customer-name, len giving name-len
        move 64 to len
        call "trim" using customer-address1, len giving address1-len
        move 64 to len
        call "trim" using customer-address2, len giving address2-len
        move 16 to len
        call "trim" using customer-phone, len giving phone-len
        display id2 ", " customer-name(1:name-len) ", " customer-address1(1:address1-len) ", " customer-address2(1:address2-len) ", " customer-phone(1:phone-len)
        perform load-orders-paragraph
    end-if.
load-orders-paragraph.
    move customer-id to order-customer-id
    start order-file key is greater than or equal to order-customer-id
        invalid key display "Error searching orders"
        not invalid key continue
    end-start
    move 'N' to eof-flag
    move 'N' to done-flag
    perform until eof-flag = 'Y' or done-flag = 'Y'
        read order-file next
            at end move 'Y' to eof-flag
            not at end perform display-order-paragraph
        end-read
    end-perform.
display-order-paragraph.
    if order-customer-id = customer-id then
        move order-id to id2
        move 16 to len
        call "trim" using order-status, len giving status-len
        move 0.00 to order-sum
        perform varying i from 1 by 1 until i > 10
            compute order-sum = order-sum + order-line-quantity(i) * order-line-price(i)
        end-perform
        move order-sum to order-sum2
        display id2 ", " order-status(1:status-len) ", " order-sum2
    else
        move 'Y' to done-flag
    end-if.

Build and run:

$ cob trim
$ cob list
$ link list + trim
$ run list

Java:

Add code:

import java.math.BigDecimal;

import dk.vajhoej.isam.IsamException;
import dk.vajhoej.isam.IsamSource;
import dk.vajhoej.isam.local.LocalIsamSource;
import dk.vajhoej.record.RecordException;

public class Add {
    public static void main(String[] args) throws IsamException, RecordException {
        IsamSource cisam = new LocalIsamSource("customer.isq", "dk.vajhoej.vms.rms.IndexSequential", false);
        IsamSource oisam = new LocalIsamSource("order.isq", "dk.vajhoej.vms.rms.IndexSequential", false);
        CustomerActual c = new CustomerActual();
        c.setId(7);
        c.setName("G company");
        c.setPhone("7777-7777");
        c.setStatus(Customer.ACTUAL);
        c.setAddress1("G road 7");
        c.setAddress2("G town");
        c.setContact("Mr. G");
        cisam.create(c);
        Order o = new Order();
        o.setId(7);
        o.setCustomer(7);
        o.setStatus("In progress");
        o.setNlines(1);
        OrderLine ol = new OrderLine();
        ol.setItem("X stuff");
        ol.setQuantity(1);
        ol.setPrice(new BigDecimal("72.00"));
        o.getLine()[0] = ol;
        for(int i = 1; i < 10; i++) o.getLine()[i] = new OrderLine();
        oisam.create(o);
        cisam.close();
        oisam.close();
    }
}

Build and run:

$ javac -classpath .:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar Add.java
$ java -classpath .:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar Add

List code:

import java.math.BigDecimal;

import dk.vajhoej.isam.IsamException;
import dk.vajhoej.isam.IsamResult;
import dk.vajhoej.isam.IsamSource;
import dk.vajhoej.isam.Key;
import dk.vajhoej.isam.local.LocalIsamSource;
import dk.vajhoej.record.RecordException;

public class List {
    public static void main(String[] args) throws IsamException, RecordException {
        IsamSource cisam = new LocalIsamSource("customer.isq", "dk.vajhoej.vms.rms.IndexSequential", false);
        IsamSource oisam = new LocalIsamSource("order.isq", "dk.vajhoej.vms.rms.IndexSequential", false);
        IsamResult<Customer> crs = cisam.readStart(Customer.class);
        while(crs.read()) {
            Customer c = crs.current();
            if(c instanceof CustomerPotential) {
                CustomerPotential cp = (CustomerPotential)c;
                System.out.printf("**future customer** %s, %s, %s\n", cp.getName(), cp.getPhone(), cp.getSource());
            }
            if(c instanceof CustomerActual) {
                CustomerActual ca = (CustomerActual)c;
                System.out.printf("%d, %s, %s, %s, %s\n", ca.getId(), ca.getName(), ca.getAddress1(), ca.getAddress2(), ca.getPhone());
                IsamResult<Order> ors = oisam.readGE(Order.class, new Key<Integer>(1, ca.getId()));
                while(ors.read()) {
                    Order o = ors.current();
                    if(o.getCustomer() > ca.getId()) break;
                    BigDecimal sum = new BigDecimal(0);
                    for(int i = 0; i < o.getNlines(); i++) {
                        sum = sum.add(o.getLine()[i].getPrice().multiply(new BigDecimal(o.getLine()[i].getQuantity())));
                    }
                    System.out.printf("  %d, %s, %.2f\n", o.getId(), o.getStatus(), sum);
                }
            }
        }
        cisam.close();
        oisam.close();
    }
}

Build and run:

$ javac -classpath .:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar List.java
$ java -classpath .:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar List

JVM script - DB style:

This is basically just writing the Java code in Python or Groovy.

Add code:

from java.math import BigDecimal

from dk.vajhoej.isam.local import LocalIsamSource

import Customer
import CustomerActual
import Order
import OrderLine

cisam = LocalIsamSource('customer.isq', 'dk.vajhoej.vms.rms.IndexSequential', False)
oisam = LocalIsamSource('order.isq', 'dk.vajhoej.vms.rms.IndexSequential', False)
c = CustomerActual()
c.id = 8
c.name = 'H company'
c.phone = '8888-8888'
c.status = Customer.ACTUAL
c.address1 = 'H road 8'
c.address2 = 'H town'
c.contact = 'Mr. H'
cisam.create(c)
o = Order()
o.id = 8
o.customer = 8
o.status = 'In progress'
o.nlines = 1
ol = OrderLine()
ol.item = 'X stuff'
ol.quantity = 1
ol.price = BigDecimal('72.00')
o.line[0] = ol
for i in range(9):
    o.line[1 + i] = OrderLine()
oisam.create(o)
cisam.close()
oisam.close()

Run:

$ define/nolog jython_libs "/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ jython add.py
import dk.vajhoej.isam.local.LocalIsamSource

cisam = new LocalIsamSource("customer.isq", "dk.vajhoej.vms.rms.IndexSequential", false)
oisam = new LocalIsamSource("order.isq", "dk.vajhoej.vms.rms.IndexSequential", false)
c = new CustomerActual()
c.id = 8
c.name = "H company"
c.phone = "8888-8888"
c.status = Customer.ACTUAL
c.address1 = "H road 8"
c.address2 = "H town"
c.contact = "Mr. H"
cisam.create(c)
o = new Order()
o.id = 8
o.customer = 8
o.status = "In progress"
o.nlines = 1
ol = new OrderLine()
ol.item = "X stuff"
ol.quantity = 1
ol.price = 72.00
o.line[0] = ol
for(i in 1..9) o.line[i] = new OrderLine()
oisam.create(o)
cisam.close()
oisam.close()

Run:

$ groovy_cp == ".:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ groovy add.groovy

List code:

from java.lang import Integer
from java.math import BigDecimal

from dk.vajhoej.isam import Key
from dk.vajhoej.isam.local import LocalIsamSource

import Customer
import Order
import CustomerPotential
import CustomerActual

cisam = LocalIsamSource('customer.isq', 'dk.vajhoej.vms.rms.IndexSequential', False)
oisam = LocalIsamSource('order.isq', 'dk.vajhoej.vms.rms.IndexSequential', False)
crs = cisam.readStart(Customer)
while crs.read():
    c = crs.current()
    if isinstance(c, CustomerPotential):
        print('**future customer** %s, %s, %s' % (c.name, c.phone, c.source))
    if isinstance(c, CustomerActual):
        print('%d, %s, %s, %s, %s' % (c.id, c.address1, c.address2, c.name, c.phone))
        ors = oisam.readGE(Order, Key(1, Integer(c.id)))
        while ors.read():
            o = ors.current()
            if o.customer > c.id:
                break
            sum = BigDecimal(0)
            for i in range(o.nlines):
                sum = sum.add(o.line[i].price.multiply(BigDecimal(o.line[i].quantity)))
            print('  %d, %s, %s' % (o.id, o.status, sum))
        ors.close();
crs.close()
cisam.close()
oisam.close()

Run:

$ define/nolog jython_libs "/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ jython list.py
import dk.vajhoej.isam.Key
import dk.vajhoej.isam.local.LocalIsamSource

cisam = new LocalIsamSource("customer.isq", "dk.vajhoej.vms.rms.IndexSequential", false)
oisam = new LocalIsamSource("order.isq", "dk.vajhoej.vms.rms.IndexSequential", false)
crs = cisam.readStart(Customer.class)
while(crs.read()) {
    c = crs.current()
    if(c instanceof CustomerPotential) {
        println(String.format("**future customer** %s, %s, %s", c.name, c.phone, c.source))
    }
    if(c instanceof CustomerActual) {
        println(String.format("%d, %s, %s, %s, %s", c.id, c.address1, c.address2, c.name, c.phone))
        ors = oisam.readGE(Order.class, new Key(1, c.id));
        while(ors.read()) {
            o = ors.current()
            if(o.customer > c.id) break;
            sum = 0.00
            for(i in 0..o.nlines-1) {
                sum += o.line[i].quantity * o.line[i].price
            }
            println(String.format("  %d, %s, %.2f", o.id, o.status, sum))
        }
        ors.close()
    }
}
crs.close()
cisam.close()
oisam.close()

Run:

$ groovy_cp == ".:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ groovy list.groovy

JVM script - map style:

This is utilizing the map view on top of the basic ISAM library.

Add code:

from java.math import BigDecimal

from dk.vajhoej.isam.map import PyIsamMap

import Customer
import CustomerActual
import Order

cmap = PyIsamMap.createIsamMapRMS('customer.isq', CustomerActual)
cmap.putDict({'id': 9, \
              'name': 'I company', \
              'phone': '9999-9999',
              'status': Customer.ACTUAL, \
              'address1': 'I road 9', \
              'address2': 'I town', \
              'contact': 'Mr. I'})
omap = PyIsamMap.createIsamMapRMS('order.isq', Order)
omap.putDict({'id': 9, \
              'customer': 9, \
              'status': 'In progress', \
              'nlines': 1, \
              'line': [{'item': 'X stuff', 'quantity': 1, 'price': BigDecimal('72.00')}]})

Run:

$ define/nolog jython_libs "/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ jython addx.py
import dk.vajhoej.isam.map.IsamMap

cmap = IsamMap.createIsamMapRMS("customer.isq", Customer)
c = new CustomerActual(id: 9,
                       name: "I company",
                       phone: "9999-9999",
                       status: Customer.ACTUAL,
                       address1: "I road 9",
                       address2: "I town",
                       contact: "Mr. I")
cmap[c.id] = c
omap = IsamMap.createIsamMapRMS("order.isq", Order)
o = new Order(id: 9,
              customer: 9,
              status: "In progress",
              nlines: 1)
o.line[0] = new OrderLine(item: "X stuff", quantity: 1, price: 72.00)
for(i in 1..9) o.line[i] = new OrderLine()
omap[o.id] = o

Run:

$ groovy_cp == ".:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ groovy addx.groovy

List code:

from java.math import BigDecimal

from dk.vajhoej.isam.map import PyIsamMap

import Customer
import Order
import CustomerPotential
import CustomerActual

cmap = PyIsamMap.createIsamMapRMS('customer.isq', Customer)
omap = PyIsamMap.createIsamMapRMS('order.isq', Order)
for c in cmap.values():
    if isinstance(c, CustomerPotential):
        print('**future customer** %s, %s, %s' % (c.name, c.phone, c.source))
    if isinstance(c, CustomerActual):
        print('%d, %s, %s, %s, %s' % (c.id, c.address1, c.address2, c.name, c.phone))
        for o in omap.keyName('customer').isInt(c.id).values():
           sum = BigDecimal(0)
           for i in range(o.nlines):
               sum = sum.add(o.line[i].price.multiply(BigDecimal(o.line[i].quantity)))
           print('  %d, %s, %s' % (o.id, o.status, sum))

Run:

$ define/nolog jython_libs "/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ jython listx.py
import dk.vajhoej.isam.map.IsamMap

cmap = IsamMap.createIsamMapRMS("customer.isq", Customer)
omap = IsamMap.createIsamMapRMS("order.isq", Order)
for(c in cmap.values()) {
    if(c instanceof CustomerPotential) {
        println(String.format("**future customer** %s, %s, %s", c.name, c.phone, c.source))
    }
    if(c instanceof CustomerActual) {
        println(String.format("%d, %s, %s, %s, %s", c.id, c.address1, c.address2, c.name, c.phone))
        for(o in omap.key(1).is(c.id).values()) {
            sum = 0.00
            for(i in 0..o.nlines-1) {
                sum += o.line[i].quantity * o.line[i].price
            }
            println(String.format("  %d, %s, %.2f", o.id, o.status, sum))
        }
    }
}

Run:

$ groovy_cp == ".:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ groovy listx.groovy

Comparison code size:

Language & style Lines for add Lines for list Lines total
Cobol 57 101 158
Pascal 38 60 98
Java 36 39 75
Jython - DB style 35/ 33 68
Jython - map style 22 21 43
Groovy - DB style 27 29 56
Groovy - map style 19 19 38

We see as expected that the script languages using the map interface instead of the database interface require much less code to implement the same functionality.

Obviously the number of lines depends on code formatting style, but I hope and believe I have chosen a reasonable common code formatting style. The size of the Cobol code has some additional uncertainty because I am not experienced with Cobol. But even with those caveats then I feel comfortable that the difference in code sizes is real.

Comparison execution speed:

Being able to write so short code in such a flexible way does not come for free - it comes with a large overhead.

There is a very large startup overhead getting JVM started, classes loaded, class structures analyzed etc..

There is a large overhead processing records due to reflection access to objects, JNI calls to RMS (yes - a JNI call adds lots of overhead) and explicit iteration over fields.

We will test how big this overhead is.

New data structure with more fields:

type
   perf = record
              id : [key(0)] integer;              
              iv1 : integer;
              iv2 : integer;
              iv3 : integer;
              iv4 : integer;
              iv5 : integer;
              sv1 : packed array[1..64] of char;
              sv2 : packed array[1..64] of char;
              sv3 : packed array[1..64] of char;
              sv4 : packed array[1..64] of char;
              sv5 : packed array[1..64] of char;
           end;

var
   perf_file : file of perf;

import dk.vajhoej.isam.KeyField;
import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;

@lombok.Getter @lombok.Setter
@Struct
public class Perf {
    @KeyField(n=0)
    @StructField(n=0, type=FieldType.INT4)
    private int id;
    @StructField(n=1, type=FieldType.INT4)
    private int iv1;
    @StructField(n=2, type=FieldType.INT4)
    private int iv2;
    @StructField(n=3, type=FieldType.INT4)
    private int iv3;
    @StructField(n=4, type=FieldType.INT4)
    private int iv4;
    @StructField(n=5, type=FieldType.INT4)
    private int iv5;
    @StructField(n=6, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
    private String sv1;
    @StructField(n=7, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
    private String sv2;
    @StructField(n=8, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
    private String sv3;
    @StructField(n=9, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
    private String sv4;
    @StructField(n=10, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
    private String sv5;
}

We create a bigger data file with 100000 records:

program loadperf(input,output);

%include 'util.pas'

%include 'perf.pas'

var
   p : perf;
   i : integer;

begin
   open(perf_file, 'perf.isq', new, organization := indexed, access_method := keyed);
   rewrite(perf_file);
   for i := 1 to 100000 do begin
      p.id := i;
      p.iv1 := i + 1;
      p.iv2 := i + 2;
      p.iv3 := i + 3;
      p.iv4 := i + 4;
      p.iv5 := i + 5;
      p.sv1 := 'Text 1 - #' + dec(i, 6);
      p.sv2 := 'Text 2 - #' + dec(i, 6);
      p.sv3 := 'Text 3 - #' + dec(i, 6);
      p.sv4 := 'Text 4 - #' + dec(i, 6);
      p.sv5 := 'Text 5 - #' + dec(i, 6);
      perf_file^ := p;
      put(perf_file);
   end;
   close(perf_file);
end.

And then we will measure query speed. We will do 100000 queries each fetching 10 consecutive records at a random location in the file.

[inherit('sys$library:pascal$lib_routines')]
program listperf(input,output);

%include 'util.pas'

%include 'perf.pas'

var
   p : perf;
   cmdlin : pstr;
   lim, i, ix, n : integer;

begin
   lib$get_foreign(resultant_string := cmdlin.body, resultant_length := cmdlin.length);
   readv(cmdlin, lim);
   open(perf_file, 'perf.isq', old, organization := indexed, access_method := keyed);
   reset(perf_file);
   seed(clock);
   n := 0;
   for i := 1 to lim do begin
      ix := 1 + trunc(99990 * random);
      findk(perf_file, 0, ix);
      while not(ufb(perf_file)) and (perf_file^.id < ix + 10) do begin
         p := perf_file^;
         n := n + 1;
         get(perf_file);
      end;
   end;
   writeln(n:1);
   close(perf_file);
end.
import java.math.BigDecimal;
import java.util.Random;

import dk.vajhoej.isam.IsamException;
import dk.vajhoej.isam.IsamResult;
import dk.vajhoej.isam.IsamSource;
import dk.vajhoej.isam.Key0;
import dk.vajhoej.isam.local.LocalIsamSource;
import dk.vajhoej.record.RecordException;

public class ListPerf {
    public static void main(String[] args) throws IsamException, RecordException {
        int lim = Integer.parseInt(args[0]);
        IsamSource pisam = new LocalIsamSource("perf.isq", "dk.vajhoej.vms.rms.IndexSequential", false);
        Random rng = new Random();
        int n = 0;
        for(int i = 0; i < lim; i++) {
            int ix = 1 + rng.nextInt(99990);
            IsamResult<Perf> prs = pisam.readGE(Perf.class, new Key0<Integer>(ix));
            while(prs.read()) {
                Perf p = prs.current();
                if(p.getId() >= ix + 10) break;
                n++;
            }
            prs.close();
        }
        System.out.println(n);
        pisam.close();
    }
}
import sys

from java.util import Random

from dk.vajhoej.isam.map import PyIsamMap

import Perf

lim = int(sys.argv[1])
pmap = PyIsamMap.createIsamMapRMS('perf.isq', Perf)
rng = Random()
n = 0
for i in range(lim):
    ix = 1 + rng.nextInt(99990)
    for p in pmap.keyName('id').betweenInt(ix, ix + 9).values():
        n = n + 1
print(n)
import java.util.Random

import dk.vajhoej.isam.map.IsamMap

lim = Integer.parseInt(args[0])
pmap = IsamMap.createIsamMapRMS("perf.isq", Perf)
rng = new Random()
n = 0
for(j in 1..lim) {
    ix = 1 + rng.nextInt(99990)
    for(p in pmap.key().between(ix, ix + 9).values()) {
        n++
    }
}
println(n)

Results:

Language & style Startup time Query time
Pascal 0 s 0.139 ms
Java 2 s 0.842 ms
Jython - map style 19 s 1.121 ms
Groovy - map style 26 s 1.093 ms

We see there is a factor 6-8 overhead of using Java or a JVM script languages (for the queries - on top of that comes startup overhead).

The reason for the relative small difference between Java and Jython/Groovy for queries is that they both call the same Java library to do the actual work - the difference is only in the orchestration of the library calls.

That is a really huge overhead. If you intend to do many queries per second then the scripting solution is not a good solution for you. If you intend to do a few queries per minute or some queries per hour, then the speed of the scripting language is really not a problem.

Conclusion:

This is another tool in the toolbox worth considering when planning roadmap for your VMS application using index-sequential files.

Article history:

Version Date Description
1.0 October 21st 2023 Initial version

Other articles:

See list of all articles here

Comments:

Please send comments to Arne Vajhøj