Index-sequential files are a part of many VMS applications. It is certainly possible to argue that those applications should switch to a relational database, but such migrations take time.
I have previously written about using Jython and my ISAM library for converting index-sequential files to another format like relational database or ISAM file on Windows/Linux:
But now we will look at permanently working with index-sequential files from JVM script languages.
This is for scenarios like:
The script languages being used will be:
Jython is JVM Python which integrates nicely with Java.
Groovy is a JVM specific language, very Java like but requires way less code and integrated nicely with Java.
Jython 2.7 and Groovy 4.0 run with Java 8 and are therefore available on VMS Itanium and VMS x86-64.
Jython 2.5 run with Java 5 and are therefore available on VMS Alpha as well.
Note that Jython is Python 2.x not 3.x. Jython 3.x was never completed. The logical 3.0 replacement for Jython is GraalPy. GraalPy is Python 3.x and tries to be Jython compatible. Unfortunatetly GraalPy is not available for VMS.
Java is available from VSI.
Jython including VMS COM wrappers is available as vmsscript bundle here.
For Groovy just unpack a standard Groovy ZIP and get VMS COM wrappers here.
Java/Jython/Groovy will all use my ISAM library that is available here.
The layers in the file access are:
Let us create a test database to use for the demo.
We will use the following class model:
Note the two potential compliactions:
The data structure is obviously unrealistic simple, but it should contain enough complexity to illustrate the points. Having 10-50 fields in each type would not help understanding just make all code examples longer.
For each language/style we will create two small programs:
No matter the language we will need to define the data structure. VMS index-sequential files only have meta data for key fields - the rest of a record is basically just a BLOB and the application need to provide the structure.
customer.pas:
const
POTENTIAL = 1;
ACTUAL = 2;
type
customer = record
id : [key(0)] integer;
name : packed array [1..32] of char;
phone : packed array [1..16] of char;
status : integer;
case integer of
POTENTIAL:
(
source : packed array[1..256] of char;
);
ACTUAL:
(
address1 : packed array [1..64] of char;
address2 : packed array [1..64] of char;
contact : packed array [1..32] of char;
discount : integer;
);
end;
var
customer_file : file of customer;
order.pas:
type
orderline = record
item : packed array [1..32] of char;
quantity : integer;
price : decimal;
end;
order = record
id : [key(0)] integer;
customer : [key(1)]integer;
status : packed array [1..16] of char;
nlines : integer;
line : array [1..10] of orderline;
end;
var
order_file : file of order;
Besides the data definition then Pascal also need some extra to support the BCD data type.
util.pas:
type
pstr = varying [255] of char;
nibble = 0..15;
decimal = packed array [1..8] of nibble;
function trim(s : packed array[$u..$l:integer] of char) : pstr;
var
i : integer;
begin
i := length(s);
while (i > 0) and (s[i] = ' ') do i := i - 1;
trim := substr(s, 1, i);
end;
function fromdecimal(v : decimal) : integer;
begin
fromdecimal := v[2] * 1000000 + v[1] * 100000 + v[4] * 10000 + v[3] * 1000 + v[6] * 100 + v[5] * 10 + v[8];
end;
function todecimal(v : integer) : decimal;
var
res : decimal;
begin
res[2] := v div 1000000;
res[1] := (v div 100000) mod 10;
res[4] := (v div 10000) mod 10;
res[3] := (v div 1000) mod 10;
res[6] := (v div 100) mod 10;
res[5] := (v div 10) mod 10;
res[8] := v mod 10;
res[7] := 12; (* assume positive/unsigned *)
todecimal := res;
end;
customer.cob:
fd customer-file.
01 customer-record.
03 customer-id pic s9(8) comp.
03 customer-name pic x(32).
03 customer-phone pic x(16).
03 customer-status pic s9(8) comp.
03 customer-potential.
05 customer-source pic x(256).
03 customer-actual redefines customer-potential.
05 customer-address1 pic x(64).
05 customer-address2 pic x(64).
05 customer-contact pic x(32).
05 customer-discount pic s9(8) comp.
order.cob:
fd order-file.
01 order-record.
03 order-id pic s9(8) comp.
03 order-customer-id pic s9(8) comp.
03 order-status pic x(16).
03 order-nlines pic s9(8) comp.
03 order-line occurs 10 times.
05 order-line-item pic x(32).
05 order-line-quantity pic s9(8) comp.
05 order-line-price pic 9(5)v9(2) packed-decimal.
Besides the data definition Cobol also need a little utility function to help trim trailing spaces.
trim.cob:
identification division.
program-id.trim.
data division.
working-storage section.
01 actlen pic s9(8) comp.
linkage section.
01 s pic x(256).
01 len pic s9(8) comp.
procedure division using s,len giving actlen.
main-paragraph.
move len to actlen.
perform varying actlen from len by -1 until s(actlen:1) not = " " or actlen = 1
continue
end-perform.
end program trim.
Customer.java:
import dk.vajhoej.isam.KeyField;
import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Selector;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;
import dk.vajhoej.record.SubType;
@lombok.Getter @lombok.Setter
@Struct
public class Customer {
public static final int POTENTIAL = 1;
public static final int ACTUAL = 2;
@KeyField(n=0)
@StructField(n=0, type=FieldType.INT4)
private int id;
@StructField(n=1, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
private String name;
@StructField(n=2, type=FieldType.FIXSTR, length=16, pad=true, padchar=' ')
private String phone;
@StructField(n=3, type=FieldType.INT4)
@Selector(subtypes= {@SubType(value=POTENTIAL, type=CustomerPotential.class),
@SubType(value=ACTUAL, type=CustomerActual.class)}, pad=true)
private int status;
}
CustomerPotential.java:
import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;
@lombok.Getter @lombok.Setter
@Struct
public class CustomerPotential extends Customer {
@StructField(n=4, type=FieldType.FIXSTR, length=256, pad=true, padchar=' ')
private String source;
}
CustomerActual.java:
import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;
@lombok.Getter @lombok.Setter
@Struct
public class CustomerActual extends Customer {
@StructField(n=4, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
private String address1;
@StructField(n=5, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
private String address2;
@StructField(n=6, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
private String contact;
@StructField(n=7, type=FieldType.INT4)
private int discount;
}
Order.java:
import dk.vajhoej.isam.KeyField;
import dk.vajhoej.record.ArrayField;
import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;
@lombok.Getter @lombok.Setter
@Struct
public class Order {
@KeyField(n=0)
@StructField(n=0, type=FieldType.INT4)
private int id;
@KeyField(n=1)
@StructField(n=1, type=FieldType.INT4)
private int customer;
@StructField(n=2, type=FieldType.FIXSTR, length=16, pad=true, padchar=' ')
private String status;
@StructField(n=3, type=FieldType.INT4)
private int nlines;
@StructField(n=4, type=FieldType.STRUCT)
@ArrayField(elements=10)
private OrderLine[] line = new OrderLine[10];
}
OrderLine.java:
import java.math.BigDecimal;
import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;
@lombok.Getter @lombok.Setter
@Struct
public class OrderLine {
@StructField(n=0, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
private String item;
@StructField(n=1, type=FieldType.INT4)
private int quantity;
@StructField(n=2, type=FieldType.PACKEDBCD, length=4, decimals=2)
private BigDecimal price;
}
There are multiple ways to implement data classes in Java:
Here we use the last because getters and setters is the Java way and using Lombok keeps focus on what is important.
Build:
$ javac -classpath .:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar:lombok.jar Customer*.java Order*.java
The data classes can also easily be be defined in Groovy.
customer.groovy:
import dk.vajhoej.isam.KeyField
import dk.vajhoej.record.FieldType
import dk.vajhoej.record.Selector
import dk.vajhoej.record.Struct
import dk.vajhoej.record.StructField
import dk.vajhoej.record.SubType
@Struct
class Customer {
public static final int POTENTIAL = 1
public static final int ACTUAL = 2
@KeyField(n=0)
@StructField(n=0, type=FieldType.INT4)
int id
@StructField(n=1, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
String name
@StructField(n=2, type=FieldType.FIXSTR, length=16, pad=true, padchar=' ')
String phone
@StructField(n=3, type=FieldType.INT4)
@Selector(subtypes= [@SubType(value=POTENTIAL, type=CustomerPotential.class),
@SubType(value=ACTUAL, type=CustomerActual.class)], pad=true)
int status
}
@Struct
class CustomerPotential extends Customer {
@StructField(n=4, type=FieldType.FIXSTR, length=256, pad=true, padchar=' ')
String source
}
@Struct
class CustomerActual extends Customer {
@StructField(n=4, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
String address1
@StructField(n=5, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
String address2
@StructField(n=6, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
String contact
@StructField(n=7, type=FieldType.INT4)
int discount
}
order.groovy:
import dk.vajhoej.isam.KeyField
import dk.vajhoej.record.ArrayField
import dk.vajhoej.record.FieldType
import dk.vajhoej.record.Struct
import dk.vajhoej.record.StructField
@Struct
class Order {
@KeyField(n=0)
@StructField(n=0, type=FieldType.INT4)
int id
@KeyField(n=1)
@StructField(n=1, type=FieldType.INT4)
int customer
@StructField(n=2, type=FieldType.FIXSTR, length=16, pad=true, padchar=' ')
String status
@StructField(n=3, type=FieldType.INT4)
int nlines
@StructField(n=4, type=FieldType.STRUCT)
@ArrayField(elements=10)
OrderLine[] line = new OrderLine[10]
}
@Struct
class OrderLine {
@StructField(n=0, type=FieldType.FIXSTR, length=32, pad=true, padchar=' ')
String item
@StructField(n=1, type=FieldType.INT4)
int quantity
@StructField(n=2, type=FieldType.PACKEDBCD, length=4, decimals=2)
BigDecimal price
}
Build:
$ groovy_cp == "/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar"
$ groovyc Customer.groovy Order.groovy
And the Groovy code is sligtly simpler than the Java code (even with Lombok), but defining it in Groovy require the Groovy jars in classpath for all programs using the data classes - not only for Groovy programs but also for Java and Jython programs.
People will obviously consider the data definition in the languages they know best to be the most readable, but the reality is that there is little difference. The Cobol definition is actually the shortest!!
Code to load data:
program load(input,output);
%include 'util.pas'
%include 'customer.pas'
%include 'order.pas'
var
c : customer;
o : order;
begin
open(customer_file, 'customer.isq', new, organization := indexed, access_method := keyed);
rewrite(customer_file);
c.id := 1;
c.name := 'A company';
c.phone := '1111-1111';
c.status := POTENTIAL;
c.source := 'Sleazy information seller';
customer_file^ := c;
put(customer_file);
c.id := 2;
c.name := 'B company';
c.phone := '2222-2222';
c.status := POTENTIAL;
c.source := 'Sleazy information seller';
customer_file^ := c;
put(customer_file);
c.id := 3;
c.name := 'C company';
c.phone := '3333-3333';
c.status := ACTUAL;
c.address1 := 'C road 3';
c.address2 := 'C town';
c.contact := 'Mr. C';
c.discount := 10;
customer_file^ := c;
put(customer_file);
c.id := 4;
c.name := 'D company';
c.phone := '4444-4444';
c.status := ACTUAL;
c.address1 := 'D road 4';
c.address2 := 'D town';
c.contact := 'Mr. D';
c.discount := 20;
customer_file^ := c;
put(customer_file);
c.id := 5;
c.name := 'E company';
c.phone := '5555-5555';
c.status := ACTUAL;
c.address1 := 'E road 5';
c.address2 := 'E town';
c.contact := 'Mr. E';
c.discount := 10;
customer_file^ := c;
put(customer_file);
close(customer_file);
open(order_file, 'order.isq', new, organization := indexed, access_method := keyed);
rewrite(order_file);
o.id := 1;
o.customer := 3;
o.status := 'Delivered';
o.nlines := 1;
o.line[1].item := 'X stuff';
o.line[1].quantity := 1;
o.line[1].price := todecimal(7500);
order_file^ := o;
put(order_file);
o.id := 2;
o.customer := 4;
o.status := 'Delivered';
o.nlines := 1;
o.line[1].item := 'Y stuff';
o.line[1].quantity := 10;
o.line[1].price := todecimal(4000);
order_file^ := o;
put(order_file);
o.id := 3;
o.customer := 4;
o.status := 'Delivered';
o.nlines := 1;
o.line[1].item := 'Y stuff';
o.line[1].quantity := 5;
o.line[1].price := todecimal(4000);
order_file^ := o;
put(order_file);
o.id := 4;
o.customer := 4;
o.status := 'Delivered';
o.nlines := 1;
o.line[1].item := 'Y stuff';
o.line[1].quantity := 20;
o.line[1].price := todecimal(4000);
order_file^ := o;
put(order_file);
o.id := 5;
o.customer := 5;
o.status := 'Delivered';
o.nlines := 3;
o.line[1].item := 'X stuff';
o.line[1].quantity := 1;
o.line[1].price := todecimal(7200);
o.line[2].item := 'Y stuff';
o.line[2].quantity := 1;
o.line[2].price := todecimal(4500);
o.line[3].item := 'Z stuff';
o.line[3].quantity := 1;
o.line[3].price := todecimal(9000);
order_file^ := o;
put(order_file);
close(order_file);
end.
Build and run:
$ pas load
$ link load
$ run load
identification division.
program-id.load.
environment division.
input-output section.
file-control.
select optional customer-file assign to "customer.isq"
organization is indexed
access mode is dynamic
record key is customer-id.
select optional order-file assign to "order.isq"
organization is indexed
access mode is dynamic
record key is order-id
alternate record key is order-customer-id with duplicates.
data division.
file section.
copy "customer.cob".
copy "order.cob".
working-storage section.
01 POTENTIAL pic s9(8) comp value 1.
01 ACTUAL pic s9(8) comp value 2.
procedure division.
main-paragraph.
open i-o customer-file
move 1 to customer-id
move "A company" to customer-name
move "1111-1111" to customer-phone
move POTENTIAL to customer-status
move "Sleazy information seller" to customer-source
perform insert-customer-paragraph
move 2 to customer-id
move "B company" to customer-name
move "2222-2222" to customer-phone
move POTENTIAL to customer-status
move "Sleazy information seller" to customer-source
perform insert-customer-paragraph
move 3 to customer-id
move "C company" to customer-name
move "3333-3333" to customer-phone
move ACTUAL to customer-status
move "C road 3" to customer-address1
move "C town" to customer-address2
move "Mr. C" to customer-contact
move 10 to customer-discount
perform insert-customer-paragraph
move 4 to customer-id
move "D company" to customer-name
move "4444-4444" to customer-phone
move ACTUAL to customer-status
move "D road 4" to customer-address1
move "D town" to customer-address2
move "Mr. D" to customer-contact
move 20 to customer-discount
perform insert-customer-paragraph
move 5 to customer-id
move "E company" to customer-name
move "5555-5555" to customer-phone
move ACTUAL to customer-status
move "E road 5" to customer-address1
move "E town" to customer-address2
move "Mr. E" to customer-contact
move 10 to customer-discount
perform insert-customer-paragraph
close customer-file
open i-o order-file
move 1 to order-id
move 3 to order-customer-id
move "Delivered" to order-status
move 1 to order-nlines
move "X stuff" to order-line-item(1)
move 1 to order-line-quantity(1)
move 72.00 to order-line-price(1)
perform insert-order-paragraph
move 2 to order-id
move 4 to order-customer-id
move "Delivered" to order-status
move 1 to order-nlines
move "Y stuff" to order-line-item(1)
move 10 to order-line-quantity(1)
move 40.00 to order-line-price(1)
perform insert-order-paragraph
move 3 to order-id
move 4 to order-customer-id
move "Delivered" to order-status
move 1 to order-nlines
move "Y stuff" to order-line-item(1)
move 5 to order-line-quantity(1)
move 40.00 to order-line-price(1)
perform insert-order-paragraph
move 4 to order-id
move 4 to order-customer-id
move "Delivered" to order-status
move 1 to order-nlines
move "Y stuff" to order-line-item(1)
move 20 to order-line-quantity(1)
move 40.00 to order-line-price(1)
perform insert-order-paragraph
move 5 to order-id
move 5 to order-customer-id
move "Delivered" to order-status
move 3 to order-nlines
move "X stuff" to order-line-item(1)
move 1 to order-line-quantity(1)
move 72.00 to order-line-price(1)
move "Y stuff" to order-line-item(2)
move 1 to order-line-quantity(2)
move 45.00 to order-line-price(2)
move "Z stuff" to order-line-item(3)
move 1 to order-line-quantity(3)
move 90.00 to order-line-price(3)
perform insert-order-paragraph
close order-file
stop run.
insert-customer-paragraph.
write customer-record
invalid key display "Error writing customer"
not invalid key continue
end-write.
insert-order-paragraph.
write order-record
invalid key display "Error writing order"
not invalid key continue
end-write.
Build and run:
$ cob load
$ link load
$ run load
Pascal and Cobol has builtin support for index-sequential files. The support in Pascal is VMS specific. The support in Cobol is standard.
Add code:
program add(input,output);
%include 'util.pas'
%include 'customer.pas'
%include 'order.pas'
var
c : customer;
o : order;
begin
open(customer_file, 'customer.isq', old, organization := indexed, access_method := keyed);
reset(customer_file);
c.id := 6;
c.name := 'F company';
c.phone := '6666-6666';
c.status := ACTUAL;
c.address1 := 'F road 6';
c.address2 := 'F town';
c.contact := 'Mr. F';
c.discount := 10;
customer_file^ := c;
put(customer_file);
close(customer_file);
open(order_file, 'order.isq', old, organization := indexed, access_method := keyed);
reset(order_file);
o.id := 6;
o.customer := 6;
o.status := 'In progress';
o.nlines := 1;
o.line[1].item := 'X stuff';
o.line[1].quantity := 1;
o.line[1].price := todecimal(7200);
order_file^ := o;
put(order_file);
close(order_file);
end.
Build and run:
$ pas add
$ link add
$ run add
identification division.
program-id.myadd.
environment division.
input-output section.
file-control.
select optional customer-file assign to "customer.isq"
organization is indexed
access mode is dynamic
record key is customer-id.
select optional order-file assign to "order.isq"
organization is indexed
access mode is dynamic
record key is order-id
alternate record key is order-customer-id with duplicates.
data division.
file section.
copy "customer.cob".
copy "order.cob".
working-storage section.
01 POTENTIAL pic s9(8) comp value 1.
01 ACTUAL pic s9(8) comp value 2.
procedure division.
main-paragraph.
open i-o customer-file
move 6 to customer-id
move "F company" to customer-name
move "6666-6666" to customer-phone
move ACTUAL to customer-status
move "F road 6" to customer-address1
move "F town" to customer-address2
move "Mr. F" to customer-contact
move 10 to customer-discount
perform insert-customer-paragraph
close customer-file
open i-o order-file
move 6 to order-id
move 6 to order-customer-id
move "In progress" to order-status
move 1 to order-nlines
move "X stuff" to order-line-item(1)
move 1 to order-line-quantity(1)
move 72.00 to order-line-price(1)
perform insert-order-paragraph
close order-file
stop run.
insert-customer-paragraph.
write customer-record
invalid key display "Error writing customer"
not invalid key continue
end-write.
insert-order-paragraph.
write order-record
invalid key display "Error writing order"
not invalid key continue
end-write.
Build and run:
$ cob add
$ link add
$ run add
List code:
program list(input,output);
%include 'util.pas'
%include 'customer.pas'
%include 'order.pas'
var
c : customer;
o : order;
sum, i : integer;
begin
open(customer_file, 'customer.isq', old, organization := indexed, access_method := keyed);
reset(customer_file);
open(order_file, 'order.isq', old, organization := indexed, access_method := keyed);
reset(order_file);
while not eof(customer_file) do begin
c := customer_file^;
if c.status = POTENTIAL then begin
write('**future customer** ');
write(trim(c.name));
write(', ');
write(trim(c.phone));
write(', ');
write(trim(c.source));
writeln;
end else if c.status = ACTUAL then begin
write(c.id:1);
write(', ');
write(trim(c.name));
write(', ');
write(trim(c.address1));
write(', ');
write(trim(c.address2));
write(', ');
write(trim(c.phone));
writeln;
findk(order_file, 1, c.id);
while (not eof(order_file)) and (not ufb(order_file)) and (order_file^.customer = c.id) do begin
o := order_file^;
sum := 0;
for i := 1 to o.nlines do begin
sum := sum + o.line[i].quantity * fromdecimal(o.line[i].price);
end;
write(' ');
write(o.id:1);
write(', ');
write(trim(o.status));
write(', ');
write((sum/100):1:2);
writeln;
get(order_file);
end;
end;
get(customer_file);
end;
close(order_file);
close(customer_file);
end.
Build and run:
$ pas list
$ link list
$ run list
identification division.
program-id.list.
environment division.
input-output section.
file-control.
select optional customer-file assign to "customer.isq"
organization is indexed
access mode is dynamic
record key is customer-id.
select optional order-file assign to "order.isq"
organization is indexed
access mode is dynamic
record key is order-id
alternate record key is order-customer-id with duplicates.
data division.
file section.
copy "customer.cob".
copy "order.cob".
working-storage section.
01 POTENTIAL pic s9(8) comp value 1.
01 ACTUAL pic s9(8) comp value 2.
01 eof-flag pic x(1).
01 done-flag pic x(1).
01 len pic s9(8) comp.
01 name-len pic s9(8) comp.
01 phone-len pic s9(8) comp.
01 source-len pic s9(8) comp.
01 address1-len pic s9(8) comp.
01 address2-len pic s9(8) comp.
01 status-len pic s9(8) comp.
01 id2 pic s9(8) display.
01 order-sum pic 9(5)v9(2) packed-decimal.
01 order-sum2 pic 9(5)v9(2) display.
01 i pic s9(8) comp.
procedure division.
main-paragraph.
open i-o customer-file
open i-o order-file
move "N" to eof-flag
perform until eof-flag = "Y"
read customer-file next
at end move "Y" to eof-flag
not at end perform display-customer-paragraph
end-read
end-perform
close customer-file
close order-file
stop run.
display-customer-paragraph.
if customer-status = POTENTIAL
move 32 to len
call "trim" using customer-name, len giving name-len
move 16 to len
call "trim" using customer-phone, len giving phone-len
move 256 to len
call "trim" using customer-source, len giving source-len
display "**future customer** " customer-name(1:name-len) ", " customer-phone(1:phone-len) ", " customer-source(1:source-len)
end-if
if customer-status = ACTUAL
move customer-id to id2
move 32 to len
call "trim" using customer-name, len giving name-len
move 64 to len
call "trim" using customer-address1, len giving address1-len
move 64 to len
call "trim" using customer-address2, len giving address2-len
move 16 to len
call "trim" using customer-phone, len giving phone-len
display id2 ", " customer-name(1:name-len) ", " customer-address1(1:address1-len) ", " customer-address2(1:address2-len) ", " customer-phone(1:phone-len)
perform load-orders-paragraph
end-if.
load-orders-paragraph.
move customer-id to order-customer-id
start order-file key is greater than or equal to order-customer-id
invalid key display "Error searching orders"
not invalid key continue
end-start
move 'N' to eof-flag
move 'N' to done-flag
perform until eof-flag = 'Y' or done-flag = 'Y'
read order-file next
at end move 'Y' to eof-flag
not at end perform display-order-paragraph
end-read
end-perform.
display-order-paragraph.
if order-customer-id = customer-id then
move order-id to id2
move 16 to len
call "trim" using order-status, len giving status-len
move 0.00 to order-sum
perform varying i from 1 by 1 until i > 10
compute order-sum = order-sum + order-line-quantity(i) * order-line-price(i)
end-perform
move order-sum to order-sum2
display id2 ", " order-status(1:status-len) ", " order-sum2
else
move 'Y' to done-flag
end-if.
Build and run:
$ cob trim
$ cob list
$ link list + trim
$ run list
Add code:
import java.math.BigDecimal;
import dk.vajhoej.isam.IsamException;
import dk.vajhoej.isam.IsamSource;
import dk.vajhoej.isam.local.LocalIsamSource;
import dk.vajhoej.record.RecordException;
public class Add {
public static void main(String[] args) throws IsamException, RecordException {
IsamSource cisam = new LocalIsamSource("customer.isq", "dk.vajhoej.vms.rms.IndexSequential", false);
IsamSource oisam = new LocalIsamSource("order.isq", "dk.vajhoej.vms.rms.IndexSequential", false);
CustomerActual c = new CustomerActual();
c.setId(7);
c.setName("G company");
c.setPhone("7777-7777");
c.setStatus(Customer.ACTUAL);
c.setAddress1("G road 7");
c.setAddress2("G town");
c.setContact("Mr. G");
cisam.create(c);
Order o = new Order();
o.setId(7);
o.setCustomer(7);
o.setStatus("In progress");
o.setNlines(1);
OrderLine ol = new OrderLine();
ol.setItem("X stuff");
ol.setQuantity(1);
ol.setPrice(new BigDecimal("72.00"));
o.getLine()[0] = ol;
for(int i = 1; i < 10; i++) o.getLine()[i] = new OrderLine();
oisam.create(o);
cisam.close();
oisam.close();
}
}
Build and run:
$ javac -classpath .:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar Add.java
$ java -classpath .:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar Add
List code:
import java.math.BigDecimal;
import dk.vajhoej.isam.IsamException;
import dk.vajhoej.isam.IsamResult;
import dk.vajhoej.isam.IsamSource;
import dk.vajhoej.isam.Key;
import dk.vajhoej.isam.local.LocalIsamSource;
import dk.vajhoej.record.RecordException;
public class List {
public static void main(String[] args) throws IsamException, RecordException {
IsamSource cisam = new LocalIsamSource("customer.isq", "dk.vajhoej.vms.rms.IndexSequential", false);
IsamSource oisam = new LocalIsamSource("order.isq", "dk.vajhoej.vms.rms.IndexSequential", false);
IsamResult<Customer> crs = cisam.readStart(Customer.class);
while(crs.read()) {
Customer c = crs.current();
if(c instanceof CustomerPotential) {
CustomerPotential cp = (CustomerPotential)c;
System.out.printf("**future customer** %s, %s, %s\n", cp.getName(), cp.getPhone(), cp.getSource());
}
if(c instanceof CustomerActual) {
CustomerActual ca = (CustomerActual)c;
System.out.printf("%d, %s, %s, %s, %s\n", ca.getId(), ca.getName(), ca.getAddress1(), ca.getAddress2(), ca.getPhone());
IsamResult<Order> ors = oisam.readGE(Order.class, new Key<Integer>(1, ca.getId()));
while(ors.read()) {
Order o = ors.current();
if(o.getCustomer() > ca.getId()) break;
BigDecimal sum = new BigDecimal(0);
for(int i = 0; i < o.getNlines(); i++) {
sum = sum.add(o.getLine()[i].getPrice().multiply(new BigDecimal(o.getLine()[i].getQuantity())));
}
System.out.printf(" %d, %s, %.2f\n", o.getId(), o.getStatus(), sum);
}
}
}
cisam.close();
oisam.close();
}
}
Build and run:
$ javac -classpath .:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar List.java
$ java -classpath .:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar List
This is basically just writing the Java code in Python or Groovy.
Add code:
from java.math import BigDecimal
from dk.vajhoej.isam.local import LocalIsamSource
import Customer
import CustomerActual
import Order
import OrderLine
cisam = LocalIsamSource('customer.isq', 'dk.vajhoej.vms.rms.IndexSequential', False)
oisam = LocalIsamSource('order.isq', 'dk.vajhoej.vms.rms.IndexSequential', False)
c = CustomerActual()
c.id = 8
c.name = 'H company'
c.phone = '8888-8888'
c.status = Customer.ACTUAL
c.address1 = 'H road 8'
c.address2 = 'H town'
c.contact = 'Mr. H'
cisam.create(c)
o = Order()
o.id = 8
o.customer = 8
o.status = 'In progress'
o.nlines = 1
ol = OrderLine()
ol.item = 'X stuff'
ol.quantity = 1
ol.price = BigDecimal('72.00')
o.line[0] = ol
for i in range(9):
o.line[1 + i] = OrderLine()
oisam.create(o)
cisam.close()
oisam.close()
Run:
$ define/nolog jython_libs "/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ jython add.py
import dk.vajhoej.isam.local.LocalIsamSource
cisam = new LocalIsamSource("customer.isq", "dk.vajhoej.vms.rms.IndexSequential", false)
oisam = new LocalIsamSource("order.isq", "dk.vajhoej.vms.rms.IndexSequential", false)
c = new CustomerActual()
c.id = 8
c.name = "H company"
c.phone = "8888-8888"
c.status = Customer.ACTUAL
c.address1 = "H road 8"
c.address2 = "H town"
c.contact = "Mr. H"
cisam.create(c)
o = new Order()
o.id = 8
o.customer = 8
o.status = "In progress"
o.nlines = 1
ol = new OrderLine()
ol.item = "X stuff"
ol.quantity = 1
ol.price = 72.00
o.line[0] = ol
for(i in 1..9) o.line[i] = new OrderLine()
oisam.create(o)
cisam.close()
oisam.close()
Run:
$ groovy_cp == ".:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ groovy add.groovy
List code:
from java.lang import Integer
from java.math import BigDecimal
from dk.vajhoej.isam import Key
from dk.vajhoej.isam.local import LocalIsamSource
import Customer
import Order
import CustomerPotential
import CustomerActual
cisam = LocalIsamSource('customer.isq', 'dk.vajhoej.vms.rms.IndexSequential', False)
oisam = LocalIsamSource('order.isq', 'dk.vajhoej.vms.rms.IndexSequential', False)
crs = cisam.readStart(Customer)
while crs.read():
c = crs.current()
if isinstance(c, CustomerPotential):
print('**future customer** %s, %s, %s' % (c.name, c.phone, c.source))
if isinstance(c, CustomerActual):
print('%d, %s, %s, %s, %s' % (c.id, c.address1, c.address2, c.name, c.phone))
ors = oisam.readGE(Order, Key(1, Integer(c.id)))
while ors.read():
o = ors.current()
if o.customer > c.id:
break
sum = BigDecimal(0)
for i in range(o.nlines):
sum = sum.add(o.line[i].price.multiply(BigDecimal(o.line[i].quantity)))
print(' %d, %s, %s' % (o.id, o.status, sum))
ors.close();
crs.close()
cisam.close()
oisam.close()
Run:
$ define/nolog jython_libs "/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ jython list.py
import dk.vajhoej.isam.Key
import dk.vajhoej.isam.local.LocalIsamSource
cisam = new LocalIsamSource("customer.isq", "dk.vajhoej.vms.rms.IndexSequential", false)
oisam = new LocalIsamSource("order.isq", "dk.vajhoej.vms.rms.IndexSequential", false)
crs = cisam.readStart(Customer.class)
while(crs.read()) {
c = crs.current()
if(c instanceof CustomerPotential) {
println(String.format("**future customer** %s, %s, %s", c.name, c.phone, c.source))
}
if(c instanceof CustomerActual) {
println(String.format("%d, %s, %s, %s, %s", c.id, c.address1, c.address2, c.name, c.phone))
ors = oisam.readGE(Order.class, new Key(1, c.id));
while(ors.read()) {
o = ors.current()
if(o.customer > c.id) break;
sum = 0.00
for(i in 0..o.nlines-1) {
sum += o.line[i].quantity * o.line[i].price
}
println(String.format(" %d, %s, %.2f", o.id, o.status, sum))
}
ors.close()
}
}
crs.close()
cisam.close()
oisam.close()
Run:
$ groovy_cp == ".:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ groovy list.groovy
This is utilizing the map view on top of the basic ISAM library.
Add code:
from java.math import BigDecimal
from dk.vajhoej.isam.map import PyIsamMap
import Customer
import CustomerActual
import Order
cmap = PyIsamMap.createIsamMapRMS('customer.isq', CustomerActual)
cmap.putDict({'id': 9, \
'name': 'I company', \
'phone': '9999-9999',
'status': Customer.ACTUAL, \
'address1': 'I road 9', \
'address2': 'I town', \
'contact': 'Mr. I'})
omap = PyIsamMap.createIsamMapRMS('order.isq', Order)
omap.putDict({'id': 9, \
'customer': 9, \
'status': 'In progress', \
'nlines': 1, \
'line': [{'item': 'X stuff', 'quantity': 1, 'price': BigDecimal('72.00')}]})
Run:
$ define/nolog jython_libs "/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ jython addx.py
import dk.vajhoej.isam.map.IsamMap
cmap = IsamMap.createIsamMapRMS("customer.isq", Customer)
c = new CustomerActual(id: 9,
name: "I company",
phone: "9999-9999",
status: Customer.ACTUAL,
address1: "I road 9",
address2: "I town",
contact: "Mr. I")
cmap[c.id] = c
omap = IsamMap.createIsamMapRMS("order.isq", Order)
o = new Order(id: 9,
customer: 9,
status: "In progress",
nlines: 1)
o.line[0] = new OrderLine(item: "X stuff", quantity: 1, price: 72.00)
for(i in 1..9) o.line[i] = new OrderLine()
omap[o.id] = o
Run:
$ groovy_cp == ".:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ groovy addx.groovy
List code:
from java.math import BigDecimal
from dk.vajhoej.isam.map import PyIsamMap
import Customer
import Order
import CustomerPotential
import CustomerActual
cmap = PyIsamMap.createIsamMapRMS('customer.isq', Customer)
omap = PyIsamMap.createIsamMapRMS('order.isq', Order)
for c in cmap.values():
if isinstance(c, CustomerPotential):
print('**future customer** %s, %s, %s' % (c.name, c.phone, c.source))
if isinstance(c, CustomerActual):
print('%d, %s, %s, %s, %s' % (c.id, c.address1, c.address2, c.name, c.phone))
for o in omap.keyName('customer').isInt(c.id).values():
sum = BigDecimal(0)
for i in range(o.nlines):
sum = sum.add(o.line[i].price.multiply(BigDecimal(o.line[i].quantity)))
print(' %d, %s, %s' % (o.id, o.status, sum))
Run:
$ define/nolog jython_libs "/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ jython listx.py
import dk.vajhoej.isam.map.IsamMap
cmap = IsamMap.createIsamMapRMS("customer.isq", Customer)
omap = IsamMap.createIsamMapRMS("order.isq", Order)
for(c in cmap.values()) {
if(c instanceof CustomerPotential) {
println(String.format("**future customer** %s, %s, %s", c.name, c.phone, c.source))
}
if(c instanceof CustomerActual) {
println(String.format("%d, %s, %s, %s, %s", c.id, c.address1, c.address2, c.name, c.phone))
for(o in omap.key(1).is(c.id).values()) {
sum = 0.00
for(i in 0..o.nlines-1) {
sum += o.line[i].quantity * o.line[i].price
}
println(String.format(" %d, %s, %.2f", o.id, o.status, sum))
}
}
}
Run:
$ groovy_cp == ".:/isamdir/isam.jar:/isamdir/isam-vms.jar:/isamdir/record.jar
$ groovy listx.groovy
| Language & style | Lines for add | Lines for list | Lines total |
|---|---|---|---|
| Cobol | 57 | 101 | 158 |
| Pascal | 38 | 60 | 98 |
| Java | 36 | 39 | 75 |
| Jython - DB style | 35/ | 33 | 68 |
| Jython - map style | 22 | 21 | 43 |
| Groovy - DB style | 27 | 29 | 56 |
| Groovy - map style | 19 | 19 | 38 |
We see as expected that the script languages using the map interface instead of the database interface require much less code to implement the same functionality.
Obviously the number of lines depends on code formatting style, but I hope and believe I have chosen a reasonable common code formatting style. The size of the Cobol code has some additional uncertainty because I am not experienced with Cobol. But even with those caveats then I feel comfortable that the difference in code sizes is real.
Being able to write so short code in such a flexible way does not come for free - it comes with a large overhead.
There is a very large startup overhead getting JVM started, classes loaded, class structures analyzed etc..
There is a large overhead processing records due to reflection access to objects, JNI calls to RMS (yes - a JNI call adds lots of overhead) and explicit iteration over fields.
We will test how big this overhead is.
New data structure with more fields:
type
perf = record
id : [key(0)] integer;
iv1 : integer;
iv2 : integer;
iv3 : integer;
iv4 : integer;
iv5 : integer;
sv1 : packed array[1..64] of char;
sv2 : packed array[1..64] of char;
sv3 : packed array[1..64] of char;
sv4 : packed array[1..64] of char;
sv5 : packed array[1..64] of char;
end;
var
perf_file : file of perf;
import dk.vajhoej.isam.KeyField;
import dk.vajhoej.record.FieldType;
import dk.vajhoej.record.Struct;
import dk.vajhoej.record.StructField;
@lombok.Getter @lombok.Setter
@Struct
public class Perf {
@KeyField(n=0)
@StructField(n=0, type=FieldType.INT4)
private int id;
@StructField(n=1, type=FieldType.INT4)
private int iv1;
@StructField(n=2, type=FieldType.INT4)
private int iv2;
@StructField(n=3, type=FieldType.INT4)
private int iv3;
@StructField(n=4, type=FieldType.INT4)
private int iv4;
@StructField(n=5, type=FieldType.INT4)
private int iv5;
@StructField(n=6, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
private String sv1;
@StructField(n=7, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
private String sv2;
@StructField(n=8, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
private String sv3;
@StructField(n=9, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
private String sv4;
@StructField(n=10, type=FieldType.FIXSTR, length=64, pad=true, padchar=' ')
private String sv5;
}
We create a bigger data file with 100000 records:
program loadperf(input,output);
%include 'util.pas'
%include 'perf.pas'
var
p : perf;
i : integer;
begin
open(perf_file, 'perf.isq', new, organization := indexed, access_method := keyed);
rewrite(perf_file);
for i := 1 to 100000 do begin
p.id := i;
p.iv1 := i + 1;
p.iv2 := i + 2;
p.iv3 := i + 3;
p.iv4 := i + 4;
p.iv5 := i + 5;
p.sv1 := 'Text 1 - #' + dec(i, 6);
p.sv2 := 'Text 2 - #' + dec(i, 6);
p.sv3 := 'Text 3 - #' + dec(i, 6);
p.sv4 := 'Text 4 - #' + dec(i, 6);
p.sv5 := 'Text 5 - #' + dec(i, 6);
perf_file^ := p;
put(perf_file);
end;
close(perf_file);
end.
And then we will measure query speed. We will do 100000 queries each fetching 10 consecutive records at a random location in the file.
[inherit('sys$library:pascal$lib_routines')]
program listperf(input,output);
%include 'util.pas'
%include 'perf.pas'
var
p : perf;
cmdlin : pstr;
lim, i, ix, n : integer;
begin
lib$get_foreign(resultant_string := cmdlin.body, resultant_length := cmdlin.length);
readv(cmdlin, lim);
open(perf_file, 'perf.isq', old, organization := indexed, access_method := keyed);
reset(perf_file);
seed(clock);
n := 0;
for i := 1 to lim do begin
ix := 1 + trunc(99990 * random);
findk(perf_file, 0, ix);
while not(ufb(perf_file)) and (perf_file^.id < ix + 10) do begin
p := perf_file^;
n := n + 1;
get(perf_file);
end;
end;
writeln(n:1);
close(perf_file);
end.
import java.math.BigDecimal;
import java.util.Random;
import dk.vajhoej.isam.IsamException;
import dk.vajhoej.isam.IsamResult;
import dk.vajhoej.isam.IsamSource;
import dk.vajhoej.isam.Key0;
import dk.vajhoej.isam.local.LocalIsamSource;
import dk.vajhoej.record.RecordException;
public class ListPerf {
public static void main(String[] args) throws IsamException, RecordException {
int lim = Integer.parseInt(args[0]);
IsamSource pisam = new LocalIsamSource("perf.isq", "dk.vajhoej.vms.rms.IndexSequential", false);
Random rng = new Random();
int n = 0;
for(int i = 0; i < lim; i++) {
int ix = 1 + rng.nextInt(99990);
IsamResult<Perf> prs = pisam.readGE(Perf.class, new Key0<Integer>(ix));
while(prs.read()) {
Perf p = prs.current();
if(p.getId() >= ix + 10) break;
n++;
}
prs.close();
}
System.out.println(n);
pisam.close();
}
}
import sys
from java.util import Random
from dk.vajhoej.isam.map import PyIsamMap
import Perf
lim = int(sys.argv[1])
pmap = PyIsamMap.createIsamMapRMS('perf.isq', Perf)
rng = Random()
n = 0
for i in range(lim):
ix = 1 + rng.nextInt(99990)
for p in pmap.keyName('id').betweenInt(ix, ix + 9).values():
n = n + 1
print(n)
import java.util.Random
import dk.vajhoej.isam.map.IsamMap
lim = Integer.parseInt(args[0])
pmap = IsamMap.createIsamMapRMS("perf.isq", Perf)
rng = new Random()
n = 0
for(j in 1..lim) {
ix = 1 + rng.nextInt(99990)
for(p in pmap.key().between(ix, ix + 9).values()) {
n++
}
}
println(n)
Results:
| Language & style | Startup time | Query time |
|---|---|---|
| Pascal | 0 s | 0.139 ms |
| Java | 2 s | 0.842 ms |
| Jython - map style | 19 s | 1.121 ms |
| Groovy - map style | 26 s | 1.093 ms |
We see there is a factor 6-8 overhead of using Java or a JVM script languages (for the queries - on top of that comes startup overhead).
The reason for the relative small difference between Java and Jython/Groovy for queries is that they both call the same Java library to do the actual work - the difference is only in the orchestration of the library calls.
That is a really huge overhead. If you intend to do many queries per second then the scripting solution is not a good solution for you. If you intend to do a few queries per minute or some queries per hour, then the speed of the scripting language is really not a problem.
This is another tool in the toolbox worth considering when planning roadmap for your VMS application using index-sequential files.
| Version | Date | Description |
|---|---|---|
| 1.0 | October 21st 2023 | Initial version |
See list of all articles here
Please send comments to Arne Vajhøj