Apache Camel was a bit hot in the Java world 10-15 years ago - not quite so much today.
But it is still an interesting framework.
And it runs fine on VMS - at least VMS Itanium and VMS x86-64 where Java 8 is available (VMS Alpha with Java 5 is a problem).
Note that there is really nothing here that is truly VMS specific - it is just stuff that also run on VMS.
Very basic Java skills are required as the code shown is in Java. But most of it is really a Camel DSL.
Apache Camel is a routing framework.
It consists of:
Camel comes with a ton of builtin components - Camel can communicate with almost anything out of the box.
But if that is not sufficient, then it is possible to add ones own components.
The example show will again focus on moving data around in different formats. It may be considered trivial, but it is an important part of a lot of business computing.
The fisrt part is somewhat similar in functionality to VMS Tech Demo 22 - data import with Spring Batch. The difference is that Spring Batch is very focused on high performance production with error handling while Camel is more focused on being fast to develop, flexible and easy to modify to support new flows. Camel is great for PoC's and adhoc solutions.
Camel require a lot of jar files. My demos were run with the following:
awaitility-4_1_1.jar;1 c3p0-0_9_5_5.jar;1 camel-api-3_14_10.jar;1 camel-attachments-3_14_10.jar;1 camel-base-3_14_10.jar;1 camel-base-engine-3_14_10.jar;1 camel-bean-3_14_10.jar;1 camel-bindy-3_14_10.jar;1 camel-browse-3_14_10.jar;1 camel-cloud-3_14_10.jar;1 camel-cluster-3_14_10.jar;1 camel-controlbus-3_14_10.jar;1 camel-core-3_14_10.jar;1 camel-core-catalog-3_14_10.jar;1 camel-core-engine-3_14_10.jar;1 camel-core-languages-3_14_10.jar;1 camel-core-model-3_14_10.jar;1 camel-core-processor-3_14_10.jar;1 camel-core-reifier-3_14_10.jar;1 camel-core-xml-3_14_10.jar;1 camel-csv-3_14_10.jar;1 camel-dataformat-3_14_10.jar;1 camel-dataset-3_14_10.jar;1 camel-direct-3_14_10.jar;1 camel-directvm-3_14_10.jar;1 camel-file-3_14_10.jar;1 camel-ftp-3_14_10.jar;1 camel-groovy-3_14_10.jar;1 camel-health-3_14_10.jar;1 camel-http-3_14_10.jar;1 camel-http-base-3_14_10.jar;1 camel-http-common-3_14_10.jar;1 camel-jackson-3_14_10.jar;1 camel-jacksonxml-3_14_10.jar;1 camel-jaxb-3_14_10.jar;1 camel-jdbc-3_14_10.jar;1 camel-jetty-3_14_10.jar;1 camel-jetty-common-3_14_10.jar;1 camel-jms-3_14_10.jar;1 camel-jpa-3_14_10.jar;1 camel-language-3_14_10.jar;1 camel-log-3_14_10.jar;1 camel-mail-3_14_10.jar;1 camel-management-api-3_14_10.jar;1 camel-mock-3_14_10.jar;1 camel-netty-3_14_10.jar;1 camel-netty-http-3_14_10.jar;1 camel-quartz-3_14_10.jar;1 camel-ref-3_14_10.jar;1 camel-rest-3_14_10.jar;1 camel-saga-3_14_10.jar;1 camel-scheduler-3_14_10.jar;1 camel-seda-3_14_10.jar;1 camel-spring-3_14_10.jar;1 camel-sql-3_14_10.jar;1 camel-stream-3_14_10.jar;1 camel-stub-3_14_10.jar;1 camel-support-3_14_10.jar;1 camel-timer-3_14_10.jar;1 camel-tooling-model-3_14_10.jar;1 camel-util-3_14_10.jar;1 camel-util-json-3_14_10.jar;1 camel-validator-3_14_10.jar;1 camel-vm-3_14_10.jar;1 camel-xml-io-util-3_14_10.jar;1 camel-xml-jaxb-3_14_10.jar;1 camel-xml-jaxp-3_14_10.jar;1 camel-xpath-3_14_10.jar;1 camel-xslt-3_14_10.jar;1 commons-codec-1_11.jar;1 commons-csv-1_8.jar;1 commons-logging-1_2.jar;1 commons-net-3_8_0.jar;1 commons-pool2-2_11_1.jar;1 geronimo-jms_2_0_spec-1_0-alpha-2.jar;1 groovy-3_0_17.jar;1 hamcrest-2_1.jar;1 HikariCP-java7-2_4_13.jar;1 httpclient-4_5_14.jar;1 httpcore-4_4_16.jar;1 jackson-annotations-2_12_7.jar;1 jackson-core-2_12_7.jar;1 jackson-databind-2_12_7.jar;1 jackson-dataformat-xml-2_12_7.jar;1 jackson-module-jaxb-annotations-2_12_7.jar;1 jakarta_activation-1_2_2.jar;1 jakarta_activation-api-1_2_2.jar;1 jakarta_mail-1_6_7.jar;1 jakarta_xml_bind-api-2_3_3.jar;1 javax_activation-1_2_0.jar;1 javax_servlet-api-3_1_0.jar;1 jaxb-core-2_3_0.jar;1 jaxb-impl-2_3_3.jar;1 jetty-client-9_4_51_v20230217.jar;1 jetty-continuation-9_4_51_v20230217.jar;1 jetty-http-9_4_51_v20230217.jar;1 jetty-io-9_4_51_v20230217.jar;1 jetty-jmx-9_4_51_v20230217.jar;1 jetty-security-9_4_51_v20230217.jar;1 jetty-server-9_4_51_v20230217.jar;1 jetty-servlet-9_4_51_v20230217.jar;1 jetty-servlets-9_4_51_v20230217.jar;1 jetty-util-9_4_51_v20230217.jar;1 jetty-util-ajax-9_4_51_v20230217.jar;1 jsch-0_2_1.jar;1 mchange-commons-java-0_2_15.jar;1 netty-buffer-4_1_94_Final.jar;1 netty-codec-4_1_94_Final.jar;1 netty-codec-http-4_1_94_Final.jar;1 netty-common-4_1_94_Final.jar;1 netty-handler-4_1_94_Final.jar;1 netty-resolver-4_1_94_Final.jar;1 netty-transport-4_1_94_Final.jar;1 netty-transport-classes-epoll-4_1_94_Final.jar;1 netty-transport-native-epoll-4_1_94_Final.jar;1 netty-transport-native-unix-common-4_1_94_Final.jar;1 quartz-2_3_2.jar;1 slf4j-api-1_7_36.jar;1 slf4j-jdk14-1_7_36.jar;1 spring-aop-5_3_27.jar;1 spring-beans-5_3_27.jar;1 spring-context-5_3_27.jar;1 spring-core-5_3_27.jar;1 spring-expression-5_3_27.jar;1 spring-jcl-5_3_27.jar;1 spring-jdbc-5_3_27.jar;1 spring-jms-5_3_27.jar;1 spring-messaging-5_3_27.jar;1 spring-orm-5_3_27.jar;1 spring-tx-5_3_27.jar;1 stax2-api-4_2_1.jar;1 woodstox-core-6_2_4.jar;1
On top of all the Camel jars I also used mysql-connector-j-8_0_33.jar for MySQL connectivity and activemq-all-5.16.7.jar for ActiveMQ connectivity.
Build and run commands:
$ javac -cp lib/*:/javalib/mysql-connector-j-8_0_33.jar:/activemq$root/000000/activemq-all-5.16.7.jar Stepx.java Data.java
$ def/user sys$input sys$command
$ java -cp .:lib/*:/javalib/mysql-connector-j-8_0_33.jar:/activemq$root/000000/activemq-all-5.16.7.jar Stepx
For more info on Camel read here.
Enough describing what Camel is - let us see what Camel is.
The scenario here is that we are a VMS developer being tasked with receiving data from some applications on VMS and stuffing them into a database. That is a relative simple task, but requirements about how data will be delivered are in total flux, which makes it more tricky.
Moving data with huge uncertainty about how data will be coming in makes a great case for Camel!
First version of requirements is that they will deliver CSV files in a specific directory and they want the files loades as soon as they are copied there.
Example of CSV:
in_1.csv:
1,A
2,BB
3,CCC
Data class:
import org.apache.camel.dataformat.bindy.annotation.*;
@CsvRecord(separator=",",crlf="UNIX")
public class Data {
@DataField(pos=1)
private int f1;
@DataField(pos=2)
private String f2;
public int getF1() {
return f1;
}
public void setF1(int f1) {
this.f1 = f1;
}
public String getF2() {
return f2;
}
public void setF2(String f2) {
this.f2 = f2;
}
@Override
public String toString() {
return String.format("(%d,%s)", f1, f2);
}
}
Matching database table:
CREATE TABLE cameldemo(id INTEGER AUTO_INCREMENT PRIMARY KEY,origin VARCHAR(255),f1 INTEGER,f2 VARCHAR(255));
Note: origin column explains where the data comes from.
Step1.java:
import org.apache.camel.*;
import org.apache.camel.impl.*;
import org.apache.camel.builder.*;
import org.apache.camel.spi.*;
import org.apache.camel.model.dataformat.BindyType;
import org.apache.camel.dataformat.bindy.csv.BindyCsvDataFormat;
import com.mysql.cj.jdbc.MysqlDataSource;
public class Step1 {
public static void main(String[] args) throws Exception {
CamelContext ctx = new DefaultCamelContext();
Registry reg = ctx.getRegistry();
reg.bind("arnepc5_mysql", new MysqlDataSource() {
{
setServerName("arnepc5");
setDatabaseName("test");
setUser("arne");
setPassword("hemmeligt");
}
});
ctx.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("file:/disk2/arne/camel/in?include=.*\\.csv")
.setHeader("origin", simple("CSV file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.to("direct:csv");
from("direct:csv")
.unmarshal().bindy(BindyType.Csv, Data.class) // convert from String with CSV to List<Data>
.to("direct:db");
from("direct:db")
.split(body()) // convert from one message of type List<Data> to many messages of type Data
.to("sql:INSERT INTO cameldemo(origin,f1,f2) VALUES (:#${headers.origin},:#${body.f1},:#${body.f2})?dataSource=#arnepc5_mysql");
}
});
ctx.start();
System.out.print("Press enter to exit");
System.in.read();
ctx.stop();
ctx.close();
}
}
But it turns out that they want to support both CSV, JSON and XML input files, so we need to add JSON and XML support.
Example of JSON:
[
{
"f1": 1,
"f2": "A"
},
{
"f1": 2,
"f2": "BB"
},
{
"f1": 3,
"f2": "CCC"
}
]
Example of XML:
<all>
<data>
<f1>1</f1>
<f2>A</f2>
</data>
<data>
<f1>2</f1>
<f2>BB</f2>
</data>
<data>
<f1>3</f1>
<f2>CCC</f2>
</data>
</all>
Step2.java:
import org.apache.camel.*;
import org.apache.camel.impl.*;
import org.apache.camel.builder.*;
import org.apache.camel.spi.*;
import org.apache.camel.model.dataformat.BindyType;
import org.apache.camel.dataformat.bindy.csv.BindyCsvDataFormat;
import org.apache.camel.component.jackson.ListJacksonDataFormat;
import org.apache.camel.component.jacksonxml.ListJacksonXMLDataFormat;
import com.mysql.cj.jdbc.MysqlDataSource;
public class Step2 {
public static void main(String[] args) throws Exception {
CamelContext ctx = new DefaultCamelContext();
Registry reg = ctx.getRegistry();
reg.bind("arnepc5_mysql", new MysqlDataSource() {
{
setServerName("arnepc5");
setDatabaseName("test");
setUser("arne");
setPassword("hemmeligt");
}
});
ctx.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("file:/disk2/arne/camel/in?include=.*\\.csv")
.setHeader("origin", simple("CSV file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.to("direct:csv");
from("file:/disk2/arne/camel/in?include=.*\\.json")
.setHeader("origin", simple("JSON file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.to("direct:json");
from("file:/disk2/arne/camel/in?include=.*\\.xml")
.setHeader("origin", simple("XML file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.to("direct:xml");
from("direct:csv")
.unmarshal().bindy(BindyType.Csv, Data.class) // convert from String with CSV to List<Data>
.to("direct:db");
from("direct:json")
.unmarshal(new ListJacksonDataFormat(Data.class)) // convert from String with JSON to List<Data>
.to("direct:db");
from("direct:xml")
.unmarshal(new ListJacksonXMLDataFormat(Data.class)) // convert from String with XML to List<Data>
.to("direct:db");
from("direct:db")
.split(body()) // convert from one message of type List<Data> to many messages of type Data
.to("sql:INSERT INTO cameldemo(origin,f1,f2) VALUES (:#${headers.origin},:#${body.f1},:#${body.f2})?dataSource=#arnepc5_mysql");
}
});
ctx.start();
System.out.print("Press enter to exit");
System.in.read();
ctx.stop();
ctx.close();
}
}
After that they decide that they want a log of everything for auditing.
Step3.java:
import org.apache.camel.*;
import org.apache.camel.impl.*;
import org.apache.camel.builder.*;
import org.apache.camel.spi.*;
import org.apache.camel.model.dataformat.BindyType;
import org.apache.camel.dataformat.bindy.csv.BindyCsvDataFormat;
import org.apache.camel.component.jackson.ListJacksonDataFormat;
import org.apache.camel.component.jacksonxml.ListJacksonXMLDataFormat;
import com.mysql.cj.jdbc.MysqlDataSource;
public class Step3 {
public static void main(String[] args) throws Exception {
CamelContext ctx = new DefaultCamelContext();
Registry reg = ctx.getRegistry();
reg.bind("arnepc5_mysql", new MysqlDataSource() {
{
setServerName("arnepc5");
setDatabaseName("test");
setUser("arne");
setPassword("hemmeligt");
}
});
ctx.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("file:/disk2/arne/camel/in?include=.*\\.csv")
.setHeader("origin", simple("CSV file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:csv", "direct:log");
from("file:/disk2/arne/camel/in?include=.*\\.json")
.setHeader("origin", simple("JSON file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:json", "direct:log");
from("file:/disk2/arne/camel/in?include=.*\\.xml")
.setHeader("origin", simple("XML file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:xml", "direct:log");
from("direct:csv")
.unmarshal().bindy(BindyType.Csv, Data.class) // convert from String with CSV to List<Data>
.to("direct:db");
from("direct:json")
.unmarshal(new ListJacksonDataFormat(Data.class)) // convert from String with JSON to List<Data>
.to("direct:db");
from("direct:xml")
.unmarshal(new ListJacksonXMLDataFormat(Data.class)) // convert from String with XML to List<Data>
.to("direct:db");
from("direct:db")
.split(body()) // convert from one message of type List<Data> to many messages of type Data
.to("sql:INSERT INTO cameldemo(origin,f1,f2) VALUES (:#${headers.origin},:#${body.f1},:#${body.f2})?dataSource=#arnepc5_mysql");
from("direct:log")
.setBody(simple("**** ${headers.origin}:\n${body}"))
.to("file:/disk2/arne/camel/out?fileName=all.log&fileExist=Append&appendChars=\n");
}
});
ctx.start();
System.out.print("Press enter to exit");
System.in.read();
ctx.stop();
ctx.close();
}
}
Next they come and explain that file transfer is too oldfashioned for one of the newer applications and they want to deliver data via a web service call (JSON/HTTP).
Test program (Python):
import requests
requests.post('http://localhost:8888/data', data='[{"f1":1,"f2":"A"},{"f1":2,"f2":"BB"},{"f1":3,"f2":"CCC"}]')
Step4.java:
import org.apache.camel.*;
import org.apache.camel.impl.*;
import org.apache.camel.builder.*;
import org.apache.camel.spi.*;
import org.apache.camel.model.dataformat.BindyType;
import org.apache.camel.dataformat.bindy.csv.BindyCsvDataFormat;
import org.apache.camel.component.jackson.ListJacksonDataFormat;
import org.apache.camel.component.jacksonxml.ListJacksonXMLDataFormat;
import com.mysql.cj.jdbc.MysqlDataSource;
public class Step4 {
public static void main(String[] args) throws Exception {
CamelContext ctx = new DefaultCamelContext();
Registry reg = ctx.getRegistry();
reg.bind("arnepc5_mysql", new MysqlDataSource() {
{
setServerName("arnepc5");
setDatabaseName("test");
setUser("arne");
setPassword("hemmeligt");
}
});
ctx.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("file:/disk2/arne/camel/in?include=.*\\.csv")
.setHeader("origin", simple("CSV file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:csv", "direct:log");
from("file:/disk2/arne/camel/in?include=.*\\.json")
.setHeader("origin", simple("JSON file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:json", "direct:log");
from("file:/disk2/arne/camel/in?include=.*\\.xml")
.setHeader("origin", simple("XML file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:xml", "direct:log");
from("jetty:http://0.0.0.0:8888/data")
.convertBodyTo(String.class)
.setHeader("origin", simple("HTTP request for path /data processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:json", "direct:log");
from("direct:csv")
.unmarshal().bindy(BindyType.Csv, Data.class) // convert from String with CSV to List<Data>
.to("direct:db");
from("direct:json")
.unmarshal(new ListJacksonDataFormat(Data.class)) // convert from String with JSON to List<Data>
.to("direct:db");
from("direct:xml")
.unmarshal(new ListJacksonXMLDataFormat(Data.class)) // convert from String with XML to List<Data>
.to("direct:db");
from("direct:db")
.split(body()) // convert from one message of type List<Data> to many messages of type Data
.to("sql:INSERT INTO cameldemo(origin,f1,f2) VALUES (:#${headers.origin},:#${body.f1},:#${body.f2})?dataSource=#arnepc5_mysql");
from("direct:log")
.setBody(simple("**** ${headers.origin}:\n${body}"))
.to("file:/disk2/arne/camel/out?fileName=all.log&fileExist=Append&appendChars=\n");
}
});
ctx.start();
System.out.print("Press enter to exit");
System.in.read();
ctx.stop();
ctx.close();
}
}
And finally they come and say that an application on an external system prefer to send data via a message queue (XML format on ActiveMQ).
Test program (Python):
import stomp
con = stomp.Connection([('localhost', 61613)])
con.connect()
con.send(destination='/queue/inQ', body='<all><data><f1>1</f1><f2>A</f2></data><data><f1>2</f1><f2>BB</f2></data><data><f1>3</f1><f2>CCC</f2></data></all>', headers={"persistent": "false", "amq-msg-type": "text"})
con.disconnect()
Step5.java:
import org.apache.camel.*;
import org.apache.camel.impl.*;
import org.apache.camel.builder.*;
import org.apache.camel.spi.*;
import org.apache.camel.model.dataformat.BindyType;
import org.apache.camel.dataformat.bindy.csv.BindyCsvDataFormat;
import org.apache.camel.component.jackson.ListJacksonDataFormat;
import org.apache.camel.component.jacksonxml.ListJacksonXMLDataFormat;
import com.mysql.cj.jdbc.MysqlDataSource;
import org.apache.activemq.ActiveMQConnectionFactory;
public class Step5 {
public static void main(String[] args) throws Exception {
CamelContext ctx = new DefaultCamelContext();
Registry reg = ctx.getRegistry();
reg.bind("arnepc5_mysql", new MysqlDataSource() {
{
setServerName("arnepc5");
setDatabaseName("test");
setUser("arne");
setPassword("hemmeligt");
}
});
reg.bind("arne4_activemq", new ActiveMQConnectionFactory("tcp://localhost:61616"));
ctx.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("file:/disk2/arne/camel/in?include=.*\\.csv")
.setHeader("origin", simple("CSV file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:csv", "direct:log");
from("file:/disk2/arne/camel/in?include=.*\\.json")
.setHeader("origin", simple("JSON file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:json", "direct:log");
from("file:/disk2/arne/camel/in?include=.*\\.xml")
.setHeader("origin", simple("XML file ${headers.CamelFileName} processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:xml", "direct:log");
from("jetty:http://0.0.0.0:8888/data")
.convertBodyTo(String.class)
.setHeader("origin", simple("HTTP request for path /data processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:json", "direct:log");
from("jms:queue:inQ?connectionFactory=#arne4_activemq")
.setHeader("origin", simple("MQ message from queue inQ processed ${date:now:dd-MMM-yyyy HH:mm:ss}"))
.multicast().to("direct:xml", "direct:log");
from("direct:csv")
.unmarshal().bindy(BindyType.Csv, Data.class) // convert from String with CSV to List<Data>
.to("direct:db");
from("direct:json")
.unmarshal(new ListJacksonDataFormat(Data.class)) // convert from String with JSON to List<Data>
.to("direct:db");
from("direct:xml")
.unmarshal(new ListJacksonXMLDataFormat(Data.class)) // convert from String with XML to List<Data>
.to("direct:db");
from("direct:db")
.split(body()) // convert from one message of type List<Data> to many messages of type Data
.to("sql:INSERT INTO cameldemo(origin,f1,f2) VALUES (:#${headers.origin},:#${body.f1},:#${body.f2})?dataSource=#arnepc5_mysql");
from("direct:log")
.setBody(simple("**** ${headers.origin}:\n${body}"))
.to("file:/disk2/arne/camel/out?fileName=all.log&fileExist=Append&appendChars=\n");
}
});
ctx.start();
System.out.print("Press enter to exit");
System.in.read();
ctx.stop();
ctx.close();
}
}
We ended up with:
We can look at lines of code:
| Functionality | Lines of code |
|---|---|
| support CSV files | 43 lines and 25 lines data class |
| add JSON and XML support | +14 lines |
| add log | +3 lines |
| add web service support | +4 lines |
| add message queue support | + 5 lines |
I think we can conclude that Camel is pretty good for changing flows (adding new capablities).
The initial program could easily have been handwritten in a high level language like Python or Groovy in the same number of lines. But I can not see them add all the extra stuff in so few lines.
For a production system that will run for 10-20-30 years, then someone will probably spend time defining a specific requirement for input, insist on that being the only type of input and implement that. Without Camel. But possible with Spring Batch if that is a good fit for the problem.
But then there is all the rest. And for that Camel can do a lot with very few lines of code and are damn easy to adjust to new requirements.
Note that Camel can talk to way more than what is shown here. What is shown here is for the business data transfers that I assume most VMS systems would be doing.
Camel also comes with components to talk to:
That is practically everything!
The downsides are:
| Version | Date | Description |
|---|---|---|
| 1.0 | September 30th 2025 | Initial version |
See list of all articles here
Please send comments to Arne Vajhøj